Electronic device directional audio capture

• US 20100026780A1
• Filed: 07/31/2008
• Published: 02/04/2010
• Est. Priority Date: 07/31/2008
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: M K. Vehvilainen, Nokia Corporation
• Inventors: Hamalainen, Matti S., Tico, Marius, Vehvilainen, Markku K.
• Granted Patent: US 9,445,193 B2
• Time in Patent Office: Days
• US Class Current: 348/14.20
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: H04M 1/6041 Portable telephones adapted for handsfree use
• CPC Class Codes: H04M 2250/52 including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F
• CPC Class Codes: H04R 29/005 Microphone arrays
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

Disclosed herein is an apparatus. The apparatus includes a housing, electronic circuitry, and an audio-visual source tracking system. The electronic circuitry is in the housing. The audio-visual source tracking system includes a first video camera and an array of microphones. The first video camera and the array of microphones are attached to the housing. The audio-visual source tracking system is configured to receive video information from the first video camera. The audio-visual source tracking system is configured to capture audio information from the array of microphones at least partially in response to the video information. The audio-visual source tracking system might include a second video camera that is attached to the housing, wherein the first and second video cameras together estimate the beam orientation of the array of microphones.

Claims

1. An apparatus comprising:

a housing;

electronic circuitry in the housing; and

an audio-visual source tracking system comprising a first video camera and an array of microphones, wherein the first video camera and the array of microphones are attached to the housing, wherein the audio-visual source tracking system is configured to receive video information from the first video camera, and wherein the audio-visual source tracking system is configured to capture audio information from the array of microphones at least partially in response to the video information.

2. The apparatus of claim 1 wherein the array of microphones are proximate the first video camera.

3. The apparatus of claim 1 wherein the array of microphones comprises at least three microphones.

4. The apparatus of claim 3 wherein the audio-visual source tracking system is configured to adjust and direct the sensitivity of the microphone array at least partially in response to the video information.

5. The apparatus of claim 1 wherein the audio-visual source tracking system is configured to receive video information corresponding to a user of the apparatus from the first video camera.

6. The apparatus of claim 1 wherein the apparatus comprises a mobile handset.

7. The apparatus of claim 1 wherein the audio-visual source tracking system is configured to capture a voice of a user of the apparatus at least partially in response to an orientation of the user.

8. The apparatus of claim 1 wherein the first video camera is configured to be oriented toward a user of the apparatus.

9. The apparatus of claim 1 wherein the array of microphones surround the first video camera.

10. The apparatus of claim 1 further comprising a second video camera, wherein the first and the second video cameras are configured to estimate a beam orientation for the array of microphones.

11. A method comprising:

providing a housing;

installing electronic circuitry in the housing; and

providing a tracking system proximate the housing, wherein the tracking system comprises a first video camera and an array of microphones, wherein the first video camera is attached to the housing, wherein the array of microphones is proximate the first video camera, wherein the tracking system is configured to receive a video signal from the first video camera, and wherein the tracking system is configured to capture an audio signal from the microphone at least partially in response to the video signal.

12. The method of claim 11 wherein the array of microphones comprises at least three microphones.

13. The method of claim 12 wherein the providing of the tracking system further comprises providing a face detection system configured to adjust and direct the sensitivity of the array of microphones based at least partially upon a detected face of a user.

14. The method of claim 12 wherein the providing of the tracking system further comprises attaching the array of microphones around the first video camera.

15. The method of claim 11 wherein the housing forms a portion of an apparatus, and wherein the providing of the tracking system further comprises orienting the first video camera toward a user of the apparatus.

16. The method of claim 11 wherein the providing of the tracking system further comprises providing a second video camera, wherein the first and the second video cameras are configured to estimate a beam orientation for the microphone.

17. A method comprising:

capturing a first image with a camera of an apparatus;

determining an orientation of a portion of the first image; and

controlling a predetermined characteristic of a microphone array of the apparatus based at least partially on the orientation of the portion of the first image.

18. The method of claim 17 wherein the controlling of the predetermined characteristic of the microphone array further comprises controlling a sensitivity of the microphone.

19. The method of claim 17 wherein the determining of the orientation of the portion of the first image further comprises detecting a face of a user of the apparatus in the first image.

20. The method of claim 17 wherein the capturing of the first image further comprises capturing an image of a user of the apparatus, and wherein the determining of the orientation of the portion of the image, further comprises determining an orientation of a head of the user.

21. The method of claim 20 wherein the controlling of the predetermined characteristic of the microphone array further comprises controlling a predetermined characteristic of a microphone array based at least partially on the determined head orientation.

22. The method of claim 21 wherein the controlling of the predetermined characteristic of the microphone array further comprises adjusting and directing a sensitivity of the microphone array based at least partially on the determined head orientation.

23. The method of claim 17 further comprising:

capturing a second image with the camera of the apparatus; and

determining an orientation of a portion of the another image based at least partially on the determined orientation of the portion of the first image.

24. A program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine for performing operations to sense an audio signal, the operations comprising:

analyzing a portion of an image;

calculating a direction corresponding to the portion of the image, wherein the direction is relative to an apparatus; and

directing a microphone array from the apparatus based at least partially upon the calculated direction.

25. The program storage device of claim 24 wherein the directing of the microphone array further comprises directing a beam of the microphone array.

26. The program storage device of claim 24 wherein the calculating of the direction corresponding to the portion of the image further comprises calculating a position of a mouth of a user of the apparatus captured in the image.