Electronic Device Directional Audio-Video Capture

• US 20110164141A1
• Filed: 11/26/2008
• Published: 07/07/2011
• Est. Priority Date: 07/31/2008
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Technologies Oy (Nokia Corporation)
• Inventors: Ozcan, Koray, Vehvilainen, Markku K., Hamalainen, Matti S., Tico, Marius
• Granted Patent: US 9,668,077 B2
• Time in Patent Office: Days
• US Class Current: 348/207.99
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: H04M 1/6041 Portable telephones adapted for handsfree use
• CPC Class Codes: H04M 2250/52 including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F
• CPC Class Codes: H04R 29/005 Microphone arrays
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

Disclosed herein is an apparatus. The apparatus includes a housing, electronic circuitry, and an audio-visual source tracking system. The electronic circuitry is in the housing. The audio-visual source tracking system includes a first video camera and an array of microphones. The first video camera and the array of microphones are attached to the housing. The audio-visual source tracking system is configured to receive video information from the first video camera. The audio-visual source tracking system is configured to capture audio information from the array of microphones at least partially in response to the video information. The audio-visual source tracking system might include a second video camera that is attached to the housing, wherein the first and second video cameras together estimate the beam orientation of the array of microphones.

Claims

1. An apparatus comprising:

a housing;

electronic circuitry in the housing; and

an audio-visual source tracking system comprising a first video camera and an array of microphones, wherein the first video camera and the array of microphones are attached to the housing, wherein the audio-visual source tracking system is configured to receive video information from the first video camera, and wherein the audio-visual source tracking system is configured to capture audio information from the array of microphones at least partially in response to the video information.

2. An apparatus as in claim 1 wherein the array of microphones are proximate the first video camera.

3. An apparatus as in claim 1 wherein the array of microphones comprises at least three microphones.

4. An apparatus as in claim 1 wherein the audio-visual source tracking system is configured to adjust and direct the sensitivity of the microphone array at least partially in response to the video information.

5. An apparatus as in claim 1 wherein the audio-visual source tracking system is configured to receive video information corresponding to a user of the apparatus from the first video camera.

6. An apparatus as in claim 1 wherein the apparatus comprises a mobile handset.

10. An apparatus as in claim 1 further comprising a second video camera, wherein the first and the second video cameras are configured to estimate a beam orientation for the array of microphones.

7-9. -9. (canceled)

11-16. -16. (canceled)

17. A method comprising:

capturing a first image with a camera of an apparatus;

determining a direction of a portion of the first image with respect to a microphone array of the apparatus; and

controlling a predetermined characteristic of the microphone array based at least partially on the direction of the portion of the first image.

18. A method as in claim 17 wherein the controlling of the predetermined characteristic of the microphone array further comprises controlling a sensitivity of the microphone.

19. A method as in claim 17 wherein the determining of the direction of the portion of the first image further comprises detecting a face of a user of the apparatus in the first image.

20. A method as in claim 17 wherein the capturing of the first image further comprises capturing an image of a user of the apparatus, and wherein the determining of the direction of the portion of the image, further comprises determining a direction of a head of the user.

21-26. -26. (canceled)

27. An apparatus comprising:

a housing;

electronic circuitry in the housing; and

an audio-visual optimizing system comprising a first video camera and an array of microphones, wherein the first video camera and the array of microphones are attached to the housing, wherein the first video camera comprises at least one adjustable parameter, wherein the audio-visual optimizing system is configured to receive audio information from the array of microphones, and wherein the audio-visual optimizing system is configured to adjust the at least one adjustable parameter of the first video camera in response to, at least partially, the audio information.

28. An apparatus as in claim 27 wherein the at least one adjustable parameter comprises an auto-focus parameter.

29. An apparatus as in claim 27 wherein the at least one adjustable parameter comprises an auto-exposure parameter.

30. An apparatus as in claim 27 wherein the array of microphones is configured for microphone array beam steering.

31. An apparatus as in claim 27 wherein the array of microphones comprises at least three microphones.

32. An apparatus as in claim 27 wherein the audio-visual optimizing system further comprises a second video camera.

34. An apparatus as in claim 32 wherein the first and the second video cameras are configured to estimate depth information.

33. (canceled)

35. A method comprising:

capturing an image with a camera of an apparatus;

determining a first position of a portion of the image;

capturing audio information corresponding to the first position; and

controlling a parameter of the camera in response to, at least partially, the captured audio information.

36. A method as in claim 35 wherein the controlling of the parameter further comprises controlling an exposure and/or a focus parameter of the camera.

37. A method as in claim 35 wherein the capturing of the audio information further comprises adjusting a sensitivity of a microphone array.

38-41. -41. (canceled)