Apparatus having cooperating wide-angle digital camera system and microphone array

• US 20030160862A1
• Filed: 02/27/2002
• Published: 08/28/2003
• Est. Priority Date: 02/27/2002
• Status: Abandoned Application

• Current Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Original Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Inventors: Schirtzinger, Thomas R., Galvin, Christopher B., Zurek, Robert A., Reber, William L., Charlier, Michael L.
• Application Number:
  US10/083,912
• Publication Number:
  US 20030160862A1
• Time in Patent Office: Days
• US Class Current: 348/14.08
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/2627 for providing spin image effect, 3D stop motion effect or temporal freeze effect 2D image animation in general G06T13/80
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

A microphone array (20) senses an audio source (22). An audio processor (34) is responsive to the microphone array (20) to determine a direction (24) of the audio source (22) in relation to a frame of reference (32). The direction (24) comprises an azimuth angle (26) and an altitude angle (30). A wide-angle digital camera system (40) captures at least one wide-angle image. An image processor (44) is responsive to the audio processor (34) to process the at least one wide-angle image to generate at least one perspective corrected image (46) in the direction (24) of the audio source (22).

Claims

1. An apparatus comprising:

a microphone array to sense an audio source;

an audio processor responsive to the microphone array to determine a direction of the audio source in relation to a frame of reference, the direction comprising an azimuth angle and an altitude angle;

a wide-angle digital camera system; and

an image processor responsive to the audio processor

and the wide-angle digital camera system, the image processor to process at least one wide-angle image from the wide-angle digital camera system to generate at least one perspective corrected image in the direction of the audio source.

2. The apparatus of claim 1 further comprising a housing having a base and a dome-shaped portion, wherein the wide-angle digital camera system has a field of view emanating about at a peak of the dome-shaped portion.

3. The apparatus of claim 2 wherein microphone array comprises a plurality of microphones disposed about the dome-shaped portion.

4. The apparatus of claim 1 wherein the microphone array comprises a plurality of microphones disposed in a substantially semispherical three-dimensional pattern.

5. The apparatus of claim 1 wherein the microphone array comprises a first ring of microphones and at least a second ring of microphones, the first ring coaxial to and non-concentric with the second ring.

6. The apparatus of claim 1 wherein the audio processor is to determine the direction of a greatest local amplitude in a particular audio band.

7. The apparatus of claim 1 wherein the audio processor is to determine the direction of a limited-duration audio event, and wherein the image processor is to process a plurality of wide-angle images from the wide-angle digital camera system to generate a plurality of perspective corrected images in the direction after the limited-duration audio event has ended.

8. The apparatus of claim 1 wherein the image processor is to generate a first perspective corrected image sequence of a portion of the at least one wide-angle image in the direction of the audio source, and thereafter to generate a second perspective corrected image sequence of a smaller portion of the at least one wide-angle image in the direction of the audio source.

9. The apparatus of claim 1 wherein the audio processor is to determine the direction by comparing virtual outputs of each of a series of virtual compound microphones formed using the microphone array.

10. The apparatus of claim 1 wherein the image processor is to locate a plurality of possible audio sources from the at least one wide-angle image, and wherein the audio processor is responsive to the image processor to identify which of the possible audio sources is the audio source as sensed by the microphone array.

11. The apparatus of claim 1 wherein the wide-angle digital camera system has at least a 180 degree field of view.

12. The apparatus of claim 1 wherein the wide-angle digital camera system has about a 360 degree field of view.

13. The apparatus of claim 1 further comprising:

a wireless transmitter associated with the microphone array and the wide-angle digital camera system; and

a wireless receiver associated with the audio processor and the image processor;

wherein the audio processor is responsive to the microphone array via a wireless link provided between the wireless transmitter and the wireless receiver, and wherein the image processor is responsive to the wide-angle digital camera system via the wireless link.

14. The apparatus of claim 1 wherein the audio processor is to compare a profile of the audio source to a plurality of pre-stored profiles, and to initiate an alarm signal in response to a match of the profile to one of the pre-stored profiles.

15. The apparatus of claim 1 wherein the audio source is a human, wherein the audio processor is to compare a profile of the audio source to a plurality of pre-stored profiles to recognize the human, and wherein the image processor is responsive to the audio processor to augment the perspective corrected image with information identifying the human.

16. An apparatus comprising:

a housing having a dome-shaped portion;

a microphone array comprising a plurality of microphones disposed about the dome-shaped portion; and

a wide-angle digital camera system supported by the housing.

17. The apparatus of claim 16 wherein the wide-angle digital camera system has a field of view emanating about at a peak of the dome-shaped portion.

18. The apparatus of claim 16 wherein the plurality of microphones are disposed in a substantially semispherical three-dimensional pattern.

19. The apparatus of claim 16 wherein the microphone array comprises a ring of microphones.

20. The apparatus of claim 16 wherein the microphone array comprises a first ring of microphones and a second ring of microphones, the first ring coaxial to and non-concentric with the second ring.

21. The apparatus of claim 16 further comprising:

an audio processor responsive to the microphone array and housed by the housing; and

an image processor responsive to the audio processor and housed by the housing.

22. The apparatus of claim 16 further comprising:

an audio processor;

a video processor;

a wireless transmitter associated with the microphone array and the wide-angle digital camera system; and

a wireless receiver associated with the audio processor and the image processor;

wherein the audio processor is responsive to the microphone array via a wireless link provided between the wireless transmitter and the wireless receiver, and wherein the image processor is responsive to the wide-angle digital camera system via the wireless link.

23. The apparatus of claim 16 wherein the wide-angle digital camera system has at least a 180 degree field of view.

24. The apparatus of claim 16 wherein the wide-angle digital camera system has about a 360 degree field of view.

25. An apparatus comprising:

a wide-angle digital camera system to capture at least one wide-angle image;

an image processor to generate at least one perspective corrected image of a portion of the at least one wide-angle image;

a microphone array; and

an audio processor which cooperates with the image processor to modify a directionality of the microphone array to correspond to the portion of the at least one wide-angle image.

26. The apparatus of claim 25 wherein the audio processor cooperates with the image processor to modify a beam width of the microphone array to correspond to the portion of the at least one wide-angle image.

27. The apparatus of claim 26 wherein the audio processor is to modify the beam width by modifying an order of a virtual compound microphone formed using the microphone array.

28. The apparatus of claim 26 wherein the portion is defined by a pan parameter and a zoom parameter, wherein the audio processor cooperates with the image processor to modify the directionality based on the pan parameter and to modify the beam width based on the zoom parameter.

29. The apparatus of claim 28 wherein the portion is further defined by a tilt parameter, wherein the audio processor cooperates with the image processor to modify the directionality based on the tilt parameter.

30. An apparatus comprising:

a wide-angle digital camera system mounted above participants of a teleconference to capture at least one wide-angle image of the participants and of a hard copy document; and

an image processor to generate at least one perspective-corrected image of a first portion of the at least one wide-angle image which includes the hard copy document, and to generate at least one perspective-corrected image of a second portion of the at least one wide-angle image which includes at least one of the participants.

31. The apparatus of claim 30 wherein the wide-angle digital camera is ceiling-mounted.

32. The apparatus of claim 31 wherein the wide-angle digital camera is ceiling-mounted by a member which allows the wide-angle digital camera to have an adjustable vertical position.

33. An apparatus comprising:

a microphone array to sense an audio source; and

an audio processor responsive to the microphone array to determine a direction of the audio source in relation to a frame of reference, to modify a directionality of the microphone array to correspond to the direction, and to modify a beam width of the microphone array.

34. The apparatus of claim 33 wherein the audio processor is to modify the beam width by modifying an order of a virtual compound microphone formed using the microphone array.

35. The apparatus of claim 33 wherein the audio processor is to determine the direction by comparing virtual outputs of each of a series of virtual compound microphones formed using the microphone array.

36. The apparatus of claim 33 wherein the plurality of microphones are disposed in a substantially semispherical three-dimensional pattern.

37. The apparatus of claim 33 wherein the microphone array comprises a first ring of microphones and at least a second ring of microphones, the first ring coaxial to and non-concentric with the second ring.

38. The apparatus of claim 33 wherein the direction comprises an azimuth angle and an altitude angle.