Integrated design for omni-directional camera and microphone array

• US 7,852,369 B2
• Filed: 06/27/2002
• Issued: 12/14/2010
• Est. Priority Date: 06/27/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Cutler, Ross, Rui, Yong, Gupta, Anoop, Tashev, Ivan
• Primary Examiner(s): Ramakrishnaiah; Melur
• Application Number:
  US10/184,499
• Publication Number:
  US 20040001137A1
• Time in Patent Office: 3,092 Days
• Field of Search: 348 1401- 1416, 379/454, 379/453, 379/447
• US Class Current: 348/14.08
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04R 2201/401 2D or 3D arrays of transducers
• CPC Class Codes: H04R 3/02 for preventing acoustic reaction , i.e. acoustic oscillatory feedback specially adapted for hearing aids H04R25/453

Abstract

An omni-directional camera (a 360 degree camera) is proposed with an integrated microphone array. The primary application for such a camera is videoconferencing and meeting recording, and the device is designed to be placed on a meeting room table. The microphone array is in a planar configuration, and the microphones are located as close to the desktop as possible to eliminate sound reflections from the table. The camera is connected to the microphone array base with a thin cylindrical rod, which is acoustically invisible to the microphone array for the frequency range [50-4000] Hz. This provides a direct path from the person talking to all of the microphones in the array, and can therefore be used for sound source localization (determining the location of the talker) and beam-forming (improving the sound quality of the talker by filtering only sound from a particular direction). The camera array is elevated from the table to provide a near frontal viewpoint of the meeting participants.

Claims

1. A system for capturing audio and video data comprising:

one or more cameras that capture video image data;

an array of microphones, that captures audio signals; and

a cylinder connecting said one or more cameras to said microphone array, said cylinder being acoustically transparent in the frequency range of the human voice.

2. The system of claim 1 wherein said one or more cameras capture images over 360 degrees.

3. The system of claim 1 further comprising a microphone array base, and wherein said microphones are mounted to said microphone array base.

4. The system of claim 3 wherein the height of said microphone array base is small enough to position said microphone array to minimize sound reflections picked up by the array from any surface.

5. The system of claim 1 wherein the frequency range to which the cylinder is acoustically transparent is 50-4000 hertz.

6. The system of claim 1 wherein the audio signals are input into a computer, said computer using said audio signals for sound source localization, said sound source localization determining the direction of sounds producing the audio signals.

7. The system of claim 1 wherein the audio signals are input into a computer, said computer using said audio signals for beam-forming, said beam-forming improving the sound quality of a sound from one direction by filtering out the sound from all other directions.

8. The system of claim 1 wherein the audio signals are input into a computer, said computer using human detection and tracking algorithms to accurately detect a person captured in the video image data and associate the video image data of the person with the audio signals generated by the person speaking.

9. The system of claim 1 wherein the video image data is input into a computer, said computer using said video image data to stitch images together.

10. The system of claim 1 wherein microphones in the microphone array are omni-directional.

11. The system of claim 1 wherein the microphones in the microphone array are unidirectional.

12. The system of claim 1 wherein the microphone array comprises three microphones.

13. The system of claim 1 wherein the microphone array comprises eight microphones.

14. The system of claim 1 wherein the microphone array comprises a plurality of microphones that are equilaterally disposed in a circle around the circumference of a planar microphone base.

15. The system of claim 14 wherein at least one microphone is mounted in a rubber casing to protect it from extraneous sound reflections.

16. The system of claim 14 wherein sound insulation is placed below at least one microphone to protect it from extraneous sound reflections.

17. The system of claim 1 wherein said camera further comprises a lens cover, which is up in a normal operating mode, and down in a privacy mode.

18. The system of claim 17 wherein the microphones are turned off when the camera is in said privacy mode is evoked.

19. The system of claim 1 wherein a light on camera is on when the camera is active.

20. The system of claim 1 wherein the audio signals are transmitted over a network.

21. The system of claim 1 wherein the audio signals are saved to a computer-readable medium.

22. The system of claim 1 wherein the video image data is transmitted over a network.

23. The system of claim 1 wherein the video image data is saved to a computer-readable medium.

24. The system of claim 1 wherein the video signals are transferred to a computer using a 1394 bus.

25. The system of claim 1 wherein the audio signals are transferred to a computer using analog cables.

26. The system of claim 1 wherein said cameras are IEEE 1394 cameras.

27. A process of capturing video and audio for teleconferencing and meeting recording, comprising the following process actions:

capturing images of persons in a meeting with an omni-directional camera,

capturing audio signals of sounds occurring in said meeting with a microphone array, said microphone array being housed in a base which is connected to said omni-directional camera by an acoustically transparent rod.

28. The process of claim 27 wherein said microphone array is placed on a table and said wherein said persons in said meeting are seated around said table.

29. The process of claim 28 wherein the microphone array is as close to said table as possible to minimize sound reflections from the table being picked up by the array.

30. The process of claim 27 wherein the microphone array comprises 3 or more microphones.

31. The process of claim 30 wherein said microphone array comprises microphones that are equally spaced in a circle adjacent the circumference of the microphone base.

32. The process of claim 27 wherein the omni-directional camera comprises multiple video sensors.

33. The process of claim 32 wherein said video sensors are equally spaced around the circumference of a circle.

34. The process of claim 27 wherein the omni-directional camera comprises a single video sensor with a hyperbolic lens that captures light rays from 360 degrees.

35. The process of claim 27 wherein said acoustically transparent rod is hollow.

36. The process of claim 27 wherein said acoustically transparent rod is attached to a microphone base and wherein camera cables go through the acoustically transparent rod so said cables do not cause acoustic barriers.

37. The process of claim 27 wherein the outside diameter of said acoustically transparent rod is 2 centimeters or less.

38. The process of claim 27 wherein the acoustically transparent rod has a diameter so as to permit audio frequencies from 50 to 4000 Hz to pass by the acoustically transparent rod essentially undisturbed from a person speaking to all microphones in the microphone array.

39. The acoustically transparent rod of claim 27 wherein said rod height is adjustable.

40. An integrated omni-directional camera and microphone array comprising:

an omni-directional camera;

an acoustically transparent rod that elevates said camera to provide optimum camera coverage; and

a microphone array.

41. The integrated omni-directional camera and microphone array of claim 40 wherein said omni-directional camera employs multiple video sensors to achieve 360 degree camera coverage.

42. The integrated omni-directional camera and microphone array of claim 40 wherein said omni-directional camera employs one video sensor and a hyperbolic lens that captures light from 360 degrees to achieve panoramic coverage.

43. The integrated omni-directional camera and microphone array of claim 40 wherein the acoustically transparent cylindrical rod elevates said camera to provide a frontal view of people sitting around a table.

44. The integrated omni-directional camera and microphone array of claim 40 wherein the microphone array comprises a plurality of microphones disposed at equilateral distances around the circumference of a circle.

45. The integrated omni-directional camera and microphone array of claim 40 wherein the microphone array is sitting on a surface, as close to the surface as possible to minimize sound reflections from said surface.

46. The integrated camera and microphone array of claim 41 wherein video sensors with longer, narrower fields of view are used to capture images at the longer distances, and video sensors with wider, shorter fields of view are used to capture images at shorter distances.

47. A system for capturing audio and video data comprising:

one or more cameras that capture image data;

an array of microphones, wherein said microphones are disposed at equilateral distances around the circumference of a circle, that captures audio signals; and

a cylinder connecting said one or more cameras to said microphone array, said cylinder being acoustically transparent in the frequency range of the human voice, and positioned so as to be substantially in the center of the array of microphones arranged in said circle such that sound waves from a sound source pass by the cylinder in order to reach at least one microphone of the array.

48. The system of claim 47 wherein said one or more cameras capture images over 360 degrees.

49. The system of claim 47 wherein the audio signals are input into a computer, said computer using said audio signals for sound source localization, said sound source localization determining the direction of sounds producing the audio signals.

50. The system of claim 47 wherein the audio signals are input into a computer, said computer using said audio signals for beam-forming, said beam-forming improving the sound quality of a sound from one direction by filtering out the sound from all other directions.

51. A system for capturing audio and video data comprising:

one or more cameras that capture video image data;

an array of microphones, that captures audio signals; and

a straight rigid cylinder connecting said one or more cameras to said microphone array, said straight rigid cylinder being acoustically transparent in the frequency range of the human voice.