Audio source locator and tracker, a method of directing a camera to view an audio source and a video conferencing terminal

• US 8,754,925 B2
• Filed: 03/01/2011
• Issued: 06/17/2014
• Est. Priority Date: 09/30/2010
• Status: Expired due to Fees

• Current Assignee: RPX Corporation
• Original Assignee: Alcatel-Lucent SA (Nokia Corporation)
• Inventors: Sutter, Jr., Edward L., Ng, Hock M., Abbot, Richard M.
• Primary Examiner(s): Gauthier, Gerald
• Application Number:
  US13/038,183
• Publication Number:
  US 20120081504A1
• Time in Patent Office: 1,204 Days
• Field of Search: 348/14.08, 348/61, 348/169, 348/231.4, 348/14.05, 348/14.1, 348/14.16, 381/92, 381/310, 381/17, 369/4, 725/105
• US Class Current: 348/14.08
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

An audio locating and tracking apparatus, a method of directing a camera to view an audio source, and a video conferencing terminal are disclosed herein. In one embodiment, the apparatus includes: (1) an audio source identifier configured to locate an audio source based on multimodal sensor data from at least two different types of sensors and (2) an image selector configured to automatically direct a camera to view the audio source.

Claims

1. An apparatus, comprising:

an audio source identifier configured to locate an audio source based on multimodal sensor data from at least two different types of sensors, wherein said at least two different types of sensors include at least one type of sensor that is not an audio sensor nor a video sensor; and

an image selector configured to automatically direct a camera to view said audio source based on said multimodal sensor data.

2. The apparatus as recited in claim 1 further comprising said camera.

3. The apparatus as recited in claim 1 further comprising said at least two different types of sensors.

4. The apparatus as recited in claim 1 wherein said at least two different types of sensors include an audio sensor and at least one sensor selected from the group consisting of:

a video sensor,

a thermal sensor, and

a range finder.

5. The apparatus as recited in claim 1 wherein said audio source identifier is further configured to locate potential audio sources based on said multimodal sensor data.

6. The apparatus as recited in claim 5 wherein said image selector is further configured to generate a map of said potential audio sources.

7. The apparatus as recited in claim 5 wherein said audio source identifier is configured to locate said potential audio sources before locating said audio source.

8. The apparatus as recited in claim 1 wherein said audio source identifier is further configured to detect movement of said audio source based on data from at least one of said at least two different types of sensors.

9. The apparatus as recited in claim 8 wherein said image selector is further configured to automatically steer said camera in response to said detected movement to track said audio source.

10. A method of directing a camera to view an audio source, comprising:

locating an audio source based on multimodal sensor data from at least two different types of sensors, wherein said at least two different types of sensors include at least one type of sensor that is not an audio sensor nor a video sensor; and

automatically directing a camera to view said audio source based on said multimodal sensor data.

11. The method as recited in claim 10 wherein said at least two different types of sensors include an audio sensor and at least one sensor selected from the group consisting of:

a video sensor,

a thermal sensor, and

a range finder.

12. The method as recited in claim 10 further comprising locating potential audio sources based on said multimodal sensor data.

13. The method as recited in claim 12 further comprising generating a map of said potential audio sources with respect to a location of said camera.

14. The method as recited in claim 12 wherein said locating said audio source is based on said map.

15. The method as recited in claim 10 further comprising capturing an image of said audio source employing said camera.

16. The method as recited in claim 10 further comprising detecting movement of said audio source based on data from at least one of said at least two different types of sensors and automatically steering said camera in response to said detected movement to track said audio source.

17. A video conferencing terminal, comprising:

a camera configured to capture images within a field of view; and

an audio source locator and tracker configured to locate an audio source based on multimodal sensor data from at least two different types of sensors and automatically direct said camera to view said audio source, wherein said at least two different types of sensors include at least one type of sensor that is not an audio sensor nor a video sensor.

18. The videoconferencing terminal as recited in claim 17 further comprising said at least two different types of sensors.

19. The videoconferencing terminal as recited in claim 17 wherein said audio source locator and tracker is further configured to locate potential audio sources based on said multimodal sensor data and generate a map of said potential audio sources.

20. The videoconferencing terminal as recited in claim 17 wherein said audio source locator and tracker is further configured to track movement of said audio source based on data from at least one of said at least two different types of sensors and automatically steer said camera in response to said movement.