System and method for audio telepresence

• US 7,184,559 B2
• Filed: 02/23/2001
• Issued: 02/27/2007
• Est. Priority Date: 02/23/2001
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Jouppi, Norman P.
• Primary Examiner(s): Pendleton; Brian T.
• Application Number:
  US09/792,489
• Publication Number:
  US 20020141595A1
• Time in Patent Office: 2,195 Days
• Field of Search: 381/92, 381/17, 379/202.01, 379/206.01, 348/14
• US Class Current: 381/92
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04R 3/12 for distributing signals to two or more loudspeakers specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04S 3/00 Systems employing more than two channels, e.g. quadraphonic H04S5/00, H04S7/00 take precedence

Abstract

A system and method for audio telepresence. The system includes a user station and a telepresence unit. The telepresence unit includes a directional microphone for capturing sounds at the remote location, and means for converting the captured sounds into a stream of data to be communicated to the user station. The user station includes means for receiving the stream of data and a plurality of speakers for recreating the sounds of the remote location. The user station and the speakers are located within an anechoic chamber where sound reflections are substantially absorbed by anechoic linings of the chamber walls. Because of the substantial lack of sound reflection within the anechoic chamber, a user within the anechoic chamber will be able to experience an aural ambience that closely resembles the sounds captured at the remote location. The user station may include microphones for capturing the user'"'"'s voice, and the telepresence unit may include speakers for projecting the user'"'"'s voice at the remote location. Feedback suppression, audio direction steering, and head-coding techniques may also be used to enhance the user'"'"'s sense of remote presence.

Claims

1. An audio telepresence system, comprising:

a user station at a first location, the user station comprising;

a plurality of microphones adapted to be positioned around a user to capture sound produced by the user; and

a lapel microphone for capturing the sound produced by the user;

the user station comprising a computer system configured to;

compare input volumes for each of the plurality of microphones to determine directional information associated with the sound produced by the user based on which one of the plurality of microphones has the highest input volume; and

generate a stream of data representative of sound captured by at least one of the plurality of microphones, the lapel microphone, or both; and

a telepresence unit at a second location, the telepresence unit providing a three-dimensional representation of the user that simultaneously includes a front view and a profile view, the telepresence unit being remotely coupled to the user station to receive the stream of data and the directional information, the telepresence unit comprising a plurality of speakers for projecting sound interpreted from the stream of data in a direction corresponding to the directional information, the telepresence unit being further adapted to capture audio stimuli at the second location and to communicate the audio stimuli to the user station.

2. The audio telepresence system of claim 1, wherein the plurality of microphones each correspond to one of the plurality of screens of the telepresence unit.

3. The audio telepresnece of system of claim 1, wherein the directional information comprises loudness ratios of each of the plurality of microphones relative to a selected one of the plurality of microphones.

4. The audio telepresence system, of claim 1, wherein the telepresence unit includes a computer system for reconstructing a plurality of audio channels from the stream of data and the directional information, the plurality of audio channels each for rendering by one of the plurality of speakers.

5. The audio telepresence system of claim 1, wherein the computer system is configured to adjust a gain of the lapel microphone to approximate that of the one of the plurality of microphones that has the highest input volume.

6. The audio telepresence system of claim 1, wherein the plurality of speakers includes at least one speaker corresponding to each of the plurality of microphones.

7. The audio telepresence system of claim 1, wherein the plurality of speakers includes at least four speakers arranged with respect to an initial user position.

8. The audio telepresence system of claim 7, wherein the at least four speakers include a forward speaker, a rearward speaker, a left speaker, and a right speaker.

9. The audio telepresence system of claim 1, wherein the plurality of microphones includes at least four microphones arranged with respect to an initial user position.

10. The audio telepresence system of claim 9, wherein the at least four microphones include a front microphone, a back microphone, a left microphone, and a right microphone.

11. A method of recreating communication at a first location at a second location, comprising:

capturing sound at the first location, comprising;

capturing the sound at a plurality of positions around a user site with a plurality of fixed microphones;

capturing the sound with a portable microphone;

determining loudness values for sound captured by each of the plurality of fixed microphones;

comparing the loudness values for each of the plurality of fixed microphones;

determining a primary microphone of the plurality of fixed microphones based on the comparison of the loudness values for each of the plurality of fixed microphones;

converting the sound captured by the portable microphone into audio data;

transmitting the audio data to a telepresence unit at the second location; and

projecting the captured sound at the second location, comprising;

playing the audio data at a different volume at each of a plurality of speakers of the telepresence unit based a correspondence between each of the plurality of speakers, the plurality of fixed microphones, and the loudness values associated with the plurality of fixed microphones.

12. The method of claim 11, comprising transmitting a three-dimensional video representation to the telepresence unit, wherein the three-dimensional video representation simultaneously includes a front view and a profile view.

13. The method of claim 12, wherein the three-dimensional video representation simultaneously includes a rear view.

14. The method of claim 11, comprising recording video data at the first location with a plurality of video cameras positioned around the user site.

15. The method of claim 11, wherein the loudness values include loudness ratios of average input volumes for each of the plurality of fixed microphones.

16. The method of claim 11, comprising adjusting a gain of the portable microphone such that its average input volume is substantially equivalent to that of the primary microphone.

17. The method of claim 11, comprising conserving transmission bandwidth by only transmitting an audio channel of the portable microphone and loudness values for the plurality of fixed microphones as the audio data.

18. A telepresence system, comprising:

a user station, comprising;

at least four directional microphones positioned in a substantially horizontal plane around a user site;

a lapel microphone;

a local computer configured to determine input volume values associated with each of the at least four directional microphones and select a primary microphone of the at least four directional microphones based on a comparison of the input volume values;

a transmission unit configured to transmit a data stream including sound captured by the lapel microphone and loudness values to a remote telepresence unit; and

the remote telepresence unit, comprising;

a receptor configured to receive the data stream;

at least four speakers, wherein each of the four speakers corresponds to one of the four directional microphones; and

a processing unit configure to reconstruct the data stream into at least four audio channels and submit each of the at least four audio channels to a different one of the at least four speakers based on the loudness values.

19. The system of claim 18, wherein the local computer is configured to adjust a gain of the lapel microphone to substantially equal the loudness values of the primary microphone.

20. The system of claim 18, wherein the telepresence unit includes a plurality of remote microphones.

21. The system of claim 18, wherein the user station comprises a plurality of cameras positions in a substantially horizontal plane around the user site.

22. The system of claim 21, wherein the remote telepresence unit comprises a plurality of screens, wherein each of the plurality of screens corresponds to at least one of the plurality of cameras.

23. The system of claim 18, wherein the user station comprises a plurality of local speakers corresponding to the plurality of remote microphones.

24. The system of claim 23, wherein the user station comprises a sound steering unit configured to facilitate selection of relative loudness of the sound received from each of the plurality of remote microphones.

25. The system of claim 23, wherein the plurality of local speakers include at least twelve local speakers arranged in two stacked rings disposed about the user cite.