Methods and systems for creating a spatial auditory environment in an audio conference system

• US 5,889,843 A
• Filed: 03/04/1996
• Issued: 03/30/1999
• Est. Priority Date: 03/04/1996
• Status: Expired due to Term

• Current Assignee: Vulcan Patents LLC
• Original Assignee: Interval Research Corporation
• Inventors: Meike, Roger C., Charnley, Don, Hindus, Debby, Burdick, Wayne, Stifelman, Lisa, Singer, Andrew Jay, White, Sean Michael, Edens, Glenn T.
• Primary Examiner(s): Hong, Harry S.
• Application Number:
  US08/610,638
• Time in Patent Office: 1,121 Days
• Field of Search: 348/13, 348/14, 348/15, 370/62, 370/260, 370/263, 370/266, 370/267, 370/268, 370/269, 379/202, 379/203, 379/204, 379/205, 379/206, 381/17, 381/25, 381/1, 381/24, 381/26
• US Class Current: 379/202.01
• CPC Class Codes: H04M 3/56 Arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities video conference systems H04N7/15
• CPC Class Codes: H04M 3/568 audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F

Abstract

A method and a system for audio communication between a plurality of users at a plurality of sites utilizes a set of audio input sensors at each site. Each set of audio input sensors binaurally senses an auditory space in proximity thereto. A metaphorical representation of each of the sites is provided. Each metaphorical representation has a position which is variable within a metaphorical space. The metaphorical representations can be based upon, for example, a physical metaphor, a visual metaphor, an auditory metaphor, or a textual metaphor. The auditory space sensed at each site is combined to form at least one synthetic auditory space. The at least one synthetic auditory space is formed in dependence upon the position of each metaphorical representation within the metaphorical space. A binaurally perceivable auditory environment is produced at one or more sites based upon the at least one synthetic auditory space.

Claims

1. A system for audio communication between a plurality of users at a plurality of sites, the system comprising:

a set of audio input sensors at each site, each set of audio input sensors binaurally sensing an auditory space in proximity thereto;

means for providing a metaphorical representation of each of the sites, each metaphorical representation being moveable by a user to a position within a metaphorical space;

means for combining the auditory space sensed at each site to form at least one synthetic auditory space, the at least one synthetic auditory space formed in dependence upon the position of each metaphorical representation within the metaphorical space; and

means for producing a binaurally-perceivable auditory environment at each site based upon the at least one synthetic auditory space.

2. The system of claim 1 wherein the metaphorical space is delineated into a plurality of subspaces for metaphorically representing each synthetic auditory space.

3. The system of claim 2 wherein one synthetic auditory space is formed by combining the auditory space of at least one site having a metaphorical representation contained within a predetermined subspace.

4. The system of claim 1 wherein the auditory space sensed at one site is produced at a binaurally-perceived position at another site dependent upon the position of the metaphorical representation of the one site within the metaphorical space.

5. The system of claim 1 wherein the means for providing the metaphorical representation provides a physical representation of at least one of the sites.

6. The system of claim 5 wherein the means for providing the metaphorical representation includes:

at least one physical object for metaphorically representing the at least one of the sites, each physical object having a position which is variable within a physical space; and

means for sensing the position of the at least one physical object within the physical space.

7. The system of claim 6 wherein the at least one physical object is positionable on a planar surface.

8. The system of claim 6 wherein the at least one physical object is positionable on a plurality of planar surfaces.

9. The system of claim 1 wherein the means for providing the metaphorical representation provides a visual representation of at least one of the sites.

10. The system of claim 9 wherein the means for providing the metaphorical representation includes:

a display device on which at least one graphical object is displayed, the at least one graphical object metaphorically representing the at least one of the sites;

means for variably positioning the at least one graphical object on the display; and

means for determining the position of the at least one graphical object.

11. The system of claim 1 wherein the means for providing the metaphorical representation provides an auditory representation of at least one of the sites.

12. The system of claim 11 wherein the means for providing the metaphorical representation includes:

an auditory display on which at least one auditory object is displayed, the at least one auditory object metaphorically representing at least one of the sites;

means for variably positioning the at least one auditory object; and

means for determining the position of the at least one auditory object.

13. The system of claim 1 wherein the means for providing the metaphorical representation provides a textual representation of at least one of the sites.

14. The system of claim 13 wherein the means for providing the metaphorical representation includes:

a computer executing a control program which maintains a state space model of at least one mnemonic object, the at least one mnemonic object metaphorically representing the at least one of the sites;

means for receiving a textual command for modifying the state of the at least one mnemonic object within the state space model; and

means for displaying a textual indication of the state of the at least one mnemonic object within the state space model.

15. The system of claim 1 wherein the set of audio input sensors at one site includes a pair of microphones configured to simulate a predetermined intracranial distance between human ears.

16. The system of claim 1 wherein the set of audio input sensors at one site includes a pair of sensors which physically simulates a predetermined binaural head-related transfer function.

17. The system of claim 1 wherein the set of audio input sensors at one site includes a pair of displacement sensors for sensing the displacement of a pair of in vivo tympanic membranes in response to the auditory space in proximity thereto.

18. The system of claim 1 wherein the means for combining includes means for convolving an auditory space sensed at one of the sites with a predetermined binaural human head-related transfer function.

19. The system of claim 1 wherein the means for combining includes at least one digital audio processor for composing the synthetic auditory space at one of the sites based upon the auditory space sensed in at least one other site.

20. The system of claim 1 wherein the means for producing the synthetic auditory environment includes at least one analog mixer for composing the synthetic auditory environment at one of the sites based upon the auditory space sensed in at least one other site.

21. The system of claim 1 wherein the means for producing the binaurally-perceivable auditory environment includes a plurality of speakers located at one of the sites.

22. The system of claim 1 wherein the means for producing the binaurally-perceivable auditory environment includes means to servo tympanic membranes of the user.

23. A method of audio communication between a plurality of users at a plurality of sites, each site having a set of audio input sensors, the method comprising:

binaurally sensing an auditory space in proximity to each set of audio input sensors;

providing a metaphorical representation of each of the sites, each metaphorical representation being moveable by a user to a position within a metaphorical space;

combining the auditory space sensed at each site to form at least one synthetic auditory space, the at least one synthetic auditory space formed in dependence upon the position of each metaphorical representation within the metaphorical space; and

producing a binaurally-perceivable auditory environment at each site based upon the at least one synthetic auditory space.

24. The method of claim 23 wherein the metaphorical space is delineated into a plurality of subspaces for metaphorically representing each synthetic auditory space.

25. The method of claim 24 wherein one of the at least one synthetic auditory space is formed by combining the auditory space of at least one site having a metaphorical representation contained within a predetermined subspace.

26. The method of claim 23 wherein the auditory space sensed at one site is produced at a binaurally-perceived position at another site dependent upon the position of the metaphorical representation of the one site within the metaphorical space.

27. The method of claim 23 wherein the step of providing the metaphorical representation includes providing a physical representation of at least one of the sites.

28. The method of claim 27 wherein the step of providing the metaphorical representation includes:

providing at least one physical object for metaphorically representing the at least one of the sites, each physical object having a position which is variable within a physical space; and

sensing the position of the at least one physical object within the physical space.

29. The method of claim 28 wherein the at least one physical object is positionable on a planar surface.

30. The method of claim 28 wherein the at least one physical object is positionable on a plurality of planar surfaces.

31. The method of claim 23 wherein the step of providing the metaphorical representation includes providing a visual representation of at least one of the sites.

32. The method of claim 31 wherein the step of providing the metaphorical representation includes:

providing a display device on which at least one graphical object is displayed, the at least one graphical object metaphorically representing the at least one of the sites;

variably positioning the at least one graphical object on the display; and

determining the position of the at least one graphical object.

33. The method of claim 23 wherein the step of providing the metaphorical representation includes providing an auditory representation of at least one of the sites.

34. The method of claim 33 wherein the step of providing the metaphorical representation includes:

providing an auditory display on which at least one auditory object is displayed, the at least one auditory object metaphorically representing at least one of the sites;

variably positioning the at least one auditory object; and

determining the position of the at least one auditory object.

35. The method of claim 23 wherein the step of providing the metaphorical representation includes providing a textual representation of at least one of the sites.

36. The method of claim 35 wherein the step of providing the metaphorical representation includes:

executing a control program on a computer to maintain a state space model of at least one mnemonic object, the at least one mnemonic object metaphorically representing the at least one of the sites;

receiving a textual command for modifying the state of the at least one mnemonic object within the state space model; and

displaying a textual indication of the state of the at least one mnemonic object within the state space model.

37. The method of claim 23 wherein the set of audio input sensors at one site includes a pair of microphones configured to simulate a predetermined intracranial distance between human ears.

38. The method of claim 23 wherein the set of audio input sensors at one site includes a pair of sensors which physically simulates a predetermined binaural head-related transfer function.

39. The method of claim 23 wherein the set of audio input sensors at one site includes a pair of displacement sensors for sensing the displacement of a pair of in vivo tympanic membranes in response to the auditory space in proximity thereto.

40. The method of claim 23 wherein the step of combining includes convolving an auditory space sensed at one of the sites with a predetermined binaural human head-related transfer function.

41. The method of claim 23 wherein the step of combining includes:

providing at least one digital audio processor; and

composing the synthetic auditory space at one of the sites based upon the auditory space sensed in at least one other site using the at least one digital audio processor.

42. The method of claim 23 wherein the step of producing the synthetic auditory environment includes:

providing at least one analog mixer; and

composing the synthetic auditory environment at one of the sites based upon the auditory space sensed in at least one other site using the at least one analog mixer.

43. The method of claim 23 wherein the binaurally-perceivable auditory environment is produced using a plurality of speakers located at one of the sites.

44. The method of claim 23 wherein the step of producing the binaurally-perceivable auditory environment includes servoing tympanic membranes of the user.

45. A system for use with an audio conferencing apparatus which provides a user with a corresponding bidirectional audio communication link with each of a plurality of external transceivers, the audio conferencing apparatus containing an audio output device capable of producing sound waves from a first audio signal applied thereto, the audio conferencing apparatus further containing an audio input device capable of producing a second audio signal from sound waves received thereby for transmission to the plurality of external transceivers, the system comprising:

a graphical user interface which displays a corresponding icon for each of the external transceivers at a corresponding location in a display space, wherein the corresponding location of each icon is capable of being modified by the user;

a processor, operatively associated with the graphical user interface, which generates a control signal in dependence upon the corresponding location of each icon with respect to a reference location; and

an audio mixer, operatively associated with the processor, which variably amplifies and variably pans a corresponding audio signal received from each of the plurality of external transceivers in dependence upon the control signal, and forms a mixed audio signal from a combination thereof;

wherein the mixed audio signal is applied to the audio output device to create a spatial auditory environment based upon the audio signals received from the plurality of external transceivers.

46. The system of claim 45 wherein each corresponding audio signal is amplified in dependence upon a distance between the corresponding location of the corresponding icon and the reference location.

47. The system of claim 46 wherein the distance is a Cartesian distance.

48. The system of claim 46 wherein the distance is measured along a coordinate axis.

49. The system of claim 48 wherein the coordinate axis has a generally vertical orientation.

50. The system of claim 45 wherein each corresponding audio signal is panned in dependence upon an angle between a predetermined axis and a line passing simultaneously through the corresponding location of the corresponding icon and the reference location.

51. The system of claim 45 wherein each corresponding audio signal is panned in dependence upon a distance between the corresponding location of the corresponding icon and the reference location.

52. The system of claim 51 wherein the distance is measured along a coordinate axis.

53. The system of claim 52 wherein the coordinate axis has a generally horizontal orientation.

54. The system of claim 45 wherein the graphical user interface is further capable of displaying a reference icon at the reference location in the display space.

55. The system of claim 54 wherein the graphical user interface further allows the user to modify the reference location of the reference icon.

56. The system of claim 45 wherein the mixed audio signal is a stereo audio signal.

57. The system of claim 45 wherein the audio output device comprises a plurality of speakers.

58. A method of bidirectional communication with each of a plurality of external transceivers, the method comprising the steps of:

transmitting a first audio signal to each of the plurality of external transceivers;

displaying a corresponding icon for each of the external transceivers at a corresponding location in a display space;

receiving a corresponding audio signal from each of the plurality of external transceivers;

amplifying and panning each corresponding audio signal in dependence upon the corresponding location of the corresponding icon with respect to a reference location;

mixing the amplified and panned corresponding audio signals to form a mixed audio signal; and

applying the mixed audio signal to an audio output device to create a spatial auditory environment based upon the audio signals received from the plurality of external transceivers.

59. The method of claim 58 further comprising the step of modifying the corresponding location of at least one icon.

60. The method of claim 58 wherein each corresponding audio signal is amplified in dependence upon a distance between the corresponding location of the corresponding icon and the reference location.

61. The method of claim 60 wherein the stance is a Cartesian distance.

62. The method of claim 60 wherein the distance is measured along a coordinate axis.

63. The method of claim 62 wherein the coordinate axis has a generally vertical orientation.

64. The method of claim 58 wherein each corresponding audio signal is panned in dependence upon an angle between a predetermined axis and a line passing simultaneously through the corresponding location of the corresponding icon and the reference location.

65. The method of claim 58 wherein each corresponding audio signal is panned in dependence upon a distance between the corresponding location of the corresponding icon and the reference location.

66. The method of claim 65 wherein the distance is measured along a coordinate axis.

67. The method of claim 66 wherein the coordinate axis has a generally horizontal orientation.

68. The method of claim 58 wherein the step of displaying comprises displaying a reference icon at the reference location in the display space.

69. The method of claim 68 wherein the step of displaying further comprises allowing the user to modify the reference location of the reference icon.

70. The method of claim 58 wherein the mixed audio signal is a stereo audio signal.