Method and apparatus for producing binaural audio for a moving listener

• US 6,243,476 B1
• Filed: 06/18/1997
• Issued: 06/05/2001
• Est. Priority Date: 06/18/1997
• Status: Expired due to Term

• Current Assignee: Massachusetts Institute of Technology
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Gardner, William G.
• Primary Examiner(s): Isen, Forester W.
• Assistant Examiner(s): PENDLETON, BRIAN T
• Application Number:
  US08/878,221
• Time in Patent Office: 1,448 Days
• Field of Search: 381/1, 381/309, 381/310, 381/17, 381/303
• US Class Current: 381/303
• CPC Class Codes: H04S 1/002 Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution control circuits for electronic adaptation of the sound field H04S7/30
• CPC Class Codes: H04S 1/005 For headphones
• CPC Class Codes: H04S 1/007 in which the audio signals are in digital form data reduction aspects thereof based on psychoacoustics G10L19/02
• CPC Class Codes: H04S 2400/01 Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
• CPC Class Codes: H04S 2420/01 Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
• CPC Class Codes: H04S 3/002 Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution control circuits for electronic adaptation of the sound field H04S7/30
• CPC Class Codes: H04S 7/303 Tracking of listener position or orientation
• CPC Class Codes: H04S 7/304 For headphones

Abstract

A system for generating loudspeaker-ready binaural signals comprises a tracking system for detecting the position and, preferably, the angle of rotation of a listener'"'"'s head; and means, responsive to the head-tracking means, for generating the binaural signal. The system may also include a crosstalk canceller responsive to the tracking system, and which adds to the binaural signal a crosstalk cancellation signal based on the position (and/or the rotation angle) of the listener'"'"'s head. The invention may also address the high-frequency components not generally affected by the crosstalk canceller by considering these frequencies in terms of power (rather than phase). By implementing the compensation in terms of power levels rather than phase adjustments, the invention avoids the shortcomings heretofore encountered in attempting to cancel high-frequency crosstalk.

Claims

1. Apparatus for generating binaural audio for a moving listener, the apparatus comprising:

a. means for tracking movement of a listener'"'"'s head; and

b. means, responsive to the tracking means, for generating a movement-responsive binaural signal for broadcast to the moving listener through a pair of non-head-mounted loudspeakers, the signal-generating means comprising (i) means for receiving an input signal, (ii) first and second means for receiving the input signal and generating therefrom first and second binaural signals, respectively, and (iii) crosstalk cancellation means, responsive to the tracking means for receiving the first and second binaural signals and adding thereto a crosstalk cancellation signal, the crosstalk cancellation signal being based on position of the listener'"'"'s head so as to compensate for head movement.

2. The apparatus of claim 1 wherein the crosstalk cancellation means comprises first and second head-shadowing filters for modeling phase and amplitude alteration of the crosstalk signal due to head diffraction.

3. The apparatus of claim 2 wherein the crosstalk cancellation means further comprises first and second ipsilateral equalization filters for compensating for position of the loudspeakers.

4. The apparatus of claim 2 wherein the crosstalk cancellation means further comprises at least one variable time delay for compensating for different path lengths from a pair of loudspeakers to the listener.

5. The apparatus of claim 2 wherein the head-shadowing filters are lowpass filters.

6. The apparatus of claim 5 wherein the head-shadowing filters comprise low-order infinite impulse response filters.

7. The apparatus of claim 4 wherein the at least one variable time delay comprises a low-order finite-impulse response interpolator.

8. The apparatus of claim 1 wherein the tracking means detects a position and a rotation angle of the listener'"'"'s head, the crosstalk cancellation means comprising:

a. a series of filters, each filter being matched to a head position and a head rotation angle, for generating a crosstalk cancellation signal;

b. selection means, responsive to the tracking means, for selecting a filter to receive the first and second binaural signals.

9. The apparatus of claim 8 wherein selection means further comprises interpolation means, the selection means identifying at least two filters associated with head positions and head rotation angles closest to the position and rotation angle detected by the tracking means, the interpolation means generating an intermediate filter based on the identified filters.

10. The apparatus of claim 1 wherein the signal-generating means comprises:

a. means for receiving an input signal;

b. first and second means for receiving the input signal and generating therefrom first and second binaural signals, respectively, the binaural signals each (i) corresponding to a synthesized source having an apparent spatial position and (ii) having high-frequency components with power levels;

c. means for varying the power levels of the high-frequency component to compensate for crosstalk.

11. The apparatus of claim 10 wherein the power-varying means comprises, for each binaural signal,

a. at least one shelving filter having a high-frequency gain;

and

b. means, responsive to the tracking means, for establishing the high-frequency gain of the shelving filter.

12. The apparatus of claim 10 wherein the tracking means detects a position and a rotation angle of the listener'"'"'s head, the establishing means establishing the high-frequency gain based on the head position, the rotation angle and the position of the synthesized source.

13. The apparatus of claim 10 wherein the high-frequency component includes frequencies above 6 kHz.

14. The apparatus of claim 11 wherein the shelving filters have identical low-frequency phase and magnitude response independent of high-frequency gain.

15. The apparatus of claim 10 wherein the binaural signals further comprise low-frequency components, the apparatus further comprising crosstalk cancellation means, responsive to the tracking means, for receiving the first and second binaural signals and adding to the low-frequency components thereof a crosstalk cancellation signal, the crosstalk cancellation signal being based on position of the listener'"'"'s head so as to compensate for head movement.

16. The apparatus of claim 15 wherein the crosstalk cancellation means comprises first and second head-shadowing filters for compensating for phase and amplitude alteration of the crosstalk signal due to head diffraction.

17. Apparatus for generating binaural audio for a listener, the apparatus comprising:

a. means for detecting (i) a position of a listener'"'"'s head with respect to a pair of non-head-mounted loudspeakers, the position comprising a distance from each loudspeaker, and (ii) an orientation of the listener'"'"'s head, the orientation comprising a head-rotation angle; and

b. means, responsive to the tracking means, for generating a movement-responsive binaural signal for broadcast to the listener through the loudspeakers, the signal-generating means comprising (i) means for receiving an input signal, (ii) first and second means for receiving the input signal and generating therefrom first and second binaural signals, respectively, and (iii) crosstalk cancellation means, responsive to the tracking means, for receiving the first and second binaural signals and adding thereto a crosstalk cancellation signal, the crosstalk cancellation signal being based on the position and the orientation of the listener'"'"'s head so as to compensate for head movement.

18. The apparatus of claim 17 wherein the crosstalk cancellation means comprises first and second head-shadowing filters for modeling phase and amplitude alteration of the crosstalk signal due to head diffraction.

19. The apparatus of claim 18 wherein the crosstalk cancellation means further comprises first and second ipsilateral equalization filters for compensating for position of the loudspeakers.

20. The apparatus of claim 18 wherein the crosstalk cancellation means further comprises at least one variable time delay for compensating for different path lengths from a pair of loudspeakers to the listener.

21. The apparatus of claim 18 wherein the head-shadowing filters are lowpass filters.

22. The apparatus of claim 21 wherein the head-shadowing filters comprise low-order infinite impulse response filters.

23. The apparatus of claim 20 wherein the at least one variable time delay comprises a low-order finite-impulse response interpolator.

24. The apparatus of claim 17 wherein the crosstalk cancellation means comprises:

a. a series of filters, each filter being matched to a head position and a head rotation angle, for generating a crosstalk cancellation signal;

b. selection means, responsive to the tracking means, for selecting a filter to receive the first and second binaural signals.

25. The apparatus of claim 24 wherein selection means further comprises interpolation means, the selection means identifying at least two filters associated with head positions and head rotation angles closest to the position and rotation angle detected by the tracking means, the interpolation means generating an intermediate filter based on the identified filters.

26. The apparatus of claim 17 wherein the signal-generating means comprises:

a. means for receiving an input signal;

b. first and second means for receiving the input signal and generating therefrom first and second binaural signals, respectively, the binaural signals each (i) corresponding to a synthesized source having an apparent spatial position and (ii) having high-frequency components with power levels;

c. means for varying the power levels of the high-frequency component to compensate for crosstalk.

27. The apparatus of claim 26 wherein the power-varying means comprises, for each binaural signal,

a. at least one shelving filter having a high-frequency gain;

and

b. means, responsive to the tracking means, for establishing the high-frequency gain of the shelving filter.

28. The apparatus of claim 26 wherein the establishing means establishes the high-frequency gain based on the head position, the head orientation and the position of the synthesized source.

29. The apparatus of claim 26 wherein the high-frequency component includes frequencies above 6 kHz.

30. The apparatus of claim 27 wherein the shelving filters have identical low-frequency phase and magnitude response independent of high-frequency gain.

31. The apparatus of claim 26 wherein the binaural signals further comprise low-frequency components, the apparatus further comprising crosstalk cancellation means, responsive to the tracking means, for receiving the first and second binaural signals and adding to the low-frequency components thereof a crosstalk cancellation signal, the crosstalk cancellation signal being based on position of the listener'"'"'s head so as to compensate for head movement.

32. The apparatus of claim 31 wherein the crosstalk cancellation means comprises first and second head-shadowing filters for modeling phase and amplitude alteration of the crosstalk signal due to head diffraction.

33. Apparatus for generating binaural audio without high-frequency crosstalk, the apparatus comprising:

a. means for generating a binaural signal for broadcast through a pair of loudspeakers;

b. first and second means for receiving the input signal and generating therefrom first and second binaural signals, respectively, the binaural signals each (i) corresponding to a synthesized source having an apparent spatial position and (ii) having high-frequency components with power levels; and

c. means for varying the power levels of the high-frequency component to compensate for crosstalk.

34. The apparatus of claim 33 wherein the power-varying means comprises, for each binaural signal,

a. at least one shelving filter having a high-frequency gain; and

b. means for establishing the high-frequency gain of the shelving filter.

35. The apparatus of claim 33 further comprising means for tracking a position and a rotation angle of a listener'"'"'s head, the establishing means establishing the high-frequency gain based on the head position, the rotation angle and the position of the synthesized source.

36. The apparatus of claim 33 wherein the high-frequency component includes frequencies above 6 kHz.

37. The apparatus of claim 34 wherein the shelving filters have identical low-frequency phase and magnitude response independent of high-frequency gain.

38. The apparatus of claim 33 wherein the binaural signals further comprise low-frequency components, the apparatus further comprising crosstalk cancellation means, responsive to the tracking means, for receiving the first and second binaural signals and adding to the low-frequency components thereof a crosstalk cancellation signal, the crosstalk cancellation signal being based on position of the listener'"'"'s head so as to compensate for head movement.

39. The apparatus of claim 38 wherein the crosstalk cancellation means comprises first and second head-shadowing filters for modeling phase and amplitude alteration of the crosstalk signal due to head diffraction.

40. A method of generating binaural audio for a moving listener, the method comprising the steps of:

a. tracking movement of a listener'"'"'s head; and

b. generating, in response to the tracked movement, a movement-responsive binaural signal for broadcast to the moving listener through a pair of non-head-mounted loudspeakers.

41. A method of generating binaural audio for a listener, the method comprising the steps of:

a. detecting (i) a position of a listener'"'"'s head with respect to a pair of non-head-mounted loudspeakers, the position comprising a distance from each loudspeaker, and (ii) an orientation of the listener'"'"'s head, the orientation comprising a head-rotation angle; and

b. generating, in response to the detected position, a movement-responsive binaural signal for broadcast to the listener through the loudspeakers, the signal containing a crosstalk-cancellation component.

42. A method of generating binaural audio without high-frequency crosstalk, the method comprising the steps of:

a. generating a binaural signal for broadcast through a pair of loudspeakers;

b. receiving the input signal and generating therefrom first and second binaural signals, respectively, the binaural signals each (i) corresponding to a synthesized source having an apparent spatial position and (ii) having high-frequency components with power levels; and

c. varying the power levels of the high-frequency component to compensate for crosstalk.