Audio signal processing system and method

• US 20050100171A1
• Filed: 10/21/2004
• Published: 05/12/2005
• Est. Priority Date: 11/12/2003
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: McKeag, Adam Richard, Reilly, Andrew Peter
• Granted Patent: US 7,949,141 B2
• Time in Patent Office: Days
• US Class Current: 381/63
• CPC Class Codes: G10H 1/0091 Means for obtaining special acoustic effects combined with modulation G10H1/043
• CPC Class Codes: G10H 1/125 using a digital filter
• CPC Class Codes: G10H 2210/281 Reverberation or echo
• CPC Class Codes: G10H 2210/301 Soundscape or sound field simulation, reproduction or control for musical purposes, e.g. surround or 3D sound; Granular synthesis
• CPC Class Codes: G10K 15/12 using electronic time-delay networks
• CPC Class Codes: H04S 2400/01 Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
• CPC Class Codes: H04S 2420/01 Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
• CPC Class Codes: H04S 7/306 For headphones

Abstract

A method, an apparatus, and a software product to process a plurality of input audio signals. The apparatus accepts a plurality of input signals and includes a multi-input, multi-output reverberator arranged to generate a set of output signals including delayed reverberation components simulating the reverberations a listener is likely to hear in a listening environment. The apparatus further includes a multi-input, two-output filter accepting the outputs of the reverberator and the plurality of input terminals, providing outputs for the left and right ears, and configured to implement a set of head related transfer functions corresponding to a listening environment and a set of directions of a listener in the listening environment. The apparatus is such that a listener listening to the outputs through headphones has the sensation of listening to the plurality of input audio signals as if they are emanating from a plurality of loudspeakers spatially located in the listening environment at a corresponding plurality of directions.

Claims

1. An apparatus to process a plurality of input audio signals comprising:

a plurality of input terminals to accept a plurality of input signals;

a multi-input, multi-output reverberator accepting the plurality of input signals and arranged to generate a set of output signals, including delayed reverberation components simulating the reverberations a listener is likely to hear in a listening environment; and

a multi-input, two-output filter with inputs coupled to the outputs of the reverberator, the inputs further coupled to the plurality of input terminals, the filter having two outputs, one for the left ear and one for the right ear, the filter arranged to implement a set of head related transfer functions corresponding to a listening environment and a set of directions of a listener in the listening environment, the two outputs playable through headphones,

such that a listener listening to the left and right output signals in the listening environment through headphones has the sensation of listening to the plurality of input audio signals as if they are emanating from a plurality of loudspeakers spatially located in the listening environment to form a corresponding plurality of directions for the listener.

2. An apparatus as recited in claim 1, wherein the reverberator is arranged in the forming of at least one of the reverberation components to combine a plurality of the accepted input signals, and wherein the reverberator is further arranged to process each of the input signals differently.

3. An apparatus as recited in claim 2, further comprising:

a first set of combiners coupled to the inputs of the reverberator and to the input terminals, arranged to combine the plurality of inputs with the set of reverberator outputs to generate a set of inputs for the multi-input, multi-output filter.

4. An apparatus as recited in claim 2, wherein the filter is arranged to generate two sets of outputs, one set for the left ear, and one set for the right ear, and wherein the filter includes a second set of combiners arranged to combine the left ear set of outputs and the right ear set of outputs to form the left-ear output signal and the right-ear output signal, respectively.

5. An apparatus as recited in claim 2, wherein the reverberator is arranged such that that the reverberation components include a series of mixed, delayed and filtered versions of the accepted input signals.

6. An apparatus as recited in claim 5, wherein the reverberator includes a multi-input, multi-output mixer with inputs coupled to the input terminals, the mixer arranged to mix the plurality of input signals, the mixing describable by a non-diagonal matrix, such that at least one mixer output is generated by combining a plurality of mixer inputs.

7. An apparatus as recited in claim 6, wherein the matrix is a unitary matrix to within pre-multiplying by a diagonal matrix and/or post-multiplying by a diagonal matrix.

8. An apparatus as recited in claim 6, wherein the coupling of the mixer inputs to the input terminals is via a third set of combiners arranged to combine the inputs with delayed filtered versions of the mixer outputs, such that the reverberator includes a plurality of feedback signal paths, with at least one feedback signal path including a delay and filter.

9. An apparatus as recited in claim 1, wherein the multi-input two-output filter is arranged to implement a plurality of HRTF filter pairs for a corresponding plurality of HRTF directions, one pair for each formed direction for the listener, and wherein the reverberator includes a plurality of feedback signal paths, one for each formed direction for the listener, such that the coupling of the reverberator outputs to the multi-input, two-output filter couples each of the feedback signal paths to a corresponding one of the HRTF filter pairs.

10. An apparatus as recited in claim 9, wherein the reverberator further includes a multi-input, multi-output mixer with inputs coupled to the input terminals and to the outputs of the feedback signal paths, the mixer arranged to mix the plurality of inputs, the mixer outputs coupled to the feedback signal paths, the mixing describable by a non-diagonal matrix, such that at least one mixer output is generated by combining a plurality of mixer inputs.

11. An apparatus as recited in claim 9, wherein each of the feedback signal paths includes a delay and filter, each respective filter to produce a desired decay rate that varies with the frequency for echoes produced by the respective feedback signal path, and each respective delay being selected to provide a desired reverberation pattern for the listening environment.

12. An apparatus as recited in claim 11, wherein each filter is selected to achieve a desired reverberation time at low frequencies and a desired reverberation time at high frequencies.

13. An apparatus as recited in claim 11, wherein the delays of the different feedback signal paths are selected to be different with no common factor.

14. An apparatus as recited in claim 13, wherein each of the delays of the different feedback signal paths is selected to be approximately equal to the delay time of the first echo arrival in the listening environment.

15. An apparatus as recited in claim 13, wherein each of the delays of the different feedback signal paths is selected such that the patterns of echoes are uncorrelated for each feedback signal path.

16. An apparatus as recited in claim 9, wherein the number of HRTF directions is less than the number of input signals in the plurality of audio input signals.

17. An apparatus as recited in claim 9, wherein the number of HRTF directions is greater than the number of input signals in the plurality of audio input signals.

18. An apparatus as recited in claim 1, wherein the apparatus further includes a memory arranged to store at least one set of parameters for at least one listening environment, each set sufficient to simulate a listening environment.

19. An apparatus as recited in claim 18, wherein the memory is loaded with a plurality of sets of parameters for a plurality of sets of listening environments.

20. An apparatus as recited in claim 1, wherein the filter and reverberator are implemented by a DSP system having a memory.

21. A method to process a plurality of input audio signals comprising:

accepting a plurality of input signals;

generating a set of reverberator output signals from the plurality of input signals, the generating including forming delayed reverberation components that simulate the reverberations a listener is likely to hear in a listening environment; and

filtering combinations of the input signals and reverberator output signals to produce two outputs, one for the left ear and one for the right ear, the filter implementing a set of head related transfer functions corresponding to a listening environment and a set of directions of a listener in the listening environment, the two outputs playable through headphones,

such that a listener listening to the left and right output signals in the listening environment through headphones has the sensation of listening to the plurality of input audio signals as if they are emanating from a plurality of loudspeakers spatially located in the listening environment to form a corresponding plurality of directions for the listener.

22. A method as recited in claim 21, wherein the forming of at least one of the reverberation components includes combining a plurality of the accepted input signals, and wherein the generating of a set of reverberator output signals processes different input signals differently.

23. A method as recited in claim 22, further comprising:

combining the plurality of inputs with the set of reverberator outputs to generate a set of inputs for the reverberating.

24. A method as recited in claim 22, wherein the reverberation components include a series of mixed, delayed and filtered versions of the accepted input signals.

25. A method as recited in claim 24, wherein the generating of the set of reverberator output signals includes mixing the plurality of input signals, the mixing describable by a non-diagonal matrix, such that at least one mixing output is generated by combining a plurality of mixing inputs.

26. A method as recited in claim 25, wherein the matrix is a unitary matrix to within pre-multiplying by a diagonal matrix and/or post-multiplying by a diagonal matrix.

27. A method as recited in claim 25, wherein the generating of the set of reverberator output signals includes combining the accepted inputs with delayed filtered versions of the mixer outputs, such that the generating of the set of reverberator output signals includes providing a plurality of feedback signal paths, with at least one feedback signal path including delaying and filtering.

28. A method as recited in claim 21,

wherein the filtering implements a plurality of HRTF filter pairs for a corresponding plurality of HRTF directions, one pair for each formed direction for the listener,

wherein the generating of reverberator outputs includes providing a plurality of feedback signal paths, one for each formed direction for the listener,

and wherein the method further includes coupling each of the feedback signal paths to a corresponding one of the HRTF filter pairs.

29. A method as recited in claim 28, wherein the generating of a set of reverberator output signals further includes mixing the accepted inputs with the outputs of the feedback signal paths to produce inputs to the feedback signal paths, the mixing describable by a non-diagonal matrix, such that at least one mixing output is generated by combining a plurality of mixing inputs.

30. A method as recited in claim 28, wherein each of the feedback signal paths includes delaying and filtering, wherein each filter step is to produce a desired decay rate that varies with the frequency for echoes produced by the respective feedback signal path, and wherein each respective delaying step includes applying a respective delay selected to provide a desired reverberation pattern for the listening environment.

31. A method as recited in claim 30, wherein the filtering in each feedback signal path is selected to achieve a desired reverberation time at low frequencies and a desired reverberation time at high frequencies.

32. A method as recited in claim 30, wherein the delays of the different feedback signal paths are selected to be different with no common factor.

33. A method as recited in claim 32, wherein each of the delays of the different feedback signal paths is selected to be approximately equal to the delay time of the first echo arrival in the listening environment.

34. A method as recited in claim 32, wherein each of the delays of the different feedback signal paths is selected such that the patterns of echoes are uncorrelated for each feedback signal path.

35. A method as recited in claim 28, wherein the number of HRTF directions is less than the number of input signals in the plurality of audio input signals.

36. A method as recited in claim 28, wherein the number of HRTF directions is greater than the number of input signals in the plurality of audio input signals.

37. A carrier medium carrying at least one code segment to instruct at least one processor of a processing system to implement a method, the method to process a plurality of input audio signals, the method comprising:

accepting a plurality of input signals;

generating a set of reverberator output signals from a plurality of input signals, the generating including forming delayed reverberation components simulating the reverberations a listener is likely to hear in a listening environment; and

filtering combinations of the input signals and reverberator output signals to produce two outputs, one for the left ear and one for the right ear, the filter implementing a set of head related transfer functions corresponding to a listening environment and a set of directions of a listener in the listening environment, the two outputs playable through headphones,

such that a listener listening to the left and right output signals in the listening environment through headphones has the sensation of listening to the plurality of input audio signals as if they are emanating from a plurality of loudspeakers spatially located in the listening environment to form a corresponding plurality of directions for the listener.

38. A carrier medium as recited in claim 37, wherein the forming of at least one of the reverberation components includes combining a plurality of the accepted input signals, and wherein the generating of a set of reverberator output signals processes different input signals differently.

39. A carrier medium as recited in claim 38, wherein the reverberation components include a series of mixed, delayed and filtered versions of the accepted input signals.

40. A carrier medium as recited in claim 39, wherein the generating of the set of reverberator output signals includes mixing the plurality of input signals, the mixing describable by a non-diagonal matrix, such that at least one mixing output is generated by combining a plurality of mixing inputs.

41. A carrier medium as recited in claim 40, wherein the generating of the set of reverberator output signals includes combining the accepted inputs with delayed filtered versions of the mixer outputs, such that the generating of the set of reverberator output signals includes providing a plurality of feedback signal paths, with at least one feedback signal path including delaying and filtering.

42. A carrier medium as recited in claim 37,

wherein the filtering implements a plurality of HRTF filter pairs for a corresponding plurality of HRTF directions, one pair for each formed direction for the listener,

wherein the generating of reverberator outputs uses a plurality of feedback signal paths, one for each formed direction for the listener,

and wherein the method further includes coupling each of the feedback signal paths to a corresponding one of the HRTF filter pairs.

43. A carrier medium as recited in claim 42, wherein the generating of a set of reverberator output signals further includes mixing the accepted inputs with the outputs of the feedback signal paths to produce inputs to the feedback signal paths, the mixing describable by a non-diagonal matrix, such that at least one mixing output is generated by combining a plurality of mixing inputs.

44. A carrier medium as recited in claim 42, wherein each of the feedback signal paths includes delaying and filtering, wherein each filter step is to produce a desired decay rate that varies with the frequency for echoes produced by the respective feedback signal path, and wherein each respective delaying step includes applying a respective delay selected to provide a desired reverberation pattern for the listening environment.

45. A carrier medium as recited in claim 44, wherein the filtering in each feedback signal path is selected to achieve a desired reverberation time at low frequencies and a desired reverberation time at high frequencies.

46. A carrier medium as recited in claim 44, wherein the delays of the different feedback signal paths are selected to be different with no common factor.

47. A carrier medium as recited in claim 46, wherein each of the delays of the different feedback signal paths is selected to be approximately equal to the delay time of the first echo arrival in the listening environment.

48. A carrier medium as recited in claim 46, wherein each of the delays of the different feedback signal paths is selected such that the patterns of echoes are uncorrelated for each feedback signal path.

49. A carrier medium as recited in claim 42, wherein the number of HRTF directions is less than the number of input signals in the plurality of audio input signals.

50. A carrier medium as recited in claim 42, wherein the number of HRTF directions is greater than the number of input signals in the plurality of audio input signals.

51. An apparatus to process a plurality of input audio signals comprising:

means for accepting a plurality of input signals;

means for generating a set of reverberator output signals from a plurality of input signals, including forming delayed reverberation components simulating the reverberations a listener is likely to hear in a listening environment; and

means for filtering combinations of the input signals and reverberator output signals to produce two outputs, one for the left ear and one for the right ear, the filter implementing a set of head related transfer functions corresponding to a listening environment and a set of directions of a listener in the listening environment, the two outputs playable through headphones,

such that a listener listening to the left and right output signals in the listening environment through headphones has the sensation of listening to the plurality of input audio signals as if they are emanating from a plurality of loudspeakers spatially located in the listening environment to form a corresponding plurality of directions for the listener.

52. An apparatus as recited in claim 51, wherein the forming of at least one of the reverberation components includes combining a plurality of the accepted input, and wherein the generating of a set of reverberator output signals processes different input signals differently.