Reverberation Generation for Headphone Virtualization

• US 20180035233A1
• Filed: 02/11/2016
• Published: 02/01/2018
• Est. Priority Date: 02/12/2015
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: FIELDER, Louis D., SHUANG, Zhiwei, DAVIDSON, Grant A., ZHENG, Xiguang, VINTON, Mark S.
• Granted Patent: US 10,149,082 B2
• Time in Patent Office: Days
• CPC Class Codes: G10K 15/08 Arrangements for producing a reverberation or echo sound modifying acoustic properties to change reverberation time G10K11/002
• CPC Class Codes: H04S 2400/01 Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
• CPC Class Codes: H04S 2420/01 Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
• CPC Class Codes: H04S 3/004 For headphones
• CPC Class Codes: H04S 5/005 of the pseudo five- or more-channel type, e.g. virtual surround
• CPC Class Codes: H04S 7/302 Electronic adaptation of stereophonic sound system to listener position or orientation H04S7/301 takes precedence
• CPC Class Codes: H04S 7/304 For headphones

Abstract

The present disclosure relates to reverberation generation for headphone virtualization. A method of generating one or more components of a binaural room impulse response (BRIR) for headphone virtualization is described. In the method, directionally-controlled reflections are generated, wherein directionally-controlled reflections impart a desired perceptual cue to an audio input signal corresponding to a sound source location. Then at least the generated reflections are combined to obtain the one or more components of the BRIR. Corresponding system and computer program products are described as well.

Claims

1. A method for generating one or more components of a binaural room impulse response (BRIR) for headphone virtualization, comprising:

generating directionally-controlled reflections that impart a desired perceptual cue to an audio input signal corresponding to a sound source location; and

combining at least the generated reflections to obtain the one or more components of the BRIR.

2. The method of claim 1, wherein the desired perceptual cues lead to a natural sense of space with minimal audible impairments.

3. The method of claim 1, wherein the directionally-controlled reflections have a predetermined direction of arrival in which an illusion of a virtual sound source at a given location in space is enhanced.

4. The method of claim 3, wherein the predetermined direction of arrival is of a wobble shape in which reflection directions change away from a virtual sound source and oscillate back and forth therearound.

5. The method of claim 3, wherein the predetermined direction of arrival further comprises a stochastic diffuse component within a predetermined azimuths range, and wherein at least one of the wobble shapes or the stochastic diffuse components is selected based on a direction of the virtual sound source.

6. The method of claim 1, wherein generating directionally-controlled reflections comprises:

determining respective occurrence time points of the reflections scholastically under a predetermined echo density distribution constraint;

determining desired directions of the reflections based on the respective occurrence time points and the predetermined directional pattern;

determining amplitudes of the reflections at the respective occurrence time points scholastically; and

creating the reflections with the desired directions and the determined amplitudes at the respective occurrence time points.

7. The method of claim 6, wherein creating the directionally-controlled reflections comprises:

selecting, from head-related transfer function (HRTF) data sets measured for particular directions, HRTFs based on the desired directions at the respective occurrence time points; and

modifying the HRTFs based on the amplitudes of the reflections at the respective occurrence time points so as to obtain the reflections at the respective occurrence time points.

8. The method of claim 6, wherein creating the directionally-controlled reflections comprises:

determining HRTFs based on the desired directions at the respective occurrence time points and a predetermined spherical head model; and

modifying the HRTFs based on the amplitudes of the reflections at the respective occurrence time points so as to obtain the reflections at the respective occurrence time points.

9. The method of claim 5, wherein creating the directionally-controlled reflections comprises:

generating impulse responses for two ears based on the desired directions and the determined amplitudes at the respective occurrence time points and based on broadband interaural time difference and interaural level difference of a predetermined spherical head model.

10. The method of claim 9, wherein creating the directionally-controlled reflections further comprises:

filtering the created impulse responses for two ears through all-pass filters to obtain a diffusion and decorrelation.

11. The method of claim 1, wherein the method is operated in a feedback delay network, and wherein generating reflections comprises filtering the audio input signal through HRTFs, so as to control at least directions of an early part of late responses to impart desired perceptual cues to the audio input signal.

12. The method of claim 11, wherein the audio input signal is delayed by delay lines before it is filtered by the HRTFs.

13. The method of claim 11, wherein the audio input signal is filtered before signals fed back through at least one feedback matrix are added.

14. The method of claim 11, wherein the audio input signal is filtered by the HRTFs in parallel with the audio input signal being inputted into the feedback delay network, and wherein output signals from the feedback delay network and from the HRTFs are mixed to obtain the reverberation for headphone virtualization.

15. The method of claim 11, wherein for multiple audio channels or objects, an input audio signal for each of the multiple audio channels or objects is separately filtered by the HRTFs.

16. The method of claim 11, wherein for multiple audio channels or objects, input audio signals for the multiple audio channels or objects are downmixed and analyzed to obtain an audio mixture signal with a dominant source direction, which is taken as the audio input signal.

17. The method of claim 1, further comprising performing an optimal process by:

repeating the generating reflections to obtain a plurality of groups of reflections and selecting one of the plurality of groups of reflections having an optimal reflection characteristic as the reflections for the audio input signal;

or

repeating the generating reflections till a predetermined reflection characteristic is obtained.

18. The method of claim 17, wherein the generating reflections is driven in part by at least some of random variables generated based on a stochastic mode.

38. A computer program product of reverberation generation for headphone virtualization, the computer program product being tangibly stored on a non-transient computer-readable medium and comprising machine executable instructions which, when executed, cause the machine to perform steps of the method according to any one of claims 1 to 18.

19. A system for generating one or more components of a binaural room impulse response (BRIR) for headphone virtualization, comprising:

a reflection generation unit, configured to generate directionally-controlled reflections that impart a desired perceptual cue to an audio input signal corresponding to a sound source location; and

a mixing unit, configured to combine at least the generated reflections to obtain the one or more components of the BRIR.

20. The system of claim 19, wherein the desired perceptual cues lead to a natural sense of space with minimal audible impairments.

21. The system of claim 20, wherein the directionally-controlled reflections have a predetermined directional pattern in which an illusion of a virtual sound source at a given location in space is enhanced.

22. The system of claim 21, wherein the predetermined direction of arrival is of a wobble shape in which directions of reflections change away from a virtual sound source and oscillate back and forth therearound.

23. The system of claim 21, wherein the predetermined direction of arrival further comprises a stochastic diffuse component within a predetermined azimuths range, and wherein the wobble shapes and/or the stochastic diffuse component are selected based on a direction of the virtual sound source.

24. The system of claim 19, wherein the reflection generation unit is configured to:

determine respective occurrence time points of the reflections scholastically determined within a predetermined echo density distribution constrain;

determine desired directions of the reflections based on the respective occurrence time points and the predetermined directional pattern;

determine amplitudes of the reflections at the respective occurrence time points scholastically;

create reflections with the desired directions and the determined amplitudes at the respective occurrence time points.

25. The system of claim 24, wherein the reflection generation unit is configured to create the reflections by:

selecting, from head-related transfer function (HRTF) data sets measured for particular directions, HRTFs based on the desired directions at the respective occurrence time points; and

modifying the HRTFs based on amplitudes of the reflections at the respective occurrence time points so as to obtain the reflections at the respective occurrence time points.

26. The system of claim 24, wherein the reflection generation unit is configured to create the reflections by:

determining HRTFs based on the desired directions at the respective occurrence time points and a predetermined spherical head model; and

modifying the HRTFs based on amplitudes of the reflections at the respective occurrence time points so as to obtain the reflections at the respective occurrence time points.

27. The system of claim 24, wherein the reflection generation unit is configured to create the reflections by:

generating impulse responses for two ears based on the desired directions and the determined amplitudes at the respective occurrence time points and broadband interaural time difference and interaural level difference of a predetermined spherical head model.

28. The system of claim 27, the reflection generation unit is configured to create the reflections further by:

filtering the created impulse responses for two ears through all-pass filters to obtain a diffusion and decorrelation.

29. The system of claim 19, wherein the system is implemented in a feedback delay network and wherein the reflection generation unit is configured to filter the audio input signal through HRTFs, so as to control at least directions of early part of late responses to impart desired perceptual cues to the input signal.

30. The system of claim 29, wherein the reflection generation unit is configured to delay the audio input signal by delay lines before it is filtered by the HRTFs.

31. The system of claim 29, wherein the reflection generation unit is configured to filter the audio input signal before signals fed back through at least one feedback matrix are added.

32. The system of claim 29, wherein the reflection generation unit is configured to filter the audio input signal by the HRTF in parallel with the audio input signal being inputted into the feedback delay network, and wherein output signals from the feedback delay network and from the HRTFs are mixed to obtain the reverberation for headphone virtualization.

33. The system of claim 29, wherein the reflection generation unit is configured to, for multiple audio channels or objects, separately filter an input audio signal for each of the multiple audio channels or objects by the HRTFs.

34. The system of claim 29, wherein the reflection generation unit is configured to, for multiple audio channels or objects, downmix and analyze input audio signals for the multiple audio channels or objects to obtain an audio mixture signal with a dominant source direction and filter the mixture audio signal as the audio input signal.

35. The system of claim 19, wherein the reflection generation unit is operated in an optimal process in which the reflection generation unit is operated repeatedly to obtain a plurality of groups of reflections and one of the plurality of groups of reflections having an optimal reflection characteristic is selected as the reflections for the input audio signal, or in which the reflection generation unit is operated repeatedly till a predetermined reflection characteristic is obtained.

36. The system of claim 35, wherein the generating reflections is driven in part by at least some of random variables generated based on a stochastic mode.

37. A method for generating left-ear and right-ear binaural signals from one or more audio input signals for headphone presentation comprising:

determining a sound source location corresponding to each of said one or more audio input signals;

convolving each of said one or more audio input signals with one or more components of a BRIR corresponding to the sound source location to obtain left-ear and right-ear intermediate signals, wherein at least one of said components of the BRIR comprises directionally-controlled reflections that impart a desired perceptual cue to said one or more audio input signals respectively;

combining the left-ear intermediate signals to produce the left-ear binaural signal and combining the right-ear intermediate signals to produce the right-ear binaural signal.