Perfected device and method for the spatialization of sound

• US 20050114121A1
• Filed: 12/31/2003
• Published: 05/26/2005
• Est. Priority Date: 11/26/2003
• Status: Active Grant

• Current Assignee: Inria Institut National de Recherche EN Informatique et EN Automatique
• Original Assignee: Inria Institut National de Recherche EN Informatique et EN Automatique
• Inventors: Gallo, Emmanuel, Tsingos, Nicolas, Drettakis, George
• Granted Patent: US 7,356,465 B2
• Time in Patent Office: Days
• US Class Current: 704/220
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 7/30 Control circuits for electronic adaptation of the sound field

Abstract

The invention relates to a computer device comprising a memory 108 for storing audio signals 114, in part pre-recorded, each corresponding to a defined source, by means of spatial position data 116, and a processing module 110 for processing these audio signals in real time as a function of the spatial position data. The processing module 110 allows for the instantaneous power level parameters to be calculated on the basis of audio signals 114, the corresponding sources being defined by instantaneous power level parameters. The processing module 110 comprises a selection module 120 for regrouping certain of the audio signals into a variable number of audio signal groups, and the processing module 110 is capable of calculating spatial position data which is representative of a group of audio signals as a function of the spatial position data 116 and instantaneous power level parameters for each corresponding source.

Claims

1. A computer device comprising:

A memory (8, 108), capable of storing audio signals (14, 114) in part pre-recorded, each corresponding to a source defined by spatial position data (16, 116), and a processing module (10, 110) to process these signals in real time as a function of the spatial position data,

characterised in that

the processing module (110) is capable of calculating instantaneous power level parameters on the basis of the audio signals (14), the corresponding sources being defined by said parameters of the instantaneous power level,

in that the processing module (110) comprises a selection module (120) capable of grouping together certain of the audio signals into a variable number of audio signal groups, and that the processing module (110) is capable of calculating the representative spatial position data of a group of audio signals as a function of the spatial position data (116) and the parameters of the instantaneous power levels of each corresponding source;

2. A computer device according to claim 1, characterised in that the selection module (120) is capable, prior to the formation of the audio signal groups, of selecting inaudible audio signals as a function of the parameters of the instantaneous power levels, comprising a power level P_k^T-δ

(f)) and a masking threshold (Mt(f)) for each source, and of preserving audible audio signals only.

3. A computer device according to claim 2, characterised in that the parameters of the power level are calculated for each source on the basis of an instantaneous power spectral density (PSD), pre-calculated on the basis of the audio signals in part pre-recorded.

4. A computer device according to claim 1, characterised in that the processing module (110) is capable of processing each group of audio signals into one pre-mixing audio signal, and of re-assembling the pre-mixing audio signals in order to obtain a mixing signal which is audible to a listener.

5. A computer device according to claim 1, characterised in that the processing module (110) comprises a video processor (130) capable of transforming the group of audio signals into a group of textured video signals, of processing each textured video signal of the group according to sound modification parameters, and of reassembling and transforming the signals from the group into one pre-mixing audio signal.

6. A computer device according to claim 5, characterised in that the sound modification parameters comprise a sound attenuation parameter and/or a sound propagation delay parameter.

7. A computer device according to claim 1, characterised in that the selection module (120) is capable of forming, on the basis of a first group of audio signals and calculated group spatial position data, two groups of audio signals and of calculating the spatial position data of a representative of each of these two groups.

8. A computer device according to claim 7, characterised in that the selection module (120) is capable of determining, on the basis of the first group of audio signals, their corresponding sources and calculated data of the spatial position of the representative of the first group, a source for which the sum of the calculated error distances between the spatial position of this source and those of the other sources of the group is minimal, and of attributing the audio signals from the first group and their corresponding sources to one of the spatial positions among the calculated spatial position data of the representative of the first group and the spatial position of the source which has been determined, as a function of evaluations of error distance, in such a way as to form two groups.

9. A computer device according to claim 8, characterised in that the selection module is capable of carrying out an error distance evaluation for an audio signal from the first group and its corresponding source, consisting of the evaluation on the one hand of the error distance between the spatial position data of this source and the calculated data of the spatial position of the representative of the first group, and, on the other, the error distance between the spatial position data of this source and the spatial position data of the source which has been determined, then of evaluating the minimum distance between them, the selection module being capable of attributing the audio signal and its corresponding source to the spatial position data of the source which has been determined or of the representative of the first group corresponding to the minimum error distance.

10. A computer device according to claim 7, characterised in that the spatial position data of the source which has been determined corresponds to the spatial position data of the representative of the second group.

11. A computer device according to claim 7, characterised in that the selection module (120) is capable of calculating the spatial position data of each group representative as a function of power level parameters of each source attributed to the group.

12. A computer device according to claim 7, characterised in that the selection module (120) is capable of calculating the spatial position data of the representative of each of the two groups, by determining a source for which the sum of the error distances between the spatial position of this source and those of the other sources of the group is minimal, and the selection module (120) is also capable of re-attributing the sources to one or the other of the representatives of one of the two groups as a function of an evaluation of the minimum error distance.

13. A computer device according to claim 12, characterised in that the selection module (120) is capable of recalculating the spatial position data of the representatives of each of the two groups and of re-attributing the sources to one or the other of the representatives of the two groups until the sum of the error distances between the representatives of the two groups and their sources attain a minimum.

14. A computer device according to claim 7, characterised in that the selection module (120) is capable of dividing a group until a predetermined number of groups is obtained or until the sum of the error distances between the representatives of the groups and their sources attains a predetermined threshold.

15. A method of processing audio signals in part pre-recorded, each corresponding to one source, comprising stages consisting of:

a. Calculating instantaneous power level parameters on the basis of audio signals, corresponding sources being defined by these parameters and by spatial position data;

b. Regrouping certain of the audio signals into a variable number of audio signal groups and calculating spatial position data representatives of each group of audio signals as a function of the spatial position data and the instantaneous power level parameters of each corresponding source (204);

c. Processing these audio signals per group, in real time, as a function of the spatial position data representative of the group (206, 208).

16. A method according to claim 15, characterised in that the stage a. additionally comprises the selection of inaudible audio signals as a function of the instantaneous power level parameters, comprising a power level and a masking threshold for each source and preserving audible audio signals only (200).

17. A method according to claim 16, characterised in that the power level parameters are calculated for each source on the basis of an instantaneous power spectral density, pre-calculated on the basis of the audio signals, which are in part pre-recorded.

18. A method according to claim 15, characterised in that the stage c. consists of:

c1. Processing each group of audio signals into one pre-mixing audio signal (206);

c2. Re-assembling the pre-mixing audio signals in order to obtain a mixing signal, which is audible to a listener (208).

19. A method according to claim 18, characterised in that the stage c1. additionally consists of transforming a group of audio signals into a group of textured video signals by making use of a video processor (2020), of processing each textured video signal of the group according to sound modification parameters (2022, 2024), and of re-assembling and transforming the group signals into one pre-mixing audio signal (2026).

20. A method according to claim 19, characterised in that the sound modification parameters comprise a sound attenuation parameter and/or a sound propagation delay parameter.

21. A method according to claim 15, characterised in that the stage b. additionally consists of forming, on the basis of a first group of audio signals and calculated spatial position data of the group (2000), two groups of audio signals, and of calculating the spatial position data of a representative of each of these two groups (2002 to 2012).

22. A method according to claim 21, characterised in that the stage b. additionally consists of determining, on the basis of the first group of audio signals, their corresponding sources, and calculated data of the spatial position of the representative of the first group, a source for which the sum of error distances calculated between the spatial position of this source and those of other sources of the first group is minimal (2002), and of attributing the audio signals of the first group and their corresponding sources to one of the spatial positions, among the calculated data of the spatial position of the representative of the first group and the spatial position data of the source which has been determined, as a function of the evaluation of the error distance, in such a way as to form two groups (2004).

23. A method according to claim 22, characterised in that the distance evaluation from stage b. consists, for an audio signal of the first group and its corresponding source, of evaluating on the one hand the error distance between the spatial position data of this source and the calculated data of the spatial position of the representative of the first group (A9) and, on the other, the error distance between the spatial position data of this source and the spatial position data of the source which has been determined, then of evaluating the minimum error distance between the two, and of attributing the audio signal and its corresponding source to the spatial position data of the source which has been determined or of the representative of the first group corresponding to the minimum error distance (2004).

24. A method according to claim 21, characterised in that the spatial position data of the source which has been determined from stage b. corresponds to the spatial position data of the representative of the second group.

25. A method according to claim 21, characterised in that stage b. consists likewise of re-calculating the spatial position data of the representative of each of these two groups (2006) and of re-attributing the sources to one or the other of the two groups (2008), until the sum of the error distances between the representatives of the two groups and their sources attains a minimum (2010).

26. A method according to claim 21, characterised in that stage b. consists of dividing a group until a predetermined number of groups is obtained or until the sum of the error distances between the representatives of the groups and their sources attains a predetermined threshold (2012).