Leveling audio signals

• US 9,575,715 B2
• Filed: 05/16/2008
• Issued: 02/21/2017
• Est. Priority Date: 05/16/2008
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Duwenhorst, Sven, Classen, Holger
• Primary Examiner(s): Lerner, Martin
• Application Number:
  US12/122,629
• Publication Number:
  US 20150205571A1
• Time in Patent Office: 3,203 Days
• Field of Search: 704/203, 704/211, 704/225, 704/226, 704/228, 704/502, 455/232.1, 455/242.1, 455/245.1, 381/94.5, 381/106, 381/107, 381/104, 708/202, 708/304
• US Class Current: 1/1
• CPC Class Codes: G05B 15/02 electric
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G11B 27/00 Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals

Abstract

The present specification relates to systems, methods, and apparatus including computer program products for editing digital audio data. In some implementations, a method is provided that includes the actions of receiving digital audio data; dividing the audio data into first audio data and second audio data; performing on-line processing of the first and second audio data including segmenting the first audio data according to one or more partitioning parameters into discrete time partitions, sequentially analyzing for each discrete time partition the first audio data of the corresponding discrete time partition, and modifying the second audio data according to the analysis of the first audio data; and outputting the modified second audio data.

Claims

1. A computer-implemented method comprising:

receiving digital audio data;

dividing the audio data into first audio data and second audio data;

segmenting a first portion of the first audio data into a plurality of discrete time partitions according to one or more partitioning parameters;

detecting, by data processing apparatus, a plurality of root mean square (RMS) values for the audio data of the plurality of discrete time partitions, wherein the plurality of RMS values include a distinct RMS value for the audio data of each respective time partition;

storing the plurality of RMS values for the audio data of the plurality of discrete time partitions in a first data structure;

sorting the plurality of RMS values according to a sorting criteria, wherein the sorted RMS values are stored in a second data structure;

selecting, from the sorted RMS values stored in the second data structure, an RMS value that is not the highest of the RMS values stored in the second data structure;

determining a current noise floor value according to a plurality of noise estimator parameters including a previous noise floor value, a damping factor, and a second RMS value that is the lowest of the RMS values stored in the second data structure;

using the selected RMS value to determine a gain adjustment for at least a portion of the second audio data corresponding to a first time partition of the first portion of the first audio data;

modifying the portion of the second audio data according to the gain adjustment based on the current noise floor value; and

outputting the modified second audio data.

2. The computer-implemented method of claim 1, wherein the partitioning parameters include one or more of a partition count, a partition size, or an offset value.

3. The computer-implemented method of claim 1, further comprising:

determining a time delay according to the partitioning parameters; and

delaying the second audio data by the time delay.

4. The computer-implemented method of claim 1, further comprising determining one or more of a peak value or a crest factor for the first audio data associated with each discrete time partition.

5. The computer-implemented method of claim 4, wherein storing the plurality of RMS values in the first data structure includes:

sequentially inserting, for each discrete time partition, an element into the first data structure, the element including one or more of the RMS value, the peak value, or the crest factor.

6. The computer-implemented method of claim 5, further comprising:

using the first data structure according to first data structure parameters, the first data structure parameters being determined according to the partitioning parameters, the first data structure parameters including a data structure size and an index, the first data structure being a first-in-first-out data structure having a first and a last element.

7. The computer-implemented method of claim 1, further comprising:

using the second data structure according to second data structure parameters, the second data structure parameters being determined according to the partitioning parameters, the second data structure parameters including a data structure size and an index.

8. The computer-implemented method of claim 1, wherein the using of the selected RMS value to determine the gain adjustment comprises:

identifying a mapping table including mappings from input data to output data;

mapping, using the mapping table, the selected RMS value to an output value; and

computing a ratio of the output value to the selected RMS value, wherein the gain adjustment is based on the ration.

9. The computer-implemented method of claim 1, further comprising:

using a noise gate having one or more noise gate parameters; and

controlling the noise gate parameters according to the noise estimator parameters at intervals determined according to the partitioning parameters.

10. The computer-implemented method of claim 1, wherein:

the sorted RMS values are stored in the second data structure in an ascending order such that the highest of the sorted RMS values is stored in a highest indexed element of the second data structure; and

the selecting of the RMS value comprises selecting the RMS value from an element other than the highest indexed element.

11. The computer-implemented method of claim 1, wherein:

the sorted RMS values are stored in the second data structure in an ascending order such that a lowest of the sorted RMS values is stored in a lowest indexed element of the second data structure; and

the method further comprises retrieving the second RMS value from the lowest indexed element.

12. A computer program product, tangibly stored on a non-transitory computer readable medium, comprising instructions that when executed on a server cause the server to perform operations comprising:

receiving digital audio data;

dividing the audio data into first audio data and second audio data;

segmenting a first portion of the first audio data into a plurality of discrete time partitions according to one or more partitioning parameters;

detecting a plurality of root mean square (RMS) values for the audio data of the plurality of discrete time partitions, wherein the plurality of RMS values include a distinct RMS value for the audio data of each respective time partition;

storing the plurality of RMS values for the audio data of the plurality of discrete time partitions in a first data structure;

sorting the plurality of RMS values according to a sorting criteria, wherein the sorted RMS values are stored in a second data structure;

selecting, from the sorted RMS values stored in the second data structure, an RMS value that is not the highest of the RMS values stored in the second data structure;

determining a current noise floor value according to a plurality of noise estimator parameters including a previous noise floor value, a damping factor, and a second RMS value that is the lowest of the RMS values stored in the second data structure;

using the selected RMS value to determine a gain adjustment for at least a portion of the second audio data corresponding to a first time partition of the first portion of the first audio data;

modifying the portion of the second audio data according to the gain adjustment based on the current noise floor value; and

outputting the modified second audio data.

13. The computer program product of claim 12, wherein the partitioning parameters include one or more of a partition count, a partition size, or an offset value.

14. The computer program product of claim 12, further comprising instructions to perform operations including:

determining a time delay according to the partitioning parameters; and

delaying the second audio data by the time delay.

15. The computer program product of claim 12, further comprising determining one or more of a peak value or a crest factor for the first audio data associated with each discrete time partition.

16. The computer program product of claim 15, wherein storing the plurality of RMS values in the first data structure includes:

sequentially inserting, for each discrete time partition, an element into the first data structure, the element including one or more of the RMS value, the peak value, or the crest factor.

17. The computer program product of claim 16, further comprising instructions to perform operations including:

using the first data structure according to first data structure parameters, the first data structure parameters being determined according to the partitioning parameters, the first data structure parameters including a data structure size and an index, the first data structure being a first-in-first-out data structure having a first and a last element.

18. The computer program product of claim 12, further comprising instructions to perform operations including:

using the second data structure according to second data structure parameters, the second data structure parameters being determined according to the partitioning parameters, the second data structure parameters including a data structure size and an index.

19. The computer program product of claim 12, wherein the using of the selected RMS value to determine the gain adjustment comprises:

identifying a mapping table including mappings from input data to output data;

mapping, using the mapping table, the selected RMS value to an output value; and

computing a ratio of the output value to the selected RMS value, wherein the gain adjustment is based on the ratio.

20. The computer program product of claim 12, further comprising instructions to perform operations including:

using a noise gate having one or more noise gate parameters; and

controlling the noise gate parameters according to the noise estimator parameters at intervals determined according to the partitioning parameters.

21. A system comprising:

one or more computers configured to perform operations including;

receiving digital audio data;

dividing the audio data into first audio data and second audio data;

segmenting a first portion of the first audio data into a plurality of discrete time partitions according to one or more partitioning parameters;

detecting a plurality of root mean square (RMS) values for the audio data of the plurality of discrete time partitions, wherein the plurality of RMS values include a distinct RMS value for the audio data of each respective time partition;

storing the plurality of RMS values for the audio data of the plurality of discrete time partitions in a first data structure;

sorting the plurality of RMS values according to a sorting criteria, where the sorted RMS values are stored in a second data structure;

selecting, from the sorted RMS values stored in the second data structure, an RMS value that is not the highest of the RMS values stored in the second data structure;

determining a current noise floor value according to a plurality of noise estimator parameters including a previous noise floor value, a damping factor, and a second RMS value that is the lowest of the RMS values stored in the second data structure;

using the selected RMS value to determine a gain adjustment for at least a portion of the second audio data corresponding to a first time partition of the first portion of the first audio data;

modifying the portion of the second audio data according to the gain adjustment based on the current noise floor value; and

outputting the modified second audio data.

22. The system of claim 21, wherein the partitioning parameters include one or more of a partition count, a partition size, or an offset value.

23. The system of claim 21, further configured to perform operations including:

determining a time delay according to the partitioning parameters; and

delaying the second audio data by the time delay.

24. The system of claim 21, further comprising determining one or more of a peak value or a crest factor for the first audio data associated with each discrete time partition.

25. The system of claim 24, wherein storing the plurality of RMS values in the first data structure includes:

sequentially inserting, for each discrete time partition, an element into the first data structure, the element including one or more of the root mean square value, the peak value, and the crest factor.

26. The system of claim 25, further configured to perform operations including:

using the first data structure according to first data structure parameters, the first data structure parameters being determined according to the partitioning parameters, the first data structure parameters including a data structure size and an index, the first data structure being a first-in-first-out data structure having a first and a last element.

27. The system of claim 21, further configured to perform operations including:

using the second data structure according to second data structure parameters, the second data structure parameters being determined according to the partitioning parameters, the second data structure parameters including a data structure size and an index.

28. The system of claim 21, wherein the using of the selected RMS value to determine the gain adjustment comprises:

identifying a mapping table including mappings from input data to output data;

mapping, using the mapping table, the selected RMS value to an output value; and

computing a ratio of the output value to the selected RMS value, wherein the gain adjustment is based on the ratio.

29. The system of claim 21, further configured to perform operations including:

using a noise gate having one or more noise gate parameters; and

controlling the noise gate parameters according to the noise estimator parameters at intervals determined according to the partitioning parameters.