Upstream quality enhancement signal processing for resource constrained client devices

• US 8,744,247 B2
• Filed: 03/17/2011
• Issued: 06/03/2014
• Est. Priority Date: 09/19/2008
• Status: Expired due to Fees

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Crockett, Brett G., Robinson, Charles Q., Bauer, Claus
• Primary Examiner(s): Chevalier, Robert
• Application Number:
  US13/050,834
• Publication Number:
  US 20110164855A1
• Time in Patent Office: 1,174 Days
• Field of Search: 386/285, 386/248, 386/353, 386/338
• US Class Current: 386/338
• CPC Class Codes: G10L 21/02 Speech enhancement, e.g. noise reduction or echo cancellation reducing echo effects in line transmission systems H04B3/20; echo suppression in hands-free telephones H04M9/08
• CPC Class Codes: H04N 5/91 Television signal processing therefor

Abstract

Methods, apparatuses and program logic in storage media to process media data for quality enhancement. Information is accepted from a resource constrained device, e.g., a wireless portable device related to the quality enhancement and/or environmental quantities such as background noise and/or ambient lighting for the device. Media data is data processed to achieve quality enhancement using at least some of the accepted information to generate processed output. The data processing of the media data includes processing when or where one or more resources sufficient for the processing are available.

Claims

1. A method of processing audio data for at least one of spatializing and loudness domain audio processing using processing hardware, the method comprising:

receiving in the processing hardware audio data streamed from a battery operated and/or processing power limited audio playback device, the audio data being for rendering by the audio playback device;

accepting in the processing hardware metadata from the audio playback device related to one or both of the spatializing and the loudness domain audio processing of the audio data; and

data processing the audio data in at least the processing hardware for at least one of spatializing and loudness domain audio processing using at least some of the accepted metadata to generate processed output, wherein the loudness domain audio processing comprises determining a measure of the specific loudness of the audio data, and wherein the spatializing comprises spatializing the audio data for playback by the audio playback device to provide a listener with the experience of listening to audio coming from a set of loudspeakers at a respective set of locations in a listening room, even if the listener is in fact listening to the audio media data with earphones, or with a simple set of stereo loudspeakers; and

streaming processed audio data in the processed output from the processing hardware to the audio playback device for in real time playback by the audio playback device,

wherein the processing hardware is external to the audio playback device, and

wherein the metadata includes information obtained at the audio playback device, and includes a set of one or more parameters related to the at least one of spatializing and loudness domain audio processing of the media data or to the environment of the audio playback device,

such that the audio data from the audio playback device is played back in real time on the audio playback device in at least one of spatialized form and loudness domain audio processed form.

2. A method as recited in claim 1, wherein the data processing uses at least one of the set of one or more parameters to generate processed output that includes processed audio for rendering by the audio playback device.

3. A method as recited in claim 2, wherein the data processing of the audio data is prior to rendering of the processed media data on the audio playback device.

4. A method as recited in claim 1, wherein some audio data processing occurs at the audio playback device, and wherein the processed output includes helper data for use by the audio playback device for the audio data processing that occurs at the audio playback device.

5. A method as recited in claim 2,

wherein the data processing of the audio data into generate processed audio data occurs where resources sufficient for the processing are available, and precedes rendering of the processed audio data on the audio playback device, and

wherein the audio data includes one or more of:

audio data streamed or broadcast in real time to the audio playback device,

audio data interactively streamed across a network to the audio playback device as part of two-way communication with the audio playback device, and

audio data originating at or generated by audio playback device for rendering by the audio playback device.

6. A method as recited in claim 5, wherein the audio data includes audio data streamed or broadcast in real time to the audio playback device.

7. A method as recited in claim 5, wherein the audio data is at least one of:

stored in the audio playback device and generated by the audio playback device,

wherein the data processing uses at least one of the set of one or more parameters to generate processed output that includes processed audio for rendering by the audio playback device.

8. A method as recited in claim 1, wherein the data processing includes processing for loudness domain audio processing, including loudness leveling according to at least one of the set of one or more parameters.

9. A method as recited in claim 1,

wherein the set of one or more parameters include one or more loudness level parameters,

wherein the data processing includes loudness domain audio processing including:

loudness leveling according the one or more loudness level parameters, the loudness leveling including;

generating modification parameters using the loudness level parameters, wherein the modification parameters are generated by carrying out operations on information in a perceptual loudness domain, and

modifying the audio data based upon the modification parameters to generate processed audio data, and

wherein the one or more loudness level parameters include one or more of a reference reproduction level, a desired reproduction level and a leveling amount.

10. A method as recited in claim 9,

wherein the modifying of the audio data includes one or more of automatic gain control, dynamic range compression, and dynamic equalization.

11. A method as recited in claim 1,

wherein the data processing of the audio data includes processing for audio spatialization according to at least one of the set of one or more parameters.

12. A method as recited in claim 1,

wherein the data processing includes processing for loudness domain audio processing, and

wherein the set of one or more parameters includes one or more parameters related to the environment of the audio playback device.

13. A method as recited in claim 12,

wherein the set of one or more parameters includes one or more parameters related to the environment of the audio playback device, including at least one parameter indicative of an acoustic profile of noise in the environment, and

wherein the data processing includes processing for noise compensation.

14. A method as recited in claim 13,

wherein the set of one or more parameters includes one or more loudness level parameters,

wherein the noise compensation comprises:

generating modification parameters from the audio data using one or more loudness level parameters and the one or more parameters of the acoustic noise profile, the modification parameters being generated by carrying out operations on information in a perceptual loudness domain, and

modifying the audio data based upon the modification parameters to generate processed audio data, and

wherein the one or more than one loudness level parameters include;

one or more of whether audio noise compensation is turned on, a reference level for the audio playback device, a desired reproduction level, and a noise compensation amount.

15. A method as recited in claim 14, wherein the data processing of the audio data includes one or more of automatic gain control, dynamic range compression, and/or equalization applied to the audio data.

16. A tangible computer-readable storage medium configured with instructions that when executed by at least one processor of a processing system causes carrying a method of processing audio data for at least one of spatializing and loudness domain audio processing using processing hardware that includes the processing system, the method comprising:

receiving in the processing hardware audio data streamed from a battery operated and/or processing power limited audio playback device, the audio data being for rendering by the audio playback device;

accepting in the processing hardware metadata from the audio playback device related to one or both of the spatializing and the loudness domain audio processing of the audio data; and

data processing the audio data in at least the processing hardware for at least one of spatializing and loudness domain audio processing using at least some of the accepted metadata to generate processed output, wherein the loudness domain audio processing comprises determining a measure of the specific loudness of the audio data, and wherein the spatializing comprises spatializing the audio data for playback by the audio playback device to provide a listener with the experience of listening to audio coming from a set of loudspeakers at a respective set of locations in a listening room, even if the listener is in fact listening to the audio media data with earphones, or with a simple set of stereo loudspeakers; and

streaming processed audio data in the processed output from the processing hardware to the audio playback device for in real time playback by the audio playback device,

wherein the processing hardware is external to the audio playback device, and

wherein the metadata includes information obtained at the audio playback device, and includes a set of one or more parameters related to the at least one of spatializing and loudness domain audio processing of the media data or to the environment of the audio playback device,

such that the audio data from the audio playback device is played back in real time on the audio playback device in at least one of spatialized form and loudness domain audio processed form.

17. A tangible computer-readable storage medium as recited in claim 16,

wherein the processing hardware is external to the audio playback device, and

wherein some audio data processing occurs at the audio playback device, and wherein the processed output includes helper data for use by the audio playback device for the audio data processing that occurs at the audio playback device.

18. A tangible computer-readable storage medium, as recited in claim 16,

wherein the data processing of the audio data by the processing hardware into generate processed audio data precedes rendering of the processed audio data on the audio playback device, and

wherein the audio data includes one or more of:

audio data streamed or broadcast in real time to the audio playback device,

audio data interactively streamed across a network to the audio playback device as part of two-way communication with the audio playback device, and

audio data originating at or generated by audio playback device for rendering by the audio playback device.

19. A tangible computer-readable storage medium as recited in claim 18, wherein the audio data includes audio data streamed or broadcast in real time to the audio playback device.

20. A tangible computer-readable storage medium as recited in claim 18, wherein the audio data is at least one of:

stored in the audio playback device and generated by the audio playback device,

wherein the data processing uses at least one parameter of the set of one or more parameters to generate processed output that includes processed audio for rendering by the audio playback device.

21. A tangible computer-readable storage medium, as recited in claim 16, wherein the data processing by the processing hardware includes processing for loudness domain audio processing, including loudness leveling according to at least one of the set of one or more parameters.

22. A tangible computer-readable storage medium as recited in claim 21, wherein the set of one or more parameters include one or more loudness level parameters,

wherein the loudness leveling is according the one or more loudness level parameters, the loudness leveling including:

generating modification parameters using the loudness level parameters, wherein the modification parameters are generated by carrying out operations on information in a perceptual loudness domain, and

modifying the audio data based upon the modification parameters to generate processed audio data, and

wherein the one or more loudness level parameters include one or more of a reference reproduction level, a desired reproduction level and a leveling amount, and

wherein the modifying of the audio data includes one or more of;

automatic gain control, dynamic range compression, and dynamic equalization.

23. A tangible computer-readable storage medium as recited in claim 16,

wherein the data processing of the audio data includes processing for audio spatialization according to at least one parameter of the set of one or more parameters.

24. A tangible computer-readable storage medium as recited in claim 16,

wherein the data processing includes processing for loudness domain audio processing, and

wherein the set of one or more parameters includes one or more parameters related to the environment of the audio playback device.

25. A tangible computer-readable storage medium as recited in claim 24,

wherein the set of one or more parameters includes one or more parameters related to the environment of the audio playback device, including at least one parameter indicative of an acoustic profile of noise in the environment, and

wherein the data processing includes processing for noise compensation.

26. A system for carrying out at least part of one or more of spatializing and loudness domain audio processing of audio data, the apparatus comprising:

an interface operative to connect to a battery operated and/or processing power limited audio playback device and to accept metadata from the audio playback device related to one or both of spatializing and loudness domain audio processing of audio data that is for rendering by the audio playback device;

processing hardware operative to receive audio data streamed from the audio playback device, and to carry out data processing of the received audio data for at least one of spatializing and loudness domain audio processing using at least some of the accepted metadata to generate processed output, wherein the loudness domain audio processing comprises determining a measure of the specific loudness of the audio data, and wherein the spatializing comprises spatializing the audio data for playback by the audio playback device to provide a listener with the experience of listening to audio coming from a set of loudspeakers at a respective set of locations in a listening room, even if the listener is in fact listening to the audio media data with earphones, or with a simple set of stereo loudspeakers; and

hardware operative to stream processed audio data in the processed output to the audio playback device for real time playback by the audio playback device,

wherein the metadata includes information obtained at the audio playback device, and includes a set of one or more parameters related to the at least one of spatializing and loudness domain audio processing of the audio data or to the environment of the audio playback device,

such that the processed output audio data from the audio playback device is played back on the audio playback device in at least one of spatialized form and loudness domain audio processed form.

27. A system as recited in claim 26,

wherein the processing hardware is external to the audio playback device, and

wherein some audio data processing occurs at the audio playback device, and wherein the processed output includes helper data for use by the audio playback device for the audio data processing that occurs at the audio playback device.

28. A system, as recited in claim 26,

wherein the data processing of the audio data by the processing hardware to generate processed audio data precedes rendering of the processed audio data on the audio playback device, and

wherein the audio data includes one or more of:

audio data streamed or broadcast in real time to the audio playback device,

audio data interactively streamed across a network to the audio playback device as part of two-way communication with the audio playback device, and

audio data originating at or generated by audio playback device for rendering by the audio playback device.

29. A system as recited in claim 28, wherein the audio data includes audio data streamed or broadcast in real time to the audio playback device.

30. A system as recited in claim 28, wherein the audio data is at least one of:

stored in the audio playback device and generated by the audio playback device,

wherein the data processing uses at least one parameter of the set of one or more parameters to generate processed output that includes processed audio for rendering by the audio playback device.

31. A system, as recited in claim 26, wherein the data processing by the processing hardware includes processing for loudness domain audio processing, including loudness leveling according to at least one of the set of one or more parameters.

32. A system as recited in claim 31, wherein the set of one or more parameters include one or more loudness level parameters,

wherein the loudness leveling is according the one or more loudness level parameters, the loudness leveling including:

generating modification parameters using the loudness level parameters, wherein the modification parameters are generated by carrying out operations on information in a perceptual loudness domain, and

modifying the audio data based upon the modification parameters to generate processed audio data, and

wherein the one or more loudness level parameters include one or more of a reference reproduction level, a desired reproduction level and a leveling amount, and

wherein the modifying of the audio data includes one or more of automatic gain control, dynamic range compression, and dynamic equalization.

33. A system as recited in claim 26,

wherein the data processing of the audio data includes processing for audio spatialization according to at least one parameter of the set of one or more parameters.

34. A system as recited in claim 26,

wherein the data processing includes processing for loudness domain audio processing, and

wherein the set of one or more parameters includes one or more parameters related to the environment of the audio playback device.

35. A system as recited in claim 34,

wherein the set of one or more parameters includes one or more parameters related to the environment of the audio playback device, including at least one parameter indicative of an acoustic profile of noise in the environment, and

wherein the data processing includes processing for noise compensation.