UPSTREAM QUALITY ENHANCEMENT SIGNAL PROCESSING FOR RESOURCE CONSTRAINED CLIENT DEVICES

• US 20110164855A1
• Filed: 03/17/2011
• Published: 07/07/2011
• Est. Priority Date: 09/19/2008
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Bauer, Claus, Robinson, Charles Q., Crockett, Brett G.
• Granted Patent: US 8,744,247 B2
• Time in Patent Office: Days
• US Class Current: 386/230
• CPC Class Codes: G10L 21/02 Speech enhancement, e.g. noise reduction or echo cancellation reducing echo effects in line transmission systems H04B3/20; echo suppression in hands-free telephones H04M9/08
• CPC Class Codes: H04N 5/91 Television signal processing therefor

Abstract

Methods, apparatuses and program logic in storage media to process media data for quality enhancement. Information is accepted from a resource constrained device, e.g., a wireless portable device related to the quality enhancement and/or environmental quantities such as background noise and/or ambient lighting for the device. Media data is data processed to achieve quality enhancement using at least some of the accepted information to generate processed output. The data processing of the media data includes processing when or where one or more resources sufficient for the processing are available.

Claims

1. A method of processing audio data for at least one of spatializing and loudness domain audio processing using processing hardware, the method comprising:

receiving in the processing hardware audio data streamed from a battery operated and/or processing power limited audio playback device, the audio data being for rendering by the audio playback device;

accepting in the processing hardware metadata from the audio playback device related to one or both of the spatializing and the loudness domain audio processing of the audio data; and

data processing the audio data in at least the processing hardware for at least one of spatializing and loudness domain audio processing using at least some of the accepted metadata to generate processed output; and

streaming the processed output from the processing hardware to the audio playback device for in real time playback by the audio playback device,

wherein the processing hardware is external to the audio playback device, and

wherein the metadata includes information obtained at the audio playback device, and includes a set of one or more parameters related to the at least one of spatializing and loudness domain audio processing of the media data or to the environment of the audio playback device,

such that the audio data from the audio playback device is played back in real time on the audio playback device in at least one of spatialized form and loudness domain audio processed form.

2. A method as recited in claim 1, wherein the data processing uses at least one of the one or more parameters to generate processed output that includes processed audio for rendering by the audio playback device.

3. A method as recited in claim 2, wherein the data processing of the audio data is prior to rendering of the processed media data on the audio playback device.

4. A method as recited in claim 1, wherein some audio data processing occurs at the audio playback device, and wherein the processed output includes helper data for use by the audio playback device for the audio data processing that occurs at the audio playback device.

5. A method as recited in claim 2,

wherein the data processing of the data is where resources sufficient for the processing are available, and is prior to rendering of the processed audio data on the audio playback device, and

wherein the audio data includes one or more of:

audio data streamed or broadcast in real time to the audio playback device,

audio data interactively streamed across a network to the audio playback device as part of two-way communication with the audio playback device, and

audio data originating at or generated by audio playback device for rendering by the audio playback device.

6. A method as recited in claim 5, wherein the audio data includes audio data streamed or broadcast in real time to the resource constrained device.

7. A method as recited in claim 5, wherein the audio data is stored in and/or generated by the audio playback device,

wherein the data processing uses one or more of the parameters to generate processed output that includes processed audio for rendering by the audio playback device.

8. A method as recited in claim 1, wherein the data processing includes processing for loudness domain audio processing, including loudness leveling according to one or more of the received parameters.

9. A method as recited in claim 1,

wherein the received parameters include one or more loudness level parameters,

wherein the data processing includes processing for loudness domain audio processing including loudness leveling according the one or more loudness level parameters,

wherein the loudness leveling includes:

generating modification parameters using the loudness level parameters, wherein the modification parameters are generated by carrying out operations on information in a perceptual loudness domain, and

modifying the audio data based upon the modification parameters to generate processed audio data, and

wherein the one or more loudness level parameters include one or more of a reference reproduction level, a desired reproduction level and/or a leveling amount.

10. A method as recited in claim 9,

wherein the modifying of the audio data includes one or more of automatic gain control, dynamic range compression, and/or dynamic equalization.

11. A method as recited in claim 1,

wherein the data processing of the audio data includes processing for audio spatialization according to one or more of the received parameters.

12. A method as recited in claim 1,

wherein the data processing includes processing for loudness domain audio processing, and

wherein the one or more parameters includes one or more parameters related to the environment of the audio playback device.

13. A method as recited in claim 12,

wherein the one or more parameters includes one or more parameters related to the environment of the audio playback device, including at least one parameter indicative of an acoustic profile of noise in the environment, and

wherein the data processing includes processing for noise compensation.

14. A method as recited in claim 13,

wherein the one or more parameters includes one or more loudness level parameters,

wherein the noise compensation comprises:

generating modification parameters from the audio data using one or more loudness level parameters and the one or more parameters of the acoustic noise profile, wherein the modification parameters are generated by carrying out operations on information in a perceptual loudness domain, and

modifying the audio data based upon the modification parameters to generate processed audio data, and

wherein the one or more than one loudness level parameters include;

one or more of whether audio noise compensation is turned on, a reference level for the audio playback device, a desired reproduction level, and/or a noise compensation amount.

15. A method as recited in claim 14, wherein the data processing of the audio data includes one or more of automatic gain control, dynamic range compression, and/or equalization applied to the audio data.

16. A method of processing video data using processing hardware, the method comprising:

streaming video data to a media playback device that is battery operated and/or that has limited processing power, the media data for rendering in real time by the media playback device, wherein the media playback device includes a flat panel display device that has location dependent backlighting elements each modulated according to image dependent modulation data;

accepting information from the media playback device related to modifying the contrast and/or brightness of the video data, including one or more parameters obtained at the media playback device related to the environment of the media playback device, including at least one parameter indicative of lighting in the environment;

data processing the video data on the processing hardware to generate processed output, the data processing including generating the image dependent modulation data according to at least one of the one or more parameters; and

sending the image dependent modulation data to the media playback device for use in rendering the video data,

wherein the processing hardware is external to the media playback device.

17. A tangible computer-readable storage medium configured with instructions that when executed by at least one processor of a processing system causes carrying a method of processing audio data for at least one of spatializing and loudness domain audio processing using processing hardware that includes the processing system, the method comprising:

receiving in the processing hardware audio data streamed from a battery operated and/or processing power limited audio playback device, the audio data being for rendering by the audio playback device;

accepting in the processing hardware metadata from the audio playback device related to one or both of the spatializing and the loudness domain audio processing of the audio data; and

data processing the audio data in at least the processing hardware for at least one of spatializing and loudness domain audio processing using at least some of the accepted metadata to generate processed output; and

streaming the processed output from the processing hardware to the audio playback device for in real time playback by the audio playback device,

wherein the processing hardware is external to the audio playback device, and

wherein the metadata includes information obtained at the audio playback device, and includes a set of one or more parameters related to the at least one of spatializing and loudness domain audio processing of the media data or to the environment of the audio playback device,

such that the audio data from the audio playback device is played back in real time on the audio playback device in at least one of spatialized form and loudness domain audio processed form.

18. A tangible computer-readable storage medium as recited in claim 17,

wherein the processing hardware is external to the audio playback device, and

wherein some audio data processing occurs at the audio playback device, and wherein the processed output includes helper data for use by the audio playback device for the audio data processing that occurs at the audio playback device.

19. A tangible computer-readable storage medium, as recited in claim 17,

wherein the data processing of the data by the processing hardware is prior to rendering of the processed audio data on the audio playback device, and

wherein the audio data includes one or more of:

audio data streamed or broadcast in real time to the audio playback device,

audio data interactively streamed across a network to the audio playback device as part of two-way communication with the audio playback device, and

audio data originating at or generated by audio playback device for rendering by the audio playback device.

20. A tangible computer-readable storage medium as recited in claim 19, wherein the audio data includes audio data streamed or broadcast in real time to the audio playback device.

21. A tangible computer-readable storage medium as recited in claim 19, wherein the audio data is stored in and/or generated by the audio playback device,

wherein the data processing uses one or more of the parameters to generate processed output that includes processed audio for rendering by the audio playback device.

22. A tangible computer-readable storage medium, as recited in claim 17, wherein the data processing by the processing hardware includes processing for loudness domain audio processing, including loudness leveling according to one or more of the received parameters.

23. A tangible computer-readable storage medium as recited in claim 22,

wherein the modifying of the audio data includes one or more of automatic gain control, dynamic range compression, and/or dynamic equalization.

24. A tangible computer-readable storage medium as recited in claim 17,

wherein the data processing of the audio data includes processing for audio spatialization according to one or more of the received parameters.

25. A tangible computer-readable storage medium as recited in claim 17,

wherein the data processing includes processing for loudness domain audio processing, and

wherein the one or more parameters includes one or more parameters related to the environment of the audio playback device.

26. A tangible computer-readable storage medium as recited in claim 25,

wherein the one or more parameters includes one or more parameters related to the environment of the audio playback device, including at least one parameter indicative of an acoustic profile of noise in the environment, and

wherein the data processing includes processing for noise compensation.

27. A tangible computer-readable storage medium configured with instructions that when executed by at least one processor of a processing system causes carrying out a method of processing video data using processing hardware, the method comprising:

streaming video data to a media playback device that is battery operated and/or that has limited processing power, the media data for rendering in real time by the media playback device, wherein the media playback device includes a flat panel display device that has location dependent backlighting elements each modulated according to image dependent modulation data;

accepting information from the media playback device related to modifying the contrast and/or brightness of the video data, including one or more parameters obtained at the media playback device related to the environment of the media playback device, including at least one parameter indicative of lighting in the environment;

data processing the video data on the processing hardware to generate processed output, the data processing including generating the image dependent modulation data according to at least one of the one or more parameters; and

sending the image dependent modulation data to the media playback device for use in rendering the video data,

wherein the processing hardware is external to the media playback device.

28. A system for carrying out at least part of spatializing and loudness domain audio processing of audio data, the apparatus comprising:

an interface operative to connect to a battery operated and/or processing power limited audio playback device and to accept metadata from the audio playback device related to one or both of spatializing and loudness domain audio processing of audio data that is for rendering by the audio playback device;

processing hardware operative to receive audio data streamed from the audio playback device, and to carry out data processing of the received audio data for at least one of spatializing and loudness domain audio processing using at least some of the accepted metadata to generate processed output; and

hardware operative to stream the processed output to the audio playback device for real time playback by the audio playback device,

wherein the metadata includes information obtained at the audio playback device, and includes a set of one or more parameters related to the at least one of spatializing and loudness domain audio processing of the audio data or to the environment of the audio playback device,

such that the processed output audio data from the audio playback device is played back on the audio playback device in at least one of spatialized form and loudness domain audio processed form.

29. A system as recited in claim 28,

wherein the processing hardware is external to the audio playback device, and

wherein some audio data processing occurs at the audio playback device, and wherein the processed output includes helper data for use by the audio playback device for the audio data processing that occurs at the audio playback device.

30. A system, as recited in claim 28,

wherein the data processing of the data by the processing hardware is prior to rendering of the processed audio data on the audio playback device, and

wherein the audio data includes one or more of:

audio data streamed or broadcast in real time to the audio playback device,

audio data interactively streamed across a network to the audio playback device as part of two-way communication with the audio playback device, and

audio data originating at or generated by audio playback device for rendering by the audio playback device.

31. A system as recited in claim 30, wherein the audio data includes audio data streamed or broadcast in real time to the audio playback device.

32. A system as recited in claim 30, wherein the audio data is stored in and/or generated by the audio playback device,

wherein the data processing uses one or more of the parameters to generate processed output that includes processed audio for rendering by the audio playback device.

33. A system, as recited in claim 28, wherein the data processing by the processing hardware includes processing for loudness domain audio processing, including loudness leveling according to one or more of the received parameters.

34. A system as recited in claim 33,

wherein the modifying of the audio data includes one or more of automatic gain control, dynamic range compression, and/or dynamic equalization.

35. A system as recited in claim 28,

wherein the data processing of the audio data includes processing for audio spatialization according to one or more of the received parameters.

36. A system as recited in claim 28,

wherein the data processing includes processing for loudness domain audio processing, and

wherein the one or more parameters includes one or more parameters related to the environment of the audio playback device.

37. A system as recited in claim 36,

wherein the one or more parameters includes one or more parameters related to the environment of the audio playback device, including at least one parameter indicative of an acoustic profile of noise in the environment, and

wherein the data processing includes processing for noise compensation.

38. A system for processing of video data, the system comprising:

an interface operative to connect to a battery operated and/or limited processing power media playback device and to accept information from the media playback device related to modifying the contrast and/or brightness of the video data, including one or more parameters obtained at the media playback device, the video data being for rendering by the media playback device, wherein the media playback device includes a flat panel display device that has location dependent backlighting elements each modulated according to image dependent modulation data sent to the media playback device with the video data;

processing hardware operative to receive video data streamed for in real time playback on the video playback device, and operative to carry out data processing of the video data including generating the image dependent modulation data according to at least one of the one or more parameters; and

wherein the system if operative to send the image dependent modulation data to the media playback device for use in rendering the video data.