Upstream signal processing for client devices in a small-cell wireless network

• US 9,300,714 B2
• Filed: 03/17/2011
• Issued: 03/29/2016
• Est. Priority Date: 09/19/2008
• Status: Expired due to Fees

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Bauer, Claus, Crockett, Brett G., Robinson, Charles Q.
• Primary Examiner(s): Ghebretinsae, Temesgh
• Assistant Examiner(s): Jansen, II, Michael J
• Application Number:
  US13/050,854
• Publication Number:
  US 20110169721A1
• Time in Patent Office: 1,839 Days
• Field of Search: 345/84
• US Class Current: 1/1
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: H04L 65/756 adapting media to device capabilities
• CPC Class Codes: H04L 65/765 intermediate
• CPC Class Codes: H04N 21/214 Specialised server platform, e.g. server located in an airplane, hotel, hospital arrangements specially adapted for local area broadcast systems H04H20/61

Abstract

Methods, apparatuses and program logic in storage media to process media data for quality enhancement. The media data is for rendering at a wireless device. Environmental quantities are accepted from one or more sensors located remote from the wireless device but sufficiently close to be indicative of similar quantities in the device'"'"'s environment. The sensors at in or connected to a network node. At the network node, the method includes processing the media data using the environmental quantities to generate processed data, and wirelessly sending the processed output to the resource constrained device for rendering, such that the processed output is usable by the resource constrained device to render or to process and render the media data.

Claims

1. A method of processing media data for quality enhancement processing, the method comprising:

receiving the media data at a network node of a wireless network for upstream signal processing on one or both of audio data and video data, the network node having a media rendering device wirelessly coupled thereto, the network node including or coupled to processing hardware that is external to media rendering device, the media rendering device being battery operated, computational-power-limited, or both battery operated and computational-power-limited;

receiving at the network node to which the media rendering device is wirelessly coupled, one or more environmental quantities determined from one or more sensors located in the vicinity of but not in or on the wireless media rendering device, the environmental quantities related to at least one or both of acoustic noise and lighting in the environment in the vicinity of the media rendering device;

data processing of the received media data at the network node using the processing hardware included in or coupled to the network node, the data processing being based on the one or more received environmental quantities, the data processing generating processed data; and

transmitting wirelessly the processed data to the media rendering device for rendering by the media rendering device, or for further processing and rendering by the media rendering device, thereby limiting the use of electric and processing power at the media rendering device,

wherein the data processing is carried out on one or both of the audio data included in the media data and the video data included in the media data,

wherein when performing the data processing includes performing the data processing on the audio data included in the media data, the one or more environmental quantities include at least one quantity indicative of an acoustic profile of acoustic noise in the environment of the media rendering device, including a spectral estimate of the acoustic noise sensed in the environment of the media rendering device, the processing the media data for quality enhancement includes the data processing of the audio data by the processing hardware, and comprises generating modification parameters for modifying the audio data, the modifying generating quality-enhanced audio data, and the quality enhancement of the audio includes noise compensation, and

wherein when performing the data processing includes performing the data processing on the video data included in the media data, the one or more environmental quantities include one or more parameters indicative of lighting in the environment of the media rendering device, the media rendering device includes a flat panel display device that has a display panel and location dependent backlighting elements each backlighting element being configured to provide a variable amount of back-light to a corresponding region of the display panel according to modulation data for the corresponding region while a scene is being displayed, the processing the media data for quality enhancement includes the data processing of the video data by the processing hardware and comprises generating the modulation data according to at least one of the one or more parameters indicative of lighting in the environment of the media rendering device, the generated modulation data is further configured to modulate the intensity of the backlighting elements according to one or more image properties in the corresponding region of the scene being displayed, the generated modulation data is further configured to modulate the intensity of the backlighting elements according to one or more image properties in the corresponding region of the scene being displayed, the one or more image properties including a contrast-related property, a brightness-related property, or both a contrast-related property and a brightness-related property,

such that at least some of the processing of the media data for quality enhancement is carried out by the processing hardware included in or coupled to the network node.

2. A method as recited in claim 1, wherein the network node includes a base station of the wireless network.

3. A method as recited in claim 1, wherein the generated processed data includes processed media data for rendering by the media rendering device.

4. A method as recited in claim 1, wherein some media data processing occurs at the media rendering device, and wherein the generated processed data includes helper data for use by the media rendering device for the media data processing at the media rendering device.

5. A method as recited in claim 1, wherein the media data received at the network code of the wireless network includes at least one of:

media data streamed to the media rendering device, and interactively streamed media data being interactively streamed across the wireless network to the media rendering device as part of two-way communication involving the media rendering device.

6. A method as recited in claim 5, wherein the media data includes the audio data, the one or more environmental quantities include said at least one quantity indicative of an acoustic profile of acoustic noise in the environment of the media rendering device, the data processing at the network node comprises the generating of the modification parameters that apply quality-enhancement to the audio data, and the quality enhancement of the audio includes the noise compensation.

7. A method as recited in claim 6, wherein the noise compensation comprises:

generating the modification parameters from the audio data using one or more loudness level parameters and the one or more parameters of the acoustic noise profile, wherein the modification parameters are generated by carrying out operations on information in a perceptual loudness domain, and modifying the audio data based upon the modification parameters to generate processed audio data, and wherein the one or more loudness level parameters include;

at least one of the parameters set consisting of;

an indication of whether audio noise compensation is turned on;

a reference level for the media rendering device;

a desired reproduction level, and a noise compensation amount.

8. A method as recited in claim 7, wherein the quality enhancement processing of the media data includes at least one of the set consisting of automatic gain control applied to the audio data, dynamic range compression applied to the audio data, and equalization applied to the audio data.

9. A method as recited in claim 5, wherein the received media data includes media data streamed to the media rendering device, wherein the streamed media data includes the video data, wherein the one or more environmental quantities include the one or more parameters indicative of the lighting in the environment, wherein the media rendering device includes the flat panel display device that has the display panel and the location dependent backlighting element, wherein the processing for quality enhancement includes the generating the modulation data according to at least one of the one or more parameters indicative of lighting in the environment;

wherein the generated modulation data is further configured to modulate the intensity of the backlighting elements according to the one or more image properties in the corresponding region of the scene being displayed, and wherein the one or more image properties including a contrast-related property, a brightness-related property, or both a contrast-related property and a brightness-related property.

10. A system for carrying out at least part of quality enhancement processing of media data, the system comprising:

a network node of a wireless network for upstream signal processing on one or both of audio data and video data, the node being wirelessly connected to a media rendering device, the media rendering device being battery operated, computational-power-limited, or both battery operated and computational-power-limited;

one or more sensors located in the vicinity of but not in or on the media rendering device, the one or more sensors being connected to or in the network node, the sensors determining one or more environmental quantities, the environmental quantities related to at least one or both of acoustic noise and lighting in the environment in the vicinity of the media rendering device;

processing hardware external to the media rendering device and connected to or included in the network node, the processing hardware in operation;

receiving media data comprising audio data, video data, or both audio data and video data,

accepting the one or more environmental quantities, and

data processing the media data using at least some of the accepted environmental quantities, the data processing generating processed output,

wherein the network node in operation wirelessly transmits the processed data to the media rendering device for rendering by the media rendering device or for rendering and further processing by the media rendering device, thereby limiting the use of electric and processing power at the media rendering device,

wherein the data processing is carried out one or both of audio data included in the media data and video data included in the media data,

wherein when performing the data processing includes performing the data processing on the audio data included in the media data, the one or more environmental quantities include at least one quantity indicative of an acoustic profile of acoustic noise in the environment of the media rendering device, including a spectral estimate of the acoustic noise sensed in the environment of the media rendering device, the quality enhancement processing includes the data processing of the audio data by the processing hardware and comprises generating modification parameters for modifying the audio data, the modifying generates quality-enhanced audio data, and the quality enhancement of the audio data includes noise compensation, and

wherein when performing the data processing includes performing the data processing on the video data included in the media data, the one or more environmental quantities include one or more parameters indicative of lighting in the environment of the media rendering device, the media rendering device includes a flat panel display device that has a display panel and location dependent backlighting elements each backlighting element being configured to provide a variable amount of back light to a corresponding region of the display panel according to modulation data for the corresponding region while a scene is being displayed, the processing the media data for quality enhancement includes the data processing of the video data by the processing hardware and comprises generating the modulation data according to at least one of the one or more parameters indicative of lighting in the environment of the media rendering device, the generated modulation data is further configured to modulate the intensity of the backlighting elements according to one or more image properties in the corresponding region of the scene being displayed, the generated modulation data is further configured to modulate the intensity of the backlighting elements according to one or more image properties in the corresponding region of the scene being displayed, the one or more image properties including a contrast-related property, a brightness-related property, or both a contrast-related property and a brightness-related property,

such that at least some of the media data processing for quality enhancement is carried out by the processing hardware included in or coupled to the network node.

11. A system as recited in claim 10, wherein the network node includes a base station of the wireless network.

12. A system as recited in claim 10, wherein the generated processed data includes processed media data for rendering by the media rendering device.

13. A system as recited in claim 10, wherein some media data processing occurs at the media rendering device, and wherein the generated processed data includes helper data for use by the media rendering device for the media data processing at the media rendering device.

14. A system as recited in claim 10, wherein the media data received at the network code of the wireless network includes at least one of:

media data streamed to the media rendering device, and interactively streamed media data being interactively streamed across the wireless network to the media rendering device as part of two-way communication involving the media rendering device.

15. A system as recited in claim 14, wherein the media data includes the audio data, the one or more environmental quantities include said at least one quantity indicative of an acoustic profile of acoustic noise in the environment of the media rendering device, the data processing at the network node comprises the generating of the modification parameters that apply quality-enhancement to the audio data, and the quality enhancement of the audio includes the noise compensation.

16. A system as recited in claim 15, wherein the noise compensation comprises:

generating the modification parameters from the audio data using one or more loudness level parameters and the one or more parameters of the acoustic noise profile, wherein the modification parameters are generated by carrying out operations on information in a perceptual loudness domain, and modifying the audio data based upon the modification parameters to generate processed audio data, and wherein the one or more loudness level parameters include;

at least one of the parameters set consisting of;

an indication of whether audio noise compensation is turned on;

a reference level for the media rendering device;

a desired reproduction level, and a noise compensation amount.

17. A system as recited in claim 16, wherein the quality enhancement processing of the media data includes at least one of the set consisting of automatic gain control applied to the audio data, dynamic range compression applied to the audio data, and equalization applied to the audio data.

18. A system as recited in claim 14, wherein the received media data includes media data streamed to the media rendering device, wherein the streamed media data includes the video data, wherein the one or more environmental quantities include the one or more parameters indicative of the lighting in the environment, wherein the media rendering device includes the flat panel display device that has the display panel and the location dependent backlighting element, wherein the quality enhancement processing includes the generating the modulation data according to the at least one of the one or more parameters indicative of the lighting in the environment;

wherein the generated modulation data is further configured to modulate the intensity of the backlighting elements according to the one or more image properties in the corresponding region of the scene being displayed, and wherein the one or more image properties including a contrast-related property, a brightness-related property, or both a contrast-related property and a brightness-related property.

19. A non-transitory computer-readable storage medium configured with instructions that that when executed by one or more processors of a processing system cause carrying out of a method of processing media data for quality enhancement processing, the method comprising:

receiving the media data at a network node of a wireless network for upstream signal processing on one or both of audio data and video data, the network node having a media rendering device wirelessly coupled thereto, the network node including or coupled to processing hardware that is external to media rendering device, the media rendering device being battery operated, computational-power-limited, or both battery operated and computational-power-limited;

receiving at the network node to which the media rendering device is wirelessly coupled one or more environmental quantities determined from one or more sensors located in the vicinity of but not in or on the wireless media rendering device, the environmental quantities related to at least one or both of acoustic noise and lighting in the environment in the vicinity of the media rendering device;

data processing the received media data at the network node using the processing hardware included in or coupled to the network node, the data processing being based on the one or more received environmental quantities, the data processing generating processed data; and

transmitting wirelessly the processed data to the media rendering device for rendering by the media rendering device, or for further processing and rendering by the media rendering device, thereby limiting the use of electric and processing power at the media rendering device,

wherein the data processing is carried out on one or both of the audio data included in the media data and the video data included in the media data,

wherein when performing the data processing includes performing the data processing on the audio data included in the media data, the one or more environmental quantities include at least one quantity indicative of an acoustic profile of acoustic noise in the environment of the media rendering device, including a spectral estimate of the acoustic noise sensed in the environment of the media rendering device, the processing the media data for quality enhancement includes the data processing of the audio data by the processing hardware, and comprises generating modification parameters for modifying the audio data, the modifying generating quality-enhanced audio data, and the quality enhancement of the audio includes noise compensation, and

wherein when performing the data processing includes performing the data processing on the video data included in the media data, the one or more environmental quantities include one or more parameters indicative of lighting in the environment of the media rendering device, the media rendering device includes a flat panel display device that has a display panel and location dependent backlighting elements each backlighting element being configured to provide a variable amount of back-light to a corresponding region of the display panel according to modulation data for the corresponding region while a scene is being displayed, the processing the media data for quality enhancement includes the data processing of the video data by the processing hardware and comprises generating the modulation data according to at least one of the one or more parameters indicative of lighting in the environment of the media rendering device, the generated modulation data is further configured to modulate the intensity of the backlighting elements according to one or more image properties in the corresponding region of the scene being displayed, the generated modulation data is further configured to modulate the intensity of the backlighting elements according to one or more image properties in the corresponding region of the scene being displayed, the one or more image properties including a contrast-related property, a brightness-related property, or both a contrast-related property and a brightness-related property,

such that at least some of the processing of the media data for quality enhancement is carried out by the processing hardware included in or coupled to the network node.

20. A non-transitory computer-readable storage medium as recited in claim 19, wherein the network node includes a base station of the wireless network.

21. A non-transitory computer-readable storage medium as recited in claim 19, wherein the generated processed data includes processed media data for rendering by the media rendering device.

22. A non-transitory computer-readable storage medium as recited in claim 19, wherein some media data processing occurs at the media rendering device, and wherein the generated processed data includes helper data for use by the media rendering device for the media data processing at the media rendering device.

23. A non-transitory computer-readable storage medium as recited in claim 19, wherein the media data received at the network code of the wireless network includes at least one of:

media data streamed to the media rendering device, and

interactively streamed media data being interactively streamed across the wireless network to the media rendering device as part of two-way communication involving the media rendering device.

24. A non-transitory computer-readable storage medium as recited in claim 23, wherein the media data includes the audio data, the one or more environmental quantities include said at least one quantity indicative of an acoustic profile of acoustic noise in the environment of the media rendering device, the data processing at the network node comprises the generating of the modification parameters that apply quality-enhancement to the audio data, and the quality enhancement of the audio includes the noise compensation.

25. A non-transitory computer-readable storage medium as recited in claim 24, wherein the noise compensation comprises:

generating the modification parameters from the audio data using one or more loudness level parameters and the one or more parameters of the acoustic noise profile, wherein the modification parameters are generated by carrying out operations on information in a perceptual loudness domain, and modifying the audio data based upon the modification parameters to generate processed audio data, and wherein the one or more loudness level parameters include;

at least one of the parameters set consisting of;

an indication of whether audio noise compensation is turned on;

a reference level for the media rendering device, a desired reproduction level; and

a noise compensation amount.

26. A non-transitory computer-readable storage medium as recited in claim 25, wherein the quality enhancement processing of the media data includes at least one of the set consisting of automatic gain control applied to the audio data, dynamic range compression applied to the audio data, and equalization applied to the audio data.

27. A non-transitory computer-readable storage medium as recited in claim 23, wherein the received media data includes media data streamed to the media rendering device, wherein the received media data includes video data, wherein the one or more environmental quantities include the one or more parameters indicative of the lighting in the environment, wherein the media rendering device includes the flat panel display device that has the display panel and the location dependent backlighting element, wherein the processing for quality enhancement includes the generating the modulation data according to at least one of the one or more parameters indicative of lighting in the environment;

wherein the generated modulation data is further configured to modulate the intensity of the backlighting elements according to the one or more image properties in the corresponding region of the scene being displayed, and wherein the one or more image properties including a contrast-related property, a brightness-related property, or both a contrast-related property and a brightness-related property.