Signal processing and tiered signal encoding

• US 8,711,943 B2
• Filed: 07/21/2011
• Issued: 04/29/2014
• Est. Priority Date: 07/21/2011
• Status: Active Grant

• Current Assignee: Nova International Limited
• Original Assignee: Guido Meardi, Luca Rossato
• Inventors: Rossato, Luca, Meardi, Guido
• Primary Examiner(s): Ustaris, Joseph
• Assistant Examiner(s): NASRI, MARYAM A
• Application Number:
  US13/188,207
• Publication Number:
  US 20130294523A1
• Time in Patent Office: 1,013 Days
• Field of Search: None
• US Class Current: 375/240.18
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/134 characterised by the element, parameter or criterion affecting or controlling the adaptive coding
• CPC Class Codes: H04N 19/182 the unit being a pixel
• CPC Class Codes: H04N 19/187 the unit being a scalable video layer
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 19/436 using parallelised computational arrangements
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/82 involving filtering within a prediction loop
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression

Abstract

An encoder receives a signal. The encoder utilizes one or more downsample operations to produce downsampled renditions of the signal at successively lower levels of quality in the hierarchy. In a reverse direction, the encoder applies the one or more upsample operations to a downsampled rendition of the signal at a first level of quality to produce an upsampled rendition of the signal at a second level of quality in the hierarchy. The second level of quality is higher than the first level of quality. The one or more upsample operations and one or more downsample operations can be asymmetrical with respect to each other. That is, the function applied during downsampling can differ from the function applied when upsampling. The encoder produces residual data indicating a difference between the downsampled rendition of the signal at the second level of quality and the upsampled rendition of the signal at the second level of quality.

Claims

1. A method of encoding at different levels of quality in a hierarchy, the method comprising:

having a signal as input;

utilizing at least one downsample operation to produce downsampled renditions of the signal at successively lower levels of quality in the hierarchy;

applying at least one upsample operation to a downsampled rendition of the signal at a first level of quality to produce an upsampled rendition of the signal at a second level of quality in the hierarchy, the second level of quality being higher than the first level of quality; and

producing residual data indicating a difference between the downsampled rendition of the signal at the second level of quality and the upsampled rendition of the signal at the second level of quality, the method further comprising;

parsing the downsampled rendition of the signal at the first level of quality into multiple regions of elements; and

simultaneously processing the multiple regions in parallel to identify adjustments to elements in the multiple regions to locally reduce an entropy associated with the residual data.

2. The method as in claim 1, wherein the at least one downsample operation is non-linear.

3. The method as in claim 2, wherein the at least one upsample operation is non-linear.

4. The method as in claim 1 further comprising:

at least occasionally performing a global check of adjustments to the elements in the multiple regions to reduce an entropy of the residual data.

5. The method as in claim 1 further comprising, wherein applying the at least one upsample operation includes:

for each of multiple elements in the rendition of the signal at the first level of quality, repeating steps of;

selecting an element to tweak in the downsampled signal at the first level of quality;

tweaking the selected element in different directions;

producing the residual data at the second level of quality based on the tweaked selected element; and

calculating an entropy of the residual data at the second level of quality to identify which tweak to the selected element at the first level of quality produces a set of residual data at the second level of quality having the substantially lowest entropy.

6. The method as in claim 5, wherein calculating the entropy includes quantization of the residual data to reduce the entropy.

7. The method as in claim 5, wherein applying the at least one upsample operation further comprises applying multiple upsample operations to the downsampled rendition of the signal at the first level of quality;

and

wherein producing the residual data includes identifying which of multiple upsample options applied to the downsampled rendition of the signal at the first level of quality produces residual data having the substantially lowest entropy.

8. The method as in claim 1, wherein applying the at least one upsample operation further comprises:

applying multiple upsample operations to the downsampled rendition of the signal at the first level of quality; and

wherein producing the residual data includes identifying which of multiple upsample options applied to the downsampled rendition of the signal at the first level of quality produces residual data having the substantially lowest entropy.

9. The method as in claim 1, wherein utilizing the at least one downsample operation to produce downsampled renditions of the signal at successively lower levels of quality in the hierarchy further comprises:

for each of multiple successively lower levels of quality in the hierarchy, repeating steps of;

downsampling the signal;

selecting an element to tweak in the downsampled signal at a selected level of quality;

tweaking the selected element in different directions to produce the residual data; and

calculating an entropy of the residual data to identify which tweak to the selected element produces a set of residual data having the substantially lowest entropy.

10. The method as in claim 1, wherein the at least one upsample operation is non-linear.

11. The method as in claim 1, wherein the signal is an image.

12. The method as in claim 1, wherein the signal is a video signal.

13. The method as in claim 1, wherein the signal is an audio signal.

14. The method as in claim 1, wherein the signal is a volumetric signal.

15. A method of encoding at different levels of quality in a hierarchy, the method comprising:

having a signal as input;

utilizing at least one downsample operation to produce downsampled renditions of the signal at successively lower levels of quality in the hierarchy;

applying at least one upsample operation to a downsampled rendition of the signal at a first level of quality to produce an upsampled rendition of the signal at a second level of quality in the hierarchy, the second level of quality being higher than the first level of quality;

producing residual data indicating a difference between the downsampled rendition of the signal at the second level of quality and the upsampled rendition of the signal at the second level of quality; and

adjusting settings of elements in the downsampled rendition of the signal at the first level of quality to reduce an entropy of the residual data needed to reconstruct a rendition of the signal at the second level of quality.

16. A method of encoding at different levels of quality in a hierarchy, the method comprising:

having a signal as input;

utilizing at least one downsample operation to produce downsampled renditions of the signal at successively lower levels of quality in the hierarchy;

applying at least one upsample operation to a downsampled rendition of the signal at a first level of quality to produce an upsampled rendition of the signal at a second level of quality in the hierarchy, the second level of quality being higher than the first level of quality;

producing residual data indicating a difference between the downsampled rendition of the signal at the second level of quality and the upsampled rendition of the signal at the second level of quality;

for each of multiple upsample options used to upsample the rendition of the signal at the first level of quality into the rendition of the signal at the second level of quality, adjusting settings of elements in the downsampled rendition of the signal at the first level of quality to reduce an entropy of the residual data needed to reconstruct a rendition of the signal at the second level of quality; and

identifying which of the multiple upsample options and adjustments to the downsampled rendition of the signal at the first level of quality results in the substantially lowest entropy for the residual data.

17. A method of encoding at different levels of quality in a hierarchy, the method comprising:

having a signal as input;

utilizing at least one downsample operation to produce downsampled renditions of the signal at successively lower levels of quality in the hierarchy;

applying at least one upsample operation to a downsampled rendition of the signal at a first level of quality to produce an upsampled rendition of the signal at a second level of quality in the hierarchy, the second level of quality being higher than the first level of quality;

producing residual data indicating a difference between the downsampled rendition of the signal at the second level of quality and the upsampled rendition of the signal at the second level of quality;

implementing a continuous transform function to simulate a substantially rectangular function, the continuous transform function facilitating quantification of entropy associated with the residual data;

wherein the continuous transform function is a sigmoid function; and

wherein implementing the continuous transform function to simulate the substantially rectangular function further comprises;

generating an entropy function, a magnitude of the entropy function varying depending on different possible adjustments to the downsampled signal at the first level of quality; and

utilizing the entropy function to identify an adjustment to the downsampled signal at the first level of quality in which a respective entropy of the residual data for the adjustment is substantially minimal amongst the different possible adjustments.

18. A method of encoding at different levels of quality in a hierarchy, the method comprising:

having a signal as input;

utilizing at least one downsample operation to produce downsampled renditions of the signal at successively lower levels of quality in the hierarchy;

applying at least one upsample operation to a downsampled rendition of the signal at a first level of quality to produce an upsampled rendition of the signal at a second level of quality in the hierarchy, the second level of quality being higher than the first level of quality;

producing residual data indicating a difference between the downsampled rendition of the signal at the second level of quality and the upsampled rendition of the signal at the second level of quality;

repeatedly adjusting settings of elements in the downsampled rendition of the signal at the first level of quality to produce a tweaked rendition of the signal at the first level of quality;

upsampling the tweaked rendition of the signal at the first level of quality to produce the upsampled rendition of the signal at the second level of quality;

applying the residual data to the upsampled rendition of the signal at the second level of quality to produce an intermediate rendition of the signal at the second level of quality;

tweaking elements in the intermediate rendition of the signal at the second level of quality to produce a tweaked rendition of the signal at the second level of quality; and

generating a second set of residual data based on a difference between the upsampled rendition of the signal at the second level of quality and the tweaked rendition of the signal at the second level of quality.

19. A method of encoding at different levels of quality in a hierarchy, the method comprising:

having a signal as input;

utilizing at least one downsample operation to produce downsampled renditions of the signal at successively lower levels of quality in the hierarchy;

applying at least one upsample operation to a downsampled rendition of the signal at a first level of quality to produce an upsampled rendition of the signal at a second level of quality in the hierarchy, the second level of quality being higher than the first level of quality;

producing residual data indicating a difference between the downsampled rendition of the signal at the second level of quality and the upsampled rendition of the signal at the second level of quality; and

wherein utilizing the at least one downsample operation to produce the downsampled renditions of the signal at successively lower levels of quality further comprises;

selecting a level of quality J in which to downsample the signal, the level of quality J representing a highest level of quality in the hierarchy;

applying a selected downsampling operation to the signal at the level of quality J into a rendition of the signal at a level of quality J-1, the level of quality J-1 being a next lower level of quality beneath the level of quality J;

tweaking settings of values in the rendition of the signal at level of quality J-1 to identify a tweaked rendition of the signal at the level of quality J-1 that reduces an entropy of residual data needed to reconstruct the signal at level of quality J.

20. The method as in claim 19 further comprising:

applying a selected downsampling operation to the tweaked rendition of the signal at the level of quality J-1 into a rendition of the signal at a level of quality J-2, the level of quality J-2 being a next lower level of quality beneath the level of quality J-1;

tweaking settings of values in the rendition of the signal at level of quality J-2 to identify a tweaked rendition of the signal at the level of quality J-2 that reduce an entropy of residual data needed to reconstruct the tweaked rendition of the signal at level of quality J-1; and

repeating steps of applying a selected downsampling operation and tweaking settings in the hierarchy at pairs of successively lower levels of quality to produce the downsampled rendition of the signal at the second level of quality and the first level of quality.

21. The method as in claim 19 further comprising:

repeatedly testing which of multiple possible upsample operations and tweaked settings of values in the rendition of the signal at level of quality J-1 reduces an entropy of residual data needed to reconstruct the signal at level of quality J.

22. Computer-readable storage hardware having instructions stored thereon, the instructions, when carried out by a processing device, causing the processing device to perform operations of:

receiving a signal;

utilizing at least one downsample operation to produce downsampled renditions of the signal at successively lower levels of quality in the hierarchy;

applying at least one upsample operation to a downsampled rendition of the signal at a first level of quality to produce an upsampled rendition of the signal at a second level of quality in the hierarchy, the second level of quality being higher than the first level of quality; and

producing residual data indicating a difference between the downsampled rendition of the signal at the second level of quality and the upsampled rendition of the signal at the second level of quality;

parsing the downsampled rendition of the signal at the first level of quality into multiple regions of elements; and

simultaneously processing the multiple regions in parallel to identify adjustments to elements in the multiple regions to locally reduce an entropy associated with the residual data.

23. A computer system comprising:

a processor;

a memory unit that stores instructions associated with an application executed by the processor; and

an interconnect coupling the processor and the memory unit, causing the computer system to execute the application and perform operations of;

having a signal as input;

utilizing at least one downsample operation to produce downsampled renditions of the signal at successively lower levels of quality in the hierarchy;

applying at least one upsample operation to a downsampled rendition of the signal at a first level of quality to produce an upsampled rendition of the signal at a second level of quality in the hierarchy, the second level of quality being higher than the first level of quality; and

producing residual data indicating a difference between the downsampled rendition of the signal at the second level of quality and the upsampled rendition of the signal at the second level of quality;

wherein applying the at least one upsample operation further comprises;

applying multiple upsample operations to the downsampled rendition of the signal at the first level of quality; and

wherein producing the residual data includes identifying which of multiple upsample options applied to the downsampled rendition of the signal at the first level of quality produces residual data having the substantially lowest entropy.

24. The computer system as in claim 23, wherein either the at least one downsample operation is non-linear.

25. The computer system as in claim 23, wherein the computer system executes the application and further performs operations of:

adjusting settings of elements in the downsampled rendition of the signal at the first level of quality to reduce an entropy of the residual data needed to reconstruct the downsampled rendition of the signal at the second level of quality.

26. The computer system as in claim 23, wherein the computer system executes the application and further performs operations of:

for each of multiple upsample options used to upsample the rendition of the signal at the first level of quality into the rendition of the signal at the second level of quality, adjusting settings of elements in the downsampled rendition of the signal at the first level of quality to reduce an entropy of the residual data needed to reconstruct the downsampled rendition of the signal at the second level of quality; and

identifying which of the multiple upsample options and tweaks to the downsampled rendition of the signal at the first level of quality results in the substantially lowest entropy for the residual data.

27. The computer system as in claim 23, wherein the computer system executes the application and further performs operations of:

parsing the downsampled rendition of the signal at the first level of quality into multiple regions of elements; and

simultaneously processing the multiple regions in parallel to identify adjustments to elements in the multiple regions to locally reduce an entropy associated with the residual data.

28. The computer system as in claim 23, wherein applying the at least one upsample operation includes:

for each of multiple elements in the rendition of the signal at the first level of quality, repeating steps of;

selecting an element to tweak in the downsampled signal at the first level of quality;

tweaking the selected element in different directions;

producing the residual data at the second level of quality based on the tweaked selected element; and

calculating an entropy of the residual data at the second level of quality to identify which tweak to the selected element at the first level of quality produces a set of residual data at the second level of quality having the substantially lowest entropy.

29. The computer system as in claim 23, wherein applying the at least one upsample operation further comprises:

applying multiple upsample operations to the downsampled rendition of the signal at the first level of quality; and

wherein producing the residual data includes identifying which of multiple upsample options applied to the downsampled rendition of the signal at the first level of quality produces residual data having the substantially lowest entropy.

30. The computer system as in claim 23, wherein the computer system executes the application and further performs operations of:

adjusting settings of elements in the downsampled rendition of the signal at the first level of quality to produce a tweaked rendition of the signal at the first level of quality;

upsampling the tweaked rendition of the signal at the first level of quality to produce the upsampled rendition of the signal at the second level of quality;

applying the residual data to the upsampled rendition of the signal at the second level of quality to produce an intermediate rendition of the signal at the second level of quality;

tweaking elements in the intermediate rendition of the signal at the second level of quality to produce a tweaked rendition of the signal at the second level of quality; and

generating a second set of residual data based on a difference between the upsampled rendition of the signal at the second level of quality and the tweaked rendition of the signal at the second level of quality.

31. The computer system as in claim 23, wherein utilizing the at least one downsample operation to produce the downsampled renditions of the signal at successively lower levels of quality further comprises:

selecting a level of quality J in which to downsample the signal, the level of quality J representing a highest level of quality in the hierarchy;

applying a selected downsampling operation to the signal at the level of quality J into a rendition of the signal at a level of quality J-1, the level of quality J-1 being a next lower level of quality beneath the level of quality J;

tweaking settings of values in the rendition of the signal at level of quality J-1 to identify a tweaked rendition of the signal at the level of quality J-1 that reduces an entropy of residual data needed to reconstruct the signal at level of quality J.

32. The computer system as in claim 31, wherein the computer system executes the application and further performs operations of:

testing which of multiple possible upsample operations and tweaked settings of values in the rendition of the signal at level of quality J-1 reduces an entropy of residual data needed to reconstruct the signal at level of quality J.