Context based Inverse Mapping Method for Layered Codec

• US 20130177066A1
• Filed: 01/02/2013
• Published: 07/11/2013
• Est. Priority Date: 01/09/2012
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Ye, Yan, Tourapis, Alexandros, Yin, Peng, He, Yuwen
• Granted Patent: US 9,549,194 B2
• Time in Patent Office: Days
• US Class Current: 375/240.02
• CPC Class Codes: H04N 19/10 using adaptive coding
• CPC Class Codes: H04N 19/103 Selection of coding mode or of prediction mode
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/182 the unit being a pixel
• CPC Class Codes: H04N 19/187 the unit being a scalable video layer
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 19/36 Scalability techniques involving formatting the layers as a function of picture distortion after decoding, e.g. signal-to-noise [SNR] scalability
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/537 Motion estimation other than block-based
• CPC Class Codes: H04N 19/55 Motion estimation with spatial constraints, e.g. at image or region borders
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution

Abstract

Context based inverse mapping methods are provided. An image with pixel values within a lower dynamic range can be mapped to an image with pixel values within a higher dynamic range by utilizing context information associated with pixels in the lower dynamic range image.

Claims

1. A method of generating a second pixel based on a first pixel, wherein a first image comprises the first pixel and a second image comprises the second pixel, the first image having a lower dynamic range and the second image having a higher dynamic range, the method comprising:

providing a value of the first pixel;

generating context information based on at least one pixel neighboring the first pixel; and

computing a value of the second pixel based on the value of the first pixel and the context information of the at least one pixel to generate the second pixel.

2. The method according to claim 1, wherein the at least one pixel neighboring the first pixel comprises one or more of spatial neighbors and temporal neighbors of the first pixel.

3. The method according to claim 1, wherein the generating of context information comprises providing values of the at least one pixel neighboring the first pixel.

4. The method according to claim 3, further comprising:

generating statistical information based on the provided values of the at least one pixel neighboring the first pixel,

wherein;

the statistical information comprises at least one of mean of the provided values, variance of the provided values, texture associated with the provided values, and gradient associated with the provided values, and

the computing is based on the statistical information.

5. The method according to claim 1 wherein the computing comprises:

estimating an inverse mapping function based on the value of the first pixel and the generated context information, the estimated inverse mapping function being adapted to map a value from the lower dynamic range to the higher dynamic range.

6. The method according to claim 5, wherein the estimated inverse mapping function comprises at least one of a linear function, a polynomial function, a multivariate multiple regression function, a slope offset power function, and a spline function.

7. The method according to claim 5, wherein the estimated inverse mapping function is performed using a lookup table.

8. The method according to claim 7, wherein the lookup table is differentially coded.

9. The method according to claim 5, wherein the estimated inverse mapping function is based on statistical information from each pixel of the first image.

10. The method according to claim 1, wherein the computing is performed by at least one of a two-dimensional separable interpolation filter, a two-dimensional non-separable interpolation filter, and an adaptive interpolation filter.

11. The method according to claim 1, wherein the generating of context information comprises performing one of:

computing a weighted average of the at least one pixel neighboring the first pixel;

computing a linear combination of the at least one pixel neighboring the first pixel;

computing a gradient of the at least one pixel neighboring the first pixel;

computing texture information based on the at least one pixel neighboring the first pixel; and

computing a gradient covariance matrix based on the at least one pixel neighboring the first pixel.

12. A method of generating a second image based on a first image by inverse mapping the first image, the first image comprising pixels with values of a lower dynamic range and the second image comprising pixels with values of a higher dynamic range, the method comprising:

providing the first image;

generating context information for one or more pixels in the first image, wherein context information associated with any particular pixel in the first image is based on at least one pixel neighboring the particular pixel;

estimating an inverse mapping function from the generated context information and values of the one or more pixels in the first image, the inverse mapping function being adapted to map a value from the lower dynamic range to the higher dynamic range; and

applying the inverse mapping function to the one or more pixels of the first image and, for each remaining pixel of the first image, inverse mapping the remaining pixel from the lower dynamic range to the higher dynamic range based on pixel value associated with the remaining pixel to generate the second image.

13. The method according to claim 12, wherein the inverse mapping of the remaining pixels comprises at least one of polynomial mapping, linear mapping, table lookup, multivariate multiple regression, or slope offset power.

14. The method according to claim 13, wherein the lookup table is differentially coded.

15. The method according to claim 12, wherein the generating of context information comprises, for any particular pixel in the first image, providing values of at least one pixel neighboring the particular pixel.

16. The method according to claim 15, further comprising:

generating statistical information based on the provided values of the at least one pixel neighboring the particular pixel,

wherein;

the statistical information comprising at least one of mean of the provided values, variance of the provided values, texture associated with the provided values, and gradient associated with the provided values, and

the estimating is based on the statistical information.

17. The method according to claim 12, wherein the estimated inverse mapping function comprises at least one of a linear function, a polynomial function, a lookup table, a multivariate multiple regression function, a slope offset power function, and a spline function.

18. The method according to claim 17, wherein the lookup table is differentially coded.

19. The method according to claim 12, wherein the applying is performed by at least one of a two-dimensional separable interpolation filter, a two-dimensional non-separable interpolation filter, and an adaptive interpolation filter.

20. The method according to claim 12, wherein the generating of context information comprises, for the particular pixel, one of:

computing a weighted average of the at least one pixel neighboring the particular pixel;

computing a linear combination of the at least one pixel neighboring the particular pixel;

computing a gradient of the at least one pixel neighboring the particular pixel;

computing texture information based on the at least one pixel neighboring the particular pixel; and

computing a gradient covariance matrix based on the at least one pixel neighboring the particular pixel.

21. A method of encoding image information, the method comprising:

providing a first image and a second image, the first image comprising a lower dynamic range representation of the image information and the second image comprising higher dynamic range representation of the image information;

encoding the first image to generate a first layer bitstream;

reconstructing the first image to obtain a reconstructed first image, wherein the reconstructed first image is adapted to be stored in a first reference picture buffer;

generating context information for one or more pixels in the reconstructed first image, wherein context information associated with any particular pixel in the one or more pixels based on at least one pixel neighboring the particular pixel;

estimating an inverse mapping function from the generated context information and values of the one or more pixels in the reconstructed first image, the inverse mapping function being adapted to map a value from the lower dynamic range to the higher dynamic range;

computing an estimated second image based on applying the inverse mapping function to the one or more pixels of the reconstructed first image and, for each remaining pixel of the reconstructed first image, inverse mapping the remaining pixel from the lower dynamic range to the higher dynamic range based on pixel value associated with the remaining pixel to generate the estimated second image, wherein the estimated second image is adapted to be stored in a second reference picture buffer; and

encoding the estimated second image to generate a second layer bitstream.

22. The method according to claim 21, wherein the inverse mapping remaining pixels comprises at least one of polynomial mapping, linear mapping, table lookup, multivariate multiple regression, or slope offset power.

23. The method according to claim 22, wherein the lookup table is differentially coded.

24. The method according to claim 21, wherein the generating of context information comprises performing one of:

computing a weighted average of the at least one pixel neighboring the particular pixel;

computing a linear combination of the at least one pixel neighboring the particular pixel;

computing a gradient of the at least one pixel neighboring the particular pixel;

computing texture information based on the at least one pixel neighboring the particular pixel; and

computing a gradient covariance matrix based on the at least one pixel neighboring the particular pixel.

25. The method according to claim 21, wherein:

the generating of context information comprises performing, for any particular pixel in the one or more pixels of the reconstructed first image, at least one of;

(i) computing a weighted average of the at least one pixel neighboring the particular pixel to generate a first set of context information;

(ii) computing a linear combination of the at least one pixel neighboring the particular pixel to generate a second set of context information;

(iii) computing a gradient of the at least one pixel neighboring the particular pixel to generate a third set of context information;

(iv) computing texture information based on the at least one pixel neighboring the particular pixel to generate a fourth set of context information; and

(v) computing a gradient covariance matrix based on the at least one pixel neighboring the particular pixel to generate a fifth set of context information; and

the estimating of the inverse mapping function comprises;

estimating a corresponding inverse mapping function for each generated set of context information from among the generated sets of context information; and

selecting the inverse mapping function corresponding to one set of context information from among the generated sets of context information, based on a metric.

26. The method according to claim 25, wherein the estimating of the inverse mapping function comprises:

for each generated set of context information;

estimating an inverse mapping function from the generated set of context information and values of the one or more pixels in the reconstructed first image, the estimated inverse mapping function being adapted to map a value from the lower dynamic range to the higher dynamic range;

computing an estimated second image based on applying the inverse mapping function to the one or more pixels of the reconstructed first image and, for each remaining pixel of the reconstructed first image, inverse mapping the remaining pixel from the lower dynamic range to the higher dynamic range based on pixel value associated with the remaining pixel to generate the estimated second image, wherein the estimated second image is adapted to be stored in a second reference picture buffer; and

computing a metric based on a difference between the estimated second image and the provided second image; and

selecting the estimated inverse mapping function associated with the generated set of context information corresponding to the lowest computed metric.

27. The method according to claim 26, wherein the inverse mapping function comprises at least one of polynomial mapping, linear mapping, table lookup, multivariate multiple regression, or slope offset power.

28. The method according to claim 27, wherein the lookup table is differentially coded.

29. The method according to claim 26, wherein the computing of the metric comprises:

computing a distortion metric as the computed metric,

wherein the distortion metric is a function of the difference between the estimated second image and the provided second image and a number of bits used to signal mode of the generating of the context information.

30. The method according to claim 21, further comprising, prior to encoding the estimated second image:

subtracting the estimated second image from the provided second image to obtain a residual,

wherein the encoding comprises encoding the residual.

31. The method according to claim 30, further comprising resampling and/or quantizing the residual prior to encoding the residual.

32. The method according to claim 21, further comprising signaling the estimated inverse mapping function.

33. The method according to claim 32, wherein the signaling comprises generating a header file comprising information associated with the estimated inverse mapping function.