Regression-Based Learning Model for Image Upscaling

• US 20130034313A1
• Filed: 08/02/2012
• Published: 02/07/2013
• Est. Priority Date: 08/05/2011
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Cohen, Scott D., Chien, Jen-Chan, Kong, Sarah A., Yang, Jianchao, Lin, Zhe
• Granted Patent: US 8,655,109 B2
• Time in Patent Office: Days
• US Class Current: 382/299
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution

Abstract

Methods and systems for a regression-based learning model in image upscaling are disclosed. In one embodiment, a set of image patch pairs for each of a set of images is generated. Each of the image patch pairs contains a natural image and a corresponding downscaled lower-resolution image. A regression model based at least in part on the set of image patch pairs is defined. The regression model represents a gradient of a function of the downscaled lower-resolution image. An image is upscaled based at least in part on the regression model.

Claims

1. A method, the method comprising:

using one or more processors to perform

generating a set of image patch pairs for each of a set of images, wherein each of the image patch pairs contains a natural image and a corresponding downscaled lower-resolution image;

defining a regression model based at least in part on the set of image patch pairs, wherein the regression model represents a gradient of a function of the downscaled lower-resolution image; and

upscaling an image based at least in part on the regression model.

2. The method of claim 1, wherein the defining the regression model based at least in part on the set of image patch pairs further comprises defining a regression model based at least in part on the set of image patch pairs using an external database.

3. The method of claim 1, wherein the defining the regression model based on the set of image patch pairs further comprises defining a regression model based on a set of traning patch pairs sampled from natural images.

4. The method of claim 1, wherein the defining the regression model based on the set of image patch pairs further comprises defining a regression model as a weighted average of regression model predictions.

5. The method of claim 1, wherein the defining the regression model based on the set of image patch pairs further comprises defining a regression model as a weighted average of first order regression model predictions.

6. The method of claim 1, wherein the defining the regression model based on the set of image patch pairs further comprises defining a regression model as a weighted average of second-or-higher order regression model predictions.

7. The method of claim 1, wherein the upscaling the image based at least in part on the regression model further comprises estimating a high resolution patch for a set of spatially overlapping, densely sampled low resolution patches to generate a set of overlapping pixels.

8. A system, comprising:

at least one processor; and

a memory comprising program instructions, wherein the program instructions are executable by the at least one processor to;

generate a set of image patch pairs for each of a set of images, wherein each of the image patch pairs contains a natural image and a corresponding downscaled lower-resolution image;

define a regression model based at least in part on the set of image patch pairs, wherein the regression model represents a gradient of a function of the downscaled lower-resolution image; and

upscale an image based at least in part on the regression model.

9. The system of claim 8, wherein the program instructions executable by the at least one processor to define the regression model based at least in part on the set of image patch pairs further comprise program instructions executable by the at least one processor to define a regression model based at least in part on the set of image patch pairs using an external database.

10. The system of claim 8, wherein the program instructions executable by the at least one processor to define the regression model based on the set of image patch pairs further comprise program instructions executable by the at least one processor to define a regression model based on a set of traning patch pairs sampled from natural images.

11. The system of claim 8, wherein the program instructions executable by the at least one processor to define the regression model based on the set of image patch pairs further comprise program instructions executable by the at least one processor to define a regression model as a weighted average of regression model predictions.

12. The system of claim 8, wherein the program instructions executable by the at least one processor to define the regression model based on the set of image patch pairs further comprise program instructions executable by the at least one processor to define a regression model as a weighted average of first order regression model predictions.

13. The system of claim 8, wherein the program instructions executable by the at least one processor to upscale the image based at least in part on the regression model further comprise program instructions executable by the at least one processor to, for each low resolution image patch in an upsampled image intermediate, identify a match in a local neighborhood of the low frequency band image intermediate to the estimate a result image.

14. The system of claim 8, wherein the program instructions executable by the at least one processor to upscale the image based at least in part on the regression model further comprise program instructions executable by the at least one processor to estimate a high resolution patch for a set of spatially overlapping, densely sampled low resolution patches to generate a set of overlapping pixels.

15. A non-transitory computer-readable storage medium storing program instructions, wherein the program instructions are computer-executable to implement:

using one or more processors to perform

generating a set of image patch pairs for each of a set of images, wherein each of the image patch pairs contains a natural image and a corresponding downscaled lower-resolution image;

defining a regression model based at least in part on the set of image patch pairs, wherein the regression model represents a gradient of a function of the downscaled lower-resolution image; and

upscaling an image based at least in part on the regression model.

16. The non-transitory computer-readable storage medium of claim 15, wherein the program instructions computer-executable to implement the defining the regression model based at least in part on the set of image patch pairs further comprise program instructions computer-executable to implement defining a regression model based at least in part on the set of image patch pairs using an external database.

17. The non-transitory computer-readable storage medium of claim 15, wherein the program instructions computer-executable to implement the defining the regression model based on the set of image patch pairs further comprise program instructions computer-executable to implement defining a regression model based on a set of traning patch pairs sampled from natural images.

18. The non-transitory computer-readable storage medium of claim 15, wherein the program instructions computer-executable to implement the defining the regression model based on the set of image patch pairs further comprise program instructions computer-executable to implement defining a regression model as a weighted average of regression model predictions.

19. The non-transitory computer-readable storage medium of claim 15, wherein the program instructions computer-executable to implement defining the regression model based on the set of image patch pairs further comprise program instructions computer-executable to implement defining a regression model as a weighted average of first order regression model predictions.

20. The non-transitory computer-readable storage medium of claim 15, wherein the program instructions computer-executable to implement upscaling the image based at least in part on the regression model further comprise program instructions computer-executable to implement, for each low resolution image patch in an upsampled image intermediate, identifying a match in a local neighborhood of the low frequency band image intermediate to the estimate a result image.