IMAGE SUPER-RESOLUTION USING GRADIENT PROFILE PRIOR

• US 20100086227A1
• Filed: 10/04/2008
• Published: 04/08/2010
• Est. Priority Date: 10/04/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Sun, Jian, Shum, Heung-Yeung
• Granted Patent: US 9,064,476 B2
• Time in Patent Office: Days
• US Class Current: 382/254
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G09G 2340/0407 Resolution change, inclusive of the use of different resolutions for different screen areas
• CPC Class Codes: G09G 5/391 Resolution modifying circuits, e.g. variable screen formats

Abstract

Described is a technology by which a low-resolution image is processed into a high-resolution image, including by performing processing in the gradient domain. A gradient profile corresponding to the lower-resolution image is transform into a sharpened image gradient. A high-resolution gradient profile is estimated from a low-resolution gradient profile, e.g., by multiplying the low-resolution gradient profile by a transform ratio that is based upon learned shape parameters, learned sharpness values and a curve distance to an edge pixel along the gradient profile. The transform ratio is used to transform a low-resolution gradient field to a high-resolution gradient field. Reconstructing the higher-resolution image is performed by using the high-resolution gradient field as a gradient domain constraint, e.g., in along with a reconstruction constraint obtained from image domain data. An energy function is minimized by enforcing the gradient domain constraint and the reconstruction constraint, e.g., by performing a gradient descent algorithm.

Claims

1. In a computing environment, a method comprising, processing a lower-resolution image into a higher-resolution image, including by determining a gradient profile corresponding to the lower-resolution image, transforming the gradient profile into a sharpened image gradient, and reconstructing the higher-resolution image based upon the sharpened image gradient.

2. The method of claim 1 wherein transforming the gradient profile comprises estimating a high-resolution gradient profile from a low-resolution gradient profile.

3. The method of claim 2 wherein estimating the high-resolution gradient profile from the low-resolution gradient profile comprises multiplying the low-resolution gradient profile by a transform ratio based upon shape parameters, sharpness values and a curve distance to an edge pixel along the gradient profile.

4. The method of claim 3 further comprising, learning the shape parameters and sharpness values by analyzing other images.

5. The method of claim 3 further comprising, using the transform ratio to transform a low-resolution gradient field to a high-resolution gradient field.

6. The method of claim 5 wherein reconstructing the higher-resolution image comprises using the high-resolution gradient field as a gradient domain constraint.

7. The method of claim 5 wherein reconstructing the higher-resolution image further comprises using a reconstruction constraint.

8. The method of claim 7 wherein reconstructing the higher-resolution image comprises minimizing an energy function by enforcing the gradient domain constraint and the reconstruction constraint.

9. The method of claim 8 wherein reconstructing the higher-resolution image comprises minimizing an energy function by enforcing the gradient domain constraint and the reconstruction constraint.

10. The method of claim 9 wherein enforcing the gradient domain constraint and the reconstruction constraint comprises performing a gradient descent algorithm.

11. The method of claim 1 wherein reconstructing the higher-resolution image comprises solving Poisson equations on the transformed gradient field.

12. In a computing environment, a system comprising, an image processing mechanism that processes a lower-resolution image into a higher-resolution image, including by determining a high-resolution gradient field via a transformation from a low-resolution gradient field, and minimizing an energy function by using the high-resolution gradient field as a gradient domain constraint and using a reconstruction constraint.

13. The system of claim 12 wherein the image processing mechanism determines the high-resolution gradient field by multiplying the low-resolution gradient field by a ratio determined from shape parameters, sharpness values and a curve distance to an edge pixel along a gradient profile.

14. The system of claim 13 wherein the image processing mechanism estimates a high-resolution gradient profile from a low-resolution gradient profile.

15. The system of claim 12 wherein the image processing mechanism minimizes the energy function by performing a gradient descent algorithm.

16. One or more computer-readable media having computer-executable instructions, which when executed perform steps, comprising:

determining a gradient profile corresponding to a lower-resolution image, estimating a high-resolution gradient profile from a low-resolution gradient profile;

transforming a low-resolution gradient field to a high-resolution gradient field; and

reconstructing a higher-resolution image relative to the lower-resolution image by using a high-resolution gradient field as a gradient domain constraint.

17. The one or more computer-readable media of claim 16 wherein reconstructing the higher-resolution image further comprises using a reconstruction constraint.

18. The one or more computer-readable media of claim 16 wherein reconstructing the higher-resolution image comprises minimizing an energy function by enforcing the gradient domain constraint and the reconstruction constraint.

19. The one or more computer-readable media of claim 16 wherein enforcing the gradient domain constraint and the reconstruction constraint comprises performing a gradient descent algorithm.

20. The one or more computer-readable media of claim 16 wherein estimating the high-resolution gradient profile from a low-resolution gradient profile comprises multiplying the low-resolution gradient profile by a transform ratio that is based on shape parameters, sharpness values and a curve distance to an edge pixel along the gradient profile, and wherein the transforming a low-resolution gradient field to the high-resolution gradient field comprises multiplying the low-resolution gradient field by the transform ratio.