Image superresolution through edge extraction and contrast enhancement

• US 7,613,363 B2
• Filed: 06/23/2005
• Issued: 11/03/2009
• Est. Priority Date: 06/23/2005
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Corduneanu, Adrian, Hoppe, Hugues, Platt, John, Renshaw, Erin
• Primary Examiner(s): Ahmed; Samir A.
• Assistant Examiner(s): TSAI, TSUNG YIN
• Application Number:
  US11/165,525
• Publication Number:
  US 20060290950A1
• Time in Patent Office: 1,594 Days
• Field of Search: 382/299
• US Class Current: 382/299
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation

Abstract

A technique for generating high-resolution bitmaps from low-resolution bitmaps. A low-resolution bitmap is magnified to form a magnified image. Edge detection is performed on the magnified image to find high contrast edges. A plurality of image patches of the magnified image are generated. These images patches are analyzed by performing connected components analysis on each of them using the high contrast edges to produce a plurality of foreground and background decisions determining whether a portion of an image patch is a background or a foreground region. Then the contrast of one or more pixels in each of the plurality of image patches is enhanced based on the foreground and background decisions. Finally, the system and method of the invention combines the luminance of the enhanced output pixels with the color values generated by the magnification algorithm. This produces a high-resolution bitmap from the contrast-enhanced pixels.

Claims

1. A method for generating high-resolution bitmaps from low-resolution bitmaps, comprising the process actions of:

using a computing device for, magnifying a low-resolution bitmap to form a magnified image;

performing edge detection on the magnified image to produce an edge map;

generating a plurality of image patches of the magnified image;

performing connected components analysis coupled to each of the plurality of image patches and the edge map, to identify one or more foreground and background regions for each image patch, wherein performing connected components analysis further comprises;

determining foreground and background regions of each image patch; and

computing a blending factor based on the mean luminances of the foreground and background regions;

enhancing the contrast of one or more pixels in each of the plurality of image patches, said enhancing depending on whether the pixel is in the foreground or background region; and

producing a high-resolution bitmap from the contrast-enhanced pixels.

2. The method of claim 1 wherein the magnifying process action comprises one of;

using bilinear interpolation to magnify the image;

or

using bicubic interpolation to magnify the image.

3. The method of claim 1 wherein the edge detecting process action comprises using a Canny edge detector to find the high contrast edges used to generate the edge map.

4. The method of claim 3 wherein the edge detecting process action further comprises smoothing the edges of the edge map.

5. The method of claim 1 wherein the size of each image patch depends on the magnification used in the magnification process.

6. The method of claim 1 wherein the process action of performing a connected components analysis employs a recursive flood fill component analysis.

7. The method of claim 1 wherein the process action of performing a connected components analysis employs a non-recursive flood fill component analysis.

8. The method of claim 1 wherein the process action of performing connected components analysis comprises finding all pixels that are connected to the center pixel in the image patch whose path of connection does not cross an edge pixel within the image patch.

9. The method of claim 1 wherein the process action of enhancing the contrast of one or more pixels comprises enhancing the contrast of the center pixel in each image patch to make it more representative of the luminance of the foreground or background region the center pixel is in.

10. The method of claim 9 wherein an alpha value is used to blend the luminance of the center pixel of each image patch and a most representative pixel for the luminance in the foreground or background region that the center pixel is in to produce the contrast enhanced output pixels.

11. The method of claim 1 wherein the process action of producing a high resolution bitmap from the contrast-enhanced pixels comprises combining luminances of the contrast-enhanced output pixels with color values produced by a magnification algorithm used to form the magnified image.

12. The method of claim 1 wherein the process action of magnifying a low-resolution bitmap to form a magnified image comprises magnifying a low resolution image that contains one or more of:

graphics;

text;

or

line art.

13. A computer-readable storage medium having computer-executable instructions stored thereon for generating a high-resolution bitmap from a low-resolution bitmap, said computer-executable instructions comprising:

magnifying a low-resolution image to form a magnified image;

performing edge detection on the magnified image to produce an edge map;

generating a plurality of image patches of the magnified image;

performing connected components analysis on each of the plurality of image patches and the edge map, to produce a plurality of foreground/background decisions;

enhancing the contrast of one or more pixels in each of the plurality of image patches, said enhancing depending on the foreground/background decisions, wherein the computer-executable instruction for enhancing the contrast of one or more pixels comprises;

inputting an image patch;

computing foreground and background regions based on the foreground and background decisions;

computing an alpha value based on the mean luminance of the background and foreground region;

finding the most representative pixel for the foreground and background regions; and

computing the enhanced output pixel luminance from the input pixel luminance using the most representative pixel for the luminance of the foreground or background region and the alpha value; and

producing a high-resolution bitmap from the contrast-enhanced pixels.

14. A system for generating an enhanced image, comprising:

a general purpose computing device;

a computer program encoded on a computer-readable medium comprising program modules executable by the general purpose computing device, wherein the computing device is directed by the program modules of the computer program to, perform edge detection on an image to produce an edge map;

generate a plurality of image patches of the image using a plurality of windows applied to the image;

perform connected components analysis on each of the plurality of image patches of the image to find all pixels that are connected to the center pixel of the image patch whose path of connection does not cross an edge pixel, wherein the program module to perform connected components analysis further comprises a sub-module to;

determine foreground and background regions of each image patch; and

compute a blending factor based on the mean luminances of the foreground and the background regions;

enhance the contrast of one or more pixels in each of the plurality of image patches, said enhancing depending on the results of connected components analysis; and

produce an enhanced image from the contrast-enhanced pixels.

15. The system of claim 14 wherein the program module to detect the edges employs a Canny edge detector.

16. The system of claim 14 wherein in the program module to generate a plurality of image patches of the magnified image using a plurality of windows applied the magnified image the size of each window depending on the magnification used to create the magnified image.

17. The system of claim 14 wherein the program module to enhance the contrast of one or more pixels comprises a sub-module to:

for each image patch that contains an edge pixel,

enhance the luminance of the center pixel by using the blending factor to blend between the luminance of the center pixel of the image patch and the most representative pixel of the luminance of the background or foreground region the image patch is in.

18. The system of claim 14 wherein the plurality of windows are overlapping.