Scalable resolution enhancement of a video image

• US 20020140854A1
• Filed: 03/30/2001
• Published: 10/03/2002
• Est. Priority Date: 03/30/2001
• Status: Active Grant

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Lan, Tse-Hua
• Granted Patent: US 6,717,622 B2
• Time in Patent Office: Days
• US Class Current: 348/448
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 3/403 Edge-driven scaling; Edge-based scaling
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: H04N 11/24 High-definition television systems
• CPC Class Codes: H04N 21/440218 by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 conversion of standards in analogue television systems H04N7/01
• CPC Class Codes: H04N 5/142 Edging; Contouring
• CPC Class Codes: H04N 5/208 for compensating for attenuation of high frequency components, e.g. crispening, aperture distortion correction
• CPC Class Codes: H04N 5/46 for receiving on more than one standard at will deflecting circuits of multi-standard receivers H04N3/27
• CPC Class Codes: H04N 7/012 Conversion between an interlaced and a progressive signal for graphics images G09G2310/0229
• CPC Class Codes: H04N 7/0125 one of the standards being a high definition standard
• CPC Class Codes: H04N 7/0135 involving interpolation processes interpolation-based image scaling G06T3/4007; interpolation for video coding H04N19/587, H04N19/59

Abstract

A system and method for enhancing resolution in a video image includes acquiring a relatively low resolution video signal and applying a peaking function and analyzing the peaked signal to identify potential edges. The low resolution signal is then upconverted to a higher resolution format. Actual edges in the image are detected and linked in the higher resolution format, and a luminance transition improvement function is applied to the edges to sharpen the image. Analysis of the image while at low resolution reduces computation time, yet still produces a high quality output image.

Claims

1. A method of scaling resolution enhancement in a video signal, comprising:

acquiring a first resolution video signal;

applying a peaking function to said first resolution video signal to create a peaked signal;

analyzing said peaked signal to identify possible edge pixels in said first resolution video signal;

upconverting said peaked signal to a higher resolution video signal;

applying an edge detection function to said higher resolution video signal at pixels that correspond to said first resolution video signal possible edge pixels, thereby identifying higher resolution video signal edge pixels;

applying an edge linking function to said higher resolution video signal edge pixels;

applying a luminance transition improvement function to said higher resolution video signal edge pixels; and

outputting said higher resolution video signal.

2. A method as claimed in claim 1 wherein said peaking function further comprises:

obtaining the convolution of said first resolution video signal; and

adding a result of said convolution to said acquired first resolution video signal.

3. A method as claimed in claim 1 wherein analyzing said peaking function further comprises:

calculating a first derivative of said peaked signal; and

designating a peaked signal pixel as a potential edge if said first derivative at said first resolution video signal pixel is greater than a first threshold value.

4. A method as claimed in claim 1 wherein said acquired first resolution video signal is in an interlaced format.

5. A method as claimed in claim 4 wherein said first resolution video signal is delivered in an interlaced format and converted to a progressive format prior to application of said peaking function.

6. A method as claimed in claim 1 wherein said edge detection function further comprises:

calculating a second derivative of said upconverted signal;

if said second derivative indicates that there has been a zero crossing, calculating a third derivative of said peaked signal;

identifying said pixel as an edge if said third derivative is greater than a second threshold value.

7. A method as claimed in claim 6 wherein said second threshold value is equal to a multiple of said first derivative and a constant.

8. A method as claimed in claim 1 wherein said edge linking function further comprises:

identifying isolated edge pixels; and

removing said identified isolated edge pixels.

9. A method as claimed in claim 1 wherein said luminance transition improvement function comprises:

dividing a higher resolution video signal edge pixel into multiple sub-regions;

analyzing each subregion to determine a location, inside said pixel, where an edge is located;

using said edge location to select a filter; and

applying said selected filter to said higher resolution edge pixel.

10. A method as claimed in claim 9 wherein said edge location is defined as:

c=round[N*L″

(x−

1)/(abs(L″

(x−

1))+abs(L″

(x)))]

where c is the mathematical rounding function, N represents a number of subpixels into which each pixel is divided, x represents the pixel that is being evaluated and L″

represents the second derivative of the higher resolution video signal.

11. A method as claimed in claim 9 wherein said edge location is used as a coefficient that is mapped to filters that are stored in a look up table.

12. A method as claimed in claim 11 further comprising

selecting additional pixels that neighbor said higher resolution video signal edge pixel;

selecting a filter from said look up table for each additional selected pixel; and

applying said selected filters to said higher resolution video signal edge pixel and to said additional selected pixels.

14. A method as claimed in claim 13 wherein said edge detection function further comprises:

calculating a second derivative of said higher resolution video signal matrix;

calculating a third derivative of said higher resolution video signal matrix;

if a product of a second derivative of two adjacent pixels in said higher resolution video signal matrix is less than or equal to zero and said third derivative of one or more of said two adjacent pixels is greater than a second threshold value, identifying said one or more of said two adjacent pixels as an edge; and

processing at least a third pixel in said matrix and identifying said at least third pixel as an edge if a product of a second derivative of said at least third pixel and a second derivative of a pixel adjacent to said at least third pixel in said matrix is equal to zero and said third derivative of said at least third pixel is greater than a second threshold value.

15. A method as claimed in claim 13 wherein said luminance transition improvement function comprises:

dividing a higher resolution video signal edge pixel into multiple sub-regions;

analyzing each subregion to determine a location, inside said pixel, where an edge is located;

using said edge location to select a filter; and

applying said selected filter to said higher resolution edge pixel.

16. A method as claimed in claim 15 wherein said edge location is defined as:

c=round[N*L″

(x−

1)/(abs(L″

(x−

1))+abs(L″

(x)))]

where c is the mathematical rounding function, N represents a number of subpixels into which each pixel is divided, x represents the pixel that is being evaluated and L″

represents the second derivative of the higher resolution video signal.

17. A method as claimed in claim 15 wherein said edge location is used as a coefficient that is mapped to filters that are stored in a look up table.

18. A method as claimed in claim 17 further comprising

selecting additional pixels that neighbor said higher resolution video signal edge pixel;

selecting a filter from said look up table for each additional selected pixel; and

applying said selected filters to said higher resolution video signal edge pixel and to said additional selected pixels.

19. A method as claimed in claim 14 wherein said higher resolution video signal is generated in a progressive format and converted to an interlaced format prior to being displayed.

20. A method as claimed in claim 19 wherein said at least second pixel lies two columns from said immediately previously processed pixel.

13. A method of scaling resolution enhancement in a video signal, comprising:

acquiring a first resolution video signal matrix;

applying a peaking function to said first resolution video signal matrix, to create a peaked signal matrix;

analyzing said peaked signal matrix to identify potential edge pixels in said first resolution video signal matrix;

upconverting said peaked first resolution video signal matrix to a higher resolution video signal matrix;

applying an edge detection function to said higher resolution video signal matrix at pixels that correspond to said potential edge pixels identified in said first resolution video signal matrix, thereby identifying higher resolution video signal matrix edge pixels;

applying an edge linking function to said higher resolution video signal matrix at pixels that are identified as higher resolution video signal matrix edge pixels;

applying a luminance transition improvement function to said higher resolution video signal matrix at pixels that are identified as higher resolution video signal matrix edge pixels; and

outputting said higher resolution video signal matrix.

21. A method of enhancing the resolution of a video signal, comprising:

acquiring a first resolution video signal;

increasing a perceived contrast of said first resolution video signal, to generate a peaked video signal;

analyzing said peaked video signal to identify potential edge pixels in said first resolution video signal;

upconverting said peaked video signal to a higher resolution video signal;

applying an edge detection function to said higher resolution video signal to identify actual edge pixels in said higher resolution video signal;

applying an edge linking function to said higher resolution video signal at pixels that are identified as said actual higher resolution video signal edge pixels;

applying a luminance transition improvement function to said higher resolution video signal; and

outputting said higher resolution video signal.

22. A method of enhancing the resolution of a video signal, comprising:

acquiring a first resolution video signal;

applying a peaking function to said first resolution video signal to create a peaked signal, wherein said peaking function is calculated by obtaining the convolution of said first resolution video signal and adding a result of said convolution to said acquired first resolution video signal;

analyzing said peaked signal to identify potential edge pixels in said first resolution video signal;

upconverting said peaked signal to a higher resolution video signal;

applying an edge detection function to said higher resolution video signal at pixels that correspond to said first resolution video signal potential edge pixels, thereby identifying higher resolution video signal edge pixels, wherein said edge detection function is obtained by calculating a second derivative of said peaked signal, and if a product of a second derivative of two adjacent pixels in said higher resolution video signal is less than or equal to zero, calculating a third derivative of said peaked signal, and identifying said pixel as an edge if said third derivative is greater than a second threshold value;

applying an edge linking function to said higher resolution video signal at pixels that are identified as higher resolution video signal edge pixels;

applying a luminance transition improvement function to said higher resolution video signal at pixels that are identified as higher resolution video signal edge pixels; and

outputting said higher resolution video signal.

23. A device for scaling resolution enhancement in a video signal, comprising:

a receiver that accepts the video signal at a first resolution;

a peaking system which increases a perceived contrast of the accepted video signal to create a peaked signal;

textured signal edge detector which identifies possible edge pixels in said peaked signal;

an upconverter which transforms said peaked video signal to a higher resolution video signal;

an upconverted edge detector which analyzes pixels in said higher resolution video signal that correspond to said peaked signal possible edge pixels to identify higher resolution video signal edge pixels;

an edge linker which connects said identified higher resolution video signal edge pixels;

a luminance transition improvement device which sharpens said connected higher resolution video signal edge pixels by replacing slow luminance transitions with steep luminance transitions; and

a display coupled to said receiver to output said higher resolution video signal.

24. A device for scaling resolution enhancement in a video signal, comprising:

a receiver that accepts the video signal at a first resolution;

a peaking system which increases a perceived contrast of said accepted video signal to create a peaked signal;

a textured signal edge detector which analyzes a first derivative of said peaked signal to identify peaked signal possible edge pixels;

an upconverter which transforms said peaked signal to a higher resolution video signal;

an upconverted edge detector which calculates a second derivative of said peaked signal to identify zero crossings and calculates a third derivative at said peaked signal zero crossing pixels to identify higher resolution video signal edge pixels;

an edge linker which connects said identified higher resolution video signal edge pixels;

a luminance transition improvement device which sharpens said connected higher resolution video signal edge pixels by dividing a higher resolution video signal edge pixel into multiple sub-regions, analyzing each subregion to determine a location, inside said pixel, where an edge is located, using said edge location to select a filter; and

applying said selected filter to said higher resolution edge pixel.; and

a display coupled to said receiver to output said higher resolution video signal.

25. A television receiver, comprising:

a tuner for receiving and tuning to a television signal at a first resolution;

a television signal processing circuit coupled to a display to deliver an enhanced resolution television signal, wherein generation of said enhanced resolution television signal further comprises;

applying a peaking function to said first resolution television signal to create a peaked signal;

analyzing said peaked signal to identify possible edge pixels in said first resolution television signal;

upconverting said peaked signal to a higher resolution television signal;

applying an edge detection function to said higher resolution television signal at pixels that correspond to said first resolution television signal possible edge pixels, thereby identifying higher resolution television signal edge pixels;

applying an edge linking function to said higher resolution television signal edge pixels;

applying a luminance transition improvement function to said higher resolution television signal edge pixels; and

outputting said higher resolution television signal.

26. A television receiver, comprising:

a tuner for receiving and tuning to a television signal at a first resolution;

a television signal processing circuit coupled to a display to deliver an enhanced resolution television signal, wherein generation of said enhanced resolution television signal further comprises;

applying a peaking function to said first resolution television signal to increase a perceived contrast of said television signal;

analyzing said television signal peaked signal to identify possible edge pixels;

upconverting said first resolution television signal to a higher resolution television signal;

applying an edge detection function to said higher resolution television signal to identify zero crossings and analyzing said zero crossing pixels to identify higher resolution television signal edge pixels;

applying an edge linking function to said identified higher resolution television signal edge pixels;

applying a luminance transition improvement function to said identified higher resolution television signal edge pixels by dividing a higher resolution video signal edge pixel into multiple sub-regions, analyzing each subregion to determine a location, inside said pixel, where an edge is located, using said edge location to select a filter; and

applying said selected filter to said higher resolution edge pixel; and

outputting said higher resolution television signal.