CREATING DETAILS IN AN IMAGE WITH ADAPTIVE FREQUENCY STRENGTH CONTROLLED TRANSFORM

• US 20140270566A1
• Filed: 03/14/2014
• Published: 09/18/2014
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Tao, Li, Kim, Yeong-Taeg
• Granted Patent: US 9,349,188 B2
• Time in Patent Office: Days
• US Class Current: 382/263
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/13 Edge detection

Abstract

Input image information is received. A processor device is used for determining enhancement information based on frequency characteristics and edge information of the input image information. The enhancement information is mixed with the input image information to generate an enhanced image.

Claims

1. A method comprising:

receiving input image information;

using a processor device for determining enhancement information based on frequency characteristics and edge information of the input image information; and

mixing the enhancement information with the input image information to generate an enhanced image.

2. The method of claim 1, wherein determining the enhancement information comprises:

processing an input image to obtain the input image information;

processing detected edges to obtain the edge information;

using the processor device for transforming the image information for obtaining frequency domain information;

using the edge information for determining a frequency component adjusting ratio;

determining frequency lifting for the frequency domain information using the frequency component adjusting ratio for creating particular frequency components;

performing an inverse transformation on an updated frequency distribution for creating one or more output image blocks; and

combining the one or more output image blocks to create image details, and mixing the image details with the input image.

3. The method of claim 2, further comprising:

inputting the input image;

dividing the input image into the input image information that comprises one or more input image blocks;

detecting edges in the input image; and

dividing the detected edges into the edge information that comprises one or more edge blocks.

4. The method of claim 3, wherein an input image block and an edge block each has a block size larger than a block size of an output image block.

5. The method of claim 2, wherein determining the frequency component adjusting ratio comprises:

estimating an edge strength value based on a location of a number of edges; and

computing the frequency component adjusting ratio based on edge adaptive thresholds as a function of the edge strength value.

6. The method of claim 5, wherein the frequency domain information comprises transform coefficients, and the particular frequency components are created based on:

performing frequency lifting on the transform coefficients for lifting one or more frequency components to a higher frequency location in a transform spectrum, wherein determined distances are adjusted using the frequency component adjusting ratio; and

converting the transform coefficients to modified transform coefficients.

7. The method of claim 5, wherein the frequency domain information comprises transform coefficients, and the particular frequency components are created based on:

shifting the transform coefficients position in a matrix;

using the shifted transform coefficients for determining distances for coefficients to a zero-frequency coefficient position using associated coefficient row and column indexes;

performing frequency lifting on the transform coefficients based on the determined distances for lifting one or more frequency components to a higher frequency location in a transform spectrum, wherein the determined distances are adjusted using the frequency component adjusting ratio; and

converting the transform coefficients to modified transform coefficients.

8. The method of claim 7, further comprising:

moving the zero-frequency coefficient position to a center of the matrix after said shifting, wherein the zero-frequency coefficient position is used as an origin of a two-dimensional coordinate, and matrix elements are used as points on the two-dimensional coordinate.

9. The method of claim 8, wherein determining distances for coefficients is based on a pre-determined transfer function, and transforming received input image information comprises using one of discrete Fourier transform processing and discrete cosine transform processing.

10. The method of claim 2, wherein an output image comprising the image details and the input image is displayed on an electronic device.

11. The method of claim 10, wherein the electronic device comprises one of a television device, a monitor device, a computing device and a mobile computing device.

12. The method of claim 1, wherein the processor device performs processing on one of a local television device, a set-top-box device, and a network server device.

13. A system comprising

an edge processing module that obtains edge information from an input image;

a detail creation module that uses a processor device for;

receiving input image information; and

determining enhancement information based on the edge information and frequency characteristics of the input image information; and

a mixer module that mixes the enhancement information with the input image.

14. The system of claim 13, wherein the edge processing module detects edges of the input image and processes the edges to obtain the edge information;

and

the detail creation module uses the processor device for;

transforming the received input image information for obtaining frequency domain information;

using the edge information for determining a frequency component adjusting ratio;

determining frequency lifting for the frequency domain information using the frequency component adjusting ratio for creating particular frequency components; and

performing an inverse transformation on an updated frequency distribution for creating one or more output image blocks; and

the system further comprises;

an output block construction module that combines the one or more output image blocks to create the enhancement information that comprises image details for a lifted image;

wherein the mixer module mixes the lifted image with the input image.

15. The system of claim 14, further comprising:

a receiving module that inputs the input image and divides the input image into the input image information that comprises one or more input image blocks;

wherein the edge processing module divides detected edges into the edge information that comprises one or more edge blocks.

16. The system of claim 15, wherein an input image block and an edge block each has a block size larger than a block size of an output image block.

17. The system of claim 14, wherein the frequency component adjusting ratio is determined by the detail creation module that further:

estimates an edge strength value based on a location of a number of edges; and

uses edge adaptive thresholds as a function of the edge strength value for determining the frequency component adjusting ratio.

18. The system of claim 16, wherein the frequency domain information comprises transform coefficients, and the particular frequency components are created by the detail creation module that further:

performs frequency lifting on the transform coefficients to lift one or more frequency components to a higher frequency location in a transform spectrum, wherein determined distances are adjusted using the frequency component adjusting ratio; and

converts the transform coefficients to modified transform coefficients.

19. The system of claim 17, wherein the frequency domain information comprises transform coefficients, and the particular frequency components are created by the detail creation module that further:

shifts the transform coefficients position in a matrix;

uses the shifted transform coefficients to determine distances for coefficients to a zero-frequency coefficient position using associated coefficient row and column indexes;

performs frequency lifting on the transform coefficients based on the determined distances to lift one or more frequency components to a higher frequency location in a transform spectrum, wherein the determined distances are adjusted using the frequency component adjusting ratio; and

converts the transform coefficients to modified transform coefficients.

20. The system of claim 19, wherein the detail creation module further moves the zero-frequency coefficient position to a center of the matrix after shifting the transform coefficients position in the matrix, wherein the zero-frequency coefficient position is used as an origin of a two-dimensional coordinate, and matrix elements are used as points on the two-dimensional coordinate.

21. The system of claim 20, wherein the distances coefficients are determined based on a pre-determined transfer function, and transforming received input image information comprises the detail creation module performing one of a discrete Fourier transform and a discrete cosine transform.

22. The system of claim 14, wherein an output image that comprises the image details and the input image is displayed on an electronic device that comprises one of a television device, a monitor device, a computing device and a mobile computing device.

23. The system of claim 13, wherein the detail creation module performs processing on one of a television device, a set-top-box device, and a network server device.

24. A non-transitory computer-readable medium having instructions which when executed on a computer perform a method comprising:

receiving input image information;

determining enhancement information based on frequency characteristics and edge information of the input image information; and

mixing the enhancement information with the input image information to generate an enhanced image.

25. The medium of claim 24, further comprising:

processing an input image to obtain the input image information;

processing detected edges of the input image to obtain the edge information;

transforming the input image information for obtaining frequency domain information;

using the edge information for determining a frequency component adjusting ratio;

determining frequency lifting for the frequency domain information using the frequency component adjusting ratio for creating particular frequency components;

performing an inverse transformation on an updated frequency distribution for creating one or more output image blocks; and

combining the one or more output image blocks to create the enhancement information that comprises image details, and mixing the image details with the input image.

26. The medium of claim 25, further comprising:

receiving the input image;

dividing the input image into the input image information that comprises one or more input image blocks;

detecting edges in the input image; and

dividing the detected edges into the edge information that comprises one or more edge blocks,

wherein an input image block and an edge block each has a block size larger than a block size of an output image block.

27. The medium of claim 25, wherein determining the frequency component adjusting ratio comprises:

estimating an edge strength value based on a location of a number of edges in the input image; and

computing the frequency component adjusting ratio based on edge adaptive thresholds as a function of the edge strength value.

28. The medium of claim 27, wherein the frequency domain information comprises transform coefficients, and the particular frequency components are created based on:

performing frequency lifting on the transform coefficients for lifting one or more frequency components to a higher frequency location in a transform spectrum, wherein determined distances are adjusted using the frequency component adjusting ratio; and

converting the transform coefficients to modified transform coefficients.

29. The medium of claim 27, wherein the frequency domain information comprises transform coefficients, and the particular frequency components are created based on:

shifting the transform coefficients position in a matrix;

using the shifted transform coefficients for determining distances for coefficients to a zero-frequency coefficient position using associated coefficient row and column indexes;

performing frequency lifting on the transform coefficients based on the determined distances for lifting one or more frequency components to a higher frequency location in a transform spectrum, wherein the determined distances are adjusted using the frequency component adjusting ratio; and

converting the transform coefficients to modified transform coefficients.

30. The medium of claim 29, further comprising:

moving the zero-frequency coefficient position to a center of the matrix after said shifting, wherein the zero-frequency coefficient position is used as an origin of a two-dimensional coordinate, and matrix elements are used as points on the two-dimensional coordinate.

31. The medium of claim 30, wherein determining distances for coefficients is based on a pre-determined transfer function, and transforming the received input image information comprises using one of discrete Fourier transform processing and discrete cosine transform processing.

32. The medium of claim 25, wherein an output image that comprises the image details and the input image is displayed on an electronic device, wherein the electronic device comprises one of a television device, a monitor device, a computing device and a mobile computing device.

33. The medium of claim 24, wherein the computer performs processing on one of a local television device, a set-top-box device, and a network server device.