Creating details in an image with adaptive frequency strength controlled transform

• US 9,349,188 B2
• Filed: 03/14/2014
• Issued: 05/24/2016
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Tao, Li, Kim, Yeong-Taeg
• Primary Examiner(s): Alavi, Amir
• Application Number:
  US14/214,081
• Publication Number:
  US 20140270566A1
• Time in Patent Office: 802 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/13 Edge detection

Abstract

Input image information is received. A processor device is used for determining enhancement information based on frequency characteristics and edge information of the input image information. The enhancement information is mixed with the input image information to generate an enhanced image.

Claims

1. A method comprising:

receiving input image information representative of a graphical image for display;

determining, by a processor device, enhancement information based on frequency characteristics and graphical edge information of the input image information; and

mixing the enhancement information with the input image information to generate an enhanced image for display.

2. The method of claim 1, wherein determining the enhancement information comprises:

processing an input image to obtain the input image information;

processing detected graphical edges to obtain the graphical edge information;

using the processor device for transforming the image information for obtaining frequency domain information;

using the graphical edge information for determining a frequency component adjusting ratio;

determining frequency lifting for the frequency domain information using the frequency component adjusting ratio for creating particular frequency components;

performing an inverse transformation on an updated frequency distribution for creating one or more output image blocks;

combining the one or more output image blocks to create image details; and

mixing the enhancement information with the input image information comprises mixing the image details with the input image.

3. The method of claim 2, further comprising:

inputting the input image;

dividing the input image into the input image information that comprises one or more input image blocks;

detecting graphical edges in the input image; and

dividing the detected graphical edges into the graphical edge information that comprises one or more graphical edge blocks.

4. The method of claim 3, wherein an input image block and a graphical edge block each has a block size larger than a block size of an output image block.

5. The method of claim 2, wherein determining the frequency component adjusting ratio comprises:

estimating a graphical edge strength value based on a location of a number of graphical edges; and

computing the frequency component adjusting ratio based on edge adaptive thresholds as a function of the graphical edge strength value.

6. The method of claim 5, wherein the frequency domain information comprises transform coefficients, and the particular frequency components are created based on:

performing frequency lifting on the transform coefficients for lifting one or more frequency components to a higher frequency location in a transform spectrum, wherein determined distances are adjusted using the frequency component adjusting ratio; and

converting the transform coefficients to modified transform coefficients.

7. The method of claim 5, wherein the frequency domain information comprises transform coefficients, and the particular frequency components are created based on:

shifting the transform coefficients position in a matrix;

using the shifted transform coefficients for determining distances for coefficients to a zero-frequency coefficient position using associated coefficient row and column indexes;

performing frequency lifting on the transform coefficients based on the determined distances for lifting one or more frequency components to a higher frequency location in a transform spectrum, wherein the determined distances are adjusted using the frequency component adjusting ratio; and

converting the transform coefficients to modified transform coefficients.

8. The method of claim 7, further comprising:

moving the zero-frequency coefficient position to a center of the matrix after said shifting, wherein the zero-frequency coefficient position is used as an origin of a two-dimensional coordinate, and matrix elements are used as points on the two-dimensional coordinate.

9. The method of claim 8, wherein determining distances for coefficients is based on a pre-determined transfer function, and transforming received input image information comprises using one of discrete Fourier transform processing and discrete cosine transform processing.

10. The method of claim 2, further comprising:

displaying an output image comprising the image details and the input image on an electronic device, wherein the electronic device comprises a television device, a monitor device, a computing device or a mobile computing device.

11. The method of claim 1, wherein the processor device performs processing on a local television device, a set-top-box device, or a network server device.

12. The method of claim 1, wherein determining enhancement information is further based on using frequency lifting to lift one or more input image frequency components to a higher frequency location in a transform spectrum.

13. A system comprising

a memory storing instructions;

one or more processors configured to execute the instructions to perform operations including graphical edge processing to obtain graphical edge information from an input image and detail creation processing to;

receive input image information representative of a graphical image for display; and

determined enhancement information based on the graphical edge information and frequency characteristics of the input image information; and

a mixer configured to mix the enhancement information with the input image.

14. The system of claim 13, wherein the one or more processors are configured to execute the instructions to perform further operations including graphical edge processing to detect graphical edges of the input image and process the graphical edges to obtain the graphical edge information, and to:

transform the received input image information for obtaining frequency domain information;

use the graphical edge information for determining a frequency component adjusting ratio;

determine frequency lifting for the frequency domain information using the frequency component adjusting ratio for creating particular frequency components; and

perform an inverse transformation on an updated frequency distribution to create one or more output image blocks; and

combine the one or more output image blocks to create the enhancement information that comprises image details for a lifted image; and

the mixer mixes the lifted image with the input image.

15. The system of claim 14, wherein the one or more processors are configured to execute the instructions to perform further operations including:

processing to input the input image and divide the input image into the input image information that comprises one or more input image blocks; and

to divide detected graphical edges into the graphical edge information that comprises one or more graphical edge blocks.

16. The system of claim 15, wherein an input image block and a graphical edge block each has a block size larger than a block size of an output image block.

17. The system of claim 16, wherein the frequency domain information comprises transform coefficients, and the one or more processors are configured to execute the instructions to perform further operations including processing to:

perform frequency lifting on the transform coefficients to lift one or more frequency components to a higher frequency location in a transform spectrum, determined distances are adjusted using the frequency component adjusting ratio; and

convert the transform coefficients to modified transform coefficients.

18. The system of claim 14, wherein the one or more processors are configured to execute the instructions to perform further operations including detail creation processing to:

estimate a graphical edge strength value based on a location of a number of graphical edges; and

use edge adaptive thresholds as a function of the graphical edge strength value to determine the frequency component adjusting ratio.

19. The system of claim 18, wherein the frequency domain information comprises transform coefficients, and the one or more processors are configured to execute the instructions to perform further operations including processing to:

shift the transform coefficients position in a matrix;

use the shifted transform coefficients to determine distances for coefficients to a zero-frequency coefficient position using associated coefficient row and column indexes;

perform frequency lifting on the transform coefficients based on the determined distances to lift one or more frequency components to a higher frequency location in a transform spectrum, the determined distances are adjusted using the frequency component adjusting ratio; and

convert the transform coefficients to modified transform coefficients.

20. The system of claim 19, wherein one or more processors are configured to execute the instructions to perform further operations including processing to move the zero-frequency coefficient position to a center of the matrix after the transform coefficients position is shifted in the matrix, the zero-frequency coefficient position is used as an origin of a two-dimensional coordinate, and matrix elements are used as points on the two-dimensional coordinate.

21. The system of claim 20, wherein the distances coefficients are determined based on a pre-determined transfer function, and the one or more processors are configured to execute the instructions to perform further operations including processing to transform received input image information using one of a discrete Fourier transform and a discrete cosine transform.

22. The system of claim 14, wherein an output image that comprises the image details and the input image is displayed on an electronic device that comprises a television device, a monitor device, a computing device or a mobile computing device.

23. The system of claim 13, wherein the one or more processors are configured to execute the instructions to perform further operations including processing on a television device, a set-top-box device, or a network server device.

24. A non-transitory computer-readable medium having instructions which when executed on a computer perform a method comprising:

receiving input image information representative of a graphical image for display;

determining enhancement information based on frequency characteristics and graphical edge information of the input image information; and

mixing the enhancement information with the input image information to generate an enhanced image for display.

25. The non-transitory computer-readable medium of claim 24, wherein the program when executed by the computer performs the method further comprising:

processing an input image to obtain the input image information;

processing detected graphical edges of the input image to obtain the graphical edge information;

transforming the input image information for obtaining frequency domain information;

using the graphical edge information for determining a frequency component adjusting ratio;

determining frequency lifting for the frequency domain information using the frequency component adjusting ratio for creating particular frequency components;

performing an inverse transformation on an updated frequency distribution for creating one or more output image blocks;

combining the one or more output image blocks to create the enhancement information that comprises image details; and

the mixing of the enhancement information with the input image information comprises mixing the image details with the input image.

26. The non-transitory computer-readable medium of claim 25, wherein the program when executed by the computer performs the method further comprising:

receiving the input image;

dividing the input image into the input image information that comprises one or more input image blocks;

detecting graphical edges in the input image; and

dividing the detected graphical edges into the graphical edge information that comprises one or more graphical edge blocks,

wherein an input image block and a graphical edge block each has a block size larger than a block size of an output image block.

27. The non-transitory computer-readable medium of claim 25, wherein determining the frequency component adjusting ratio comprises:

estimating a graphical edge strength value based on a location of a number of edges in the input image; and

computing the frequency component adjusting ratio based on edge adaptive thresholds as a function of the graphical edge strength value.

28. The non-transitory computer-readable medium of claim 27, wherein the frequency domain information comprises transform coefficients, and the particular frequency components are created based on:

performing frequency lifting on the transform coefficients for lifting one or more frequency components to a higher frequency location in a transform spectrum, wherein determined distances are adjusted using the frequency component adjusting ratio; and

converting the transform coefficients to modified transform coefficients.

29. The non-transitory computer-readable medium of claim 27, wherein the frequency domain information comprises transform coefficients, and the particular frequency components are created based on:

shifting the transform coefficients position in a matrix;

using the shifted transform coefficients for determining distances for coefficients to a zero-frequency coefficient position using associated coefficient row and column indexes;

performing frequency lifting on the transform coefficients based on the determined distances for lifting one or more frequency components to a higher frequency location in a transform spectrum, wherein the determined distances are adjusted using the frequency component adjusting ratio; and

converting the transform coefficients to modified transform coefficients.

30. The non-transitory computer-readable medium of claim 29, wherein the program when executed by the computer performs the method further comprising:

moving the zero-frequency coefficient position to a center of the matrix after said shifting, wherein the zero-frequency coefficient position is used as an origin of a two-dimensional coordinate, and matrix elements are used as points on the two-dimensional coordinate.

31. The non-transitory computer-readable medium of claim 30, wherein determining distances for coefficients is based on a pre-determined transfer function, and transforming the received input image information comprises using one of discrete Fourier transform processing and discrete cosine transform processing.

32. The non-transitory computer-readable medium of claim 25, wherein an output image that comprises the image details and the input image is displayed on an electronic device, and the electronic device comprises a television device, a monitor device, a computing device or a mobile computing device.

33. The non-transitory computer-readable medium of claim 24, wherein the computer performs processing on one of a local television device, a set-top-box device, and a network server device.