Creating details in an image with frequency lifting

• US 9,305,332 B2
• Filed: 03/14/2014
• Issued: 04/05/2016
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Tao, Li, Kim, Yeong-Taeg
• Primary Examiner(s): Alavi, Amir
• Application Number:
  US14/214,056
• Publication Number:
  US 20140270564A1
• Time in Patent Office: 753 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: H04N 19/625 using discrete cosine transform [DCT]

Abstract

Input image information is received. Enhancement information is determined based on frequency characteristics of the input image information. The enhancement information is mixed with the input image information to generate an enhanced image.

Claims

1. A method comprising:

receiving input image information;

determining, by a processor device, enhancement information based on;

transforming the received input image information to obtain frequency domain information;

performing an inverse transformation on an updated frequency distribution to create one or more output image blocks; and

combining the one or more output image blocks to create the enhancement information; and

mixing the enhancement information with the input image information to generate an enhanced image.

2. The method of claim 1, wherein mixing the enhancement information with the input image information comprises:

mixing the image details with the input image.

3. The method of claim 2, further comprising:

inputting the input image; and

dividing the input image into the input image information that comprises one or more input image blocks.

4. The method of claim 3, wherein an input image block has a block size larger than a block size of an output image block.

5. The method of claim 2, further comprising:

determining, by the processor device, a frequency distribution for the frequency domain information to create particular frequency components;

wherein the frequency domain information comprises transform coefficients, and the particular frequency components are created based on;

performing frequency lifting on the transform coefficients for processing one or more frequency components to a higher frequency location in a transform spectrum; and

converting the transform coefficients to modified transform coefficients.

6. The method of claim 5, wherein transforming received input image information comprises using one of discrete Fourier transform processing and discrete cosine transform processing.

7. The method of claim 2, wherein the frequency domain information comprises transform coefficients, and the particular frequency components are created based on:

shifting the transform coefficients position in a matrix;

using the shifted transform coefficients to determine distances for coefficients to a zero-frequency coefficient position using associated coefficient row and column indexes;

performing frequency lifting on the transform coefficients based on the determined distances to lift one or more frequency components to a higher frequency location in a transform spectrum; and

converting the transform coefficients to modified transform coefficients.

8. The method of claim 7, further comprising:

moving the zero-frequency coefficient position to a center of the matrix after said shifting, wherein the zero-frequency coefficient position is used as an origin of a two-dimensional coordinate, and matrix elements are used as points on the two-dimensional coordinate.

9. The method of claim 8, wherein determining distances for the coefficients are based on a pre-determined transfer function.

10. The method of claim 2, wherein an output image comprising the image details and the input image is displayed on an electronic device.

11. The method of claim 10, wherein the electronic device comprises a television device, a monitor device, a computing device or a mobile computing device.

12. The method of claim 2, wherein the processor device performs processing on one of a local television device, a set-top-box device, and a network server device.

13. An image enhancement device configured to generate an enhanced image by a process according to the method of claim 1.

14. A system comprising

a memory device configured to store input image information;

and

a processor device configured to;

receive the input image information from the memory device; and

determine enhancement information by being configured to;

transform the received input image information to obtain frequency domain information;

perform an inverse transformation on an updated frequency distribution to create one or more output image blocks; and

combine the one or more output image blocks to create the enhancement information; and

a mixer configured to mix the enhancement information with the input image information to generate an enhanced image.

15. The system of claim 14, wherein the the processor device is further configured to:

combine the one or more output image blocks to create the enhancement information that comprises image details; and

the mixer is configured to mix the image details with the input image.

16. The system of claim 15, wherein the processor device is further configured to

input the input image and to divide the input image into the input image information that comprises one or more input image blocks.

17. The system of claim 16, wherein an input image block has a block size larger than a block size of an output image block.

18. The system of claim 15, wherein the frequency domain information comprises transform coefficients, and the processor device is configured to:

determine a frequency distribution for the frequency domain information to create particular frequency components, perform frequency lifting on the transform coefficients to shift one or more frequency components to a higher frequency location in a transform spectrum, and

convert the transform coefficients to modified transform coefficients.

19. The system of claim 15, wherein the frequency domain information comprises transform coefficients, and the particular frequency components are created by the processor device that is further configured to:

shift the transform coefficients position in a matrix;

use the shifted transform coefficients to determine distances for coefficients to a zero-frequency coefficient position using associated coefficient row and column indexes;

perform frequency lifting on the transform coefficients based on the determined distances to lift one or more frequency components to a higher frequency location in a transform spectrum; and

convert the transform coefficients to modified transform coefficients.

20. The system of claim 19, wherein the processor device is further configured to move the zero-frequency coefficient position to a center of the matrix after shifting the transform coefficients position in the matrix, the zero-frequency coefficient position is used as an origin of a two-dimensional coordinate, and matrix elements are used as points on the two-dimensional coordinate.

21. The system of claim 20, wherein the distances for the coefficients are determined, by the processor device, based on a pre-determined transfer function.

22. The system of claim 21, wherein the processor device is configured to transform received input image information based on one of a discrete Fourier transform and a discrete cosine transform.

23. The system of claim 15, wherein an output image that comprises the image details and the input image is displayed on an electronic device that comprises a television device, a monitor device, a computing device or a mobile computing device.

24. The system of claim 15, wherein the processor device is configured to perform processing on a television device, a set-top-box device, or a network server device.

25. A non-transitory computer-readable medium having instructions which when executed on a computer perform a method comprising:

receiving, by the computer, input image information;

determining, by one or more processors, enhancement information based on;

transforming the received input image information to obtain frequency domain information;

performing an inverse transformation on an updated frequency distribution to create one or more output image blocks; and

combining the one or more output image blocks to create the enhancement information; and

mixing the enhancement information with the input image information to generate an enhanced image.

26. The non-transitory computer-readable medium of claim 25, wherein mixing the enhancement information with the input image information comprises:

mixing the image details with the input image.

27. The non-transitory computer-readable medium of claim 26, wherein the computer performs the method further comprising:

receiving the input image; and

dividing the input image into the input image information that comprises one or more input image blocks, and an input image block has a block size larger than a block size of an output image block.

28. The non-transitory computer-readable medium of claim 26, wherein the computer performs the method further comprising:

determining, by the one or more processors, a frequency distribution for the frequency domain information to create particular frequency components, the frequency domain information comprises transform coefficients, and the particular frequency components are created based on;

performing, by the one or more processors, frequency lifting on the transform coefficients to lift one or more frequency components to a higher frequency location in a transform spectrum; and

converting the transform coefficients to modified transform coefficients.

29. The non-transitory computer-readable medium of claim 26, wherein the frequency domain information comprises transform coefficients, and the particular frequency components are created based on:

shifting, by the one or more processors, the transform coefficients position in a matrix;

using, by the one or more processors, the shifted transform coefficients to determine distances for coefficients to a zero-frequency coefficient position using associated coefficient row and column indexes;

performing, by the one or more processors, frequency lifting on the transform coefficients based on the determined distances for lifting one or more frequency components to a higher frequency location in a transform spectrum; and

converting, by the one or more processors, the transform coefficients to modified transform coefficients.

30. The non-transitory computer-readable medium of claim 29, wherein the computer performs the method further comprising:

moving, by the one or more processors, the zero-frequency coefficient position to a center of the matrix after said shifting, wherein the zero-frequency coefficient position is used as an origin of a two-dimensional coordinate, and matrix elements are used as points on the two-dimensional coordinate.

31. The non-transitory computer-readable medium of claim 30, wherein determining distances for coefficients are based on a pre-determined transfer function.

32. The non-transitory computer-readable medium of claim 31, wherein transforming received input image information comprises the one or more processors performing discrete Fourier transform processing or discrete cosine transform processing.

33. The non-transitory computer-readable medium of claim 26, wherein an output image comprising the image details and the input image is displayed on an electronic device, and the electronic device comprises a television device, a monitor device, a computing device or a mobile computing device.

34. The non-transitory computer-readable medium of claim 26, wherein the computer performs processing on a local television device, a set-top-box device, or a network server device.