Image processing apparatus and image processing method

• US 10,229,478 B2
• Filed: 09/25/2015
• Issued: 03/12/2019
• Est. Priority Date: 09/26/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Prasad, Bindigan Hariprasanna Pawan, Peddigari, Venkat Ramana, Ramanujam, Gayathri, Jain, Anurag Mithalal, Rao, Phanish Hanagal Srinivasa, Reddy, Loka Sudharsan, Patidar, Divya, Bhamidipati, Sreevatsa Dwaraka
• Primary Examiner(s): Wu, Jingge
• Application Number:
  US14/866,049
• Publication Number:
  US 20160093023A1
• Time in Patent Office: 1,264 Days
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

An image processing method, including preparing a first image; designating at least one region of interest (ROI) in the first image, generating restoration data; and generating a second image using the restoration data, the second image corresponding to the ROI, wherein a resolution of the second image is higher than a resolution of a region corresponding to the ROI in the first image.

Claims

1. An image processing method comprising:

storing an image and restoration data, wherein the restoration data indicates a difference between at least one region of the image and at least one region of a super resolution image corresponding to the at least one region of the image;

displaying the stored image;

selecting a region from among the at least one region of the image in response to an input;

determining a magnification for the selected region;

obtaining target restoration data corresponding to the selected region from the restoration data based on the determined magnification;

generating an enlarged image according to the determined magnification based on the stored image and the target restoration data corresponding to the selected region; and

displaying the enlarged image,

wherein the enlarged image is generated by scaling up the image and processing the scaled image with the target restoration data corresponding to the selected region,

wherein the enlarged image has at least one resolution which is greater than a resolution of the stored image, and

wherein the restoration data includes a first restoration data set having a first resolution, and a second restoration data set having a second resolution different from the first resolution.

3. The image processing method of claim 1,

wherein the determining the magnification comprises

determining the magnification for the selected region based on a resolution of a display and the input.

4. The image processing method of claim 1, wherein the generating the enlarged image further comprises:

if the magnification is greater than 1 and less than a square root of a quotient obtained by dividing the resolution of the stored image by a resolution of a display, generating the enlarged image by downscaling a portion of a third image to be displayed; and

if the magnification is substantially equal to the square root of the quotient obtained by dividing the resolution of the stored image by the resolution of the display, generating the enlarged image by cropping a portion of the stored image to be displayed.

5. The image processing method of claim 1, wherein the generating the enlarged image further comprises:

if the magnification is greater than a square root of a first quotient obtained by dividing the resolution of the stored image by a resolution of a display, and less than a product of the square root of the first quotient and a square root of a second quotient obtained by dividing a resolution of a region of the super resolution image corresponding to the selected region by a resolution of the selected region of the stored image, generating a partial image by upscaling a portion of the image to be displayed; and

generating the enlarged image by synthesizing the target restoration data with the partial image.

6. The image processing method of claim 1, wherein the generating the enlarged image comprises:

if the magnification is greater than a square root of a quotient obtained by dividing a resolution of the stored image by a resolution of a display and less than a product of the square root of the quotient obtained by dividing the resolution of the stored image by the resolution of the display and a square root of a quotient obtained by dividing a resolution of a region of the super resolution image corresponding to the selected region by a resolution of the selected region of the stored image, generating a first partial image by upscaling a first portion of the image, the first portion corresponding to one of the at least one region of the image;

generating a second partial image by upscaling a second portion of the stored image, the second portion not corresponding to the one of the at least one region of the image;

generating a super resolution partial image by synthesizing the target restoration data with the first partial image; and

generating the enlarged image by combining the super resolution partial image with the second partial image.

7. The image processing method of claim 1, wherein the generating the enlarged image comprises:

if the magnification is greater than a product of a square root of a quotient obtained by dividing a resolution of the stored image by a resolution of a display and a square root of a quotient obtained by dividing a resolution of a region of the super resolution image corresponding to the selected region by a resolution of the selected region of the stored image, generating a partial image by upscaling a portion of the image;

generating a synthesized image by synthesizing the target restoration data with the partial image; and

generating the enlarged image by upscaling the synthesized image.

8. The image processing method of claim 1, wherein the restoration data is obtained by comparing the at least one region of the image and the at least one region of the super resolution image corresponding to the at least one region of the image,

wherein the super resolution image is obtained based on a plurality of images.

9. The image processing method of claim 1,

wherein the displaying the stored image further comprises displaying at least one indicator, wherein each indicator of the at least one indicator indicates one of the at least one region of the image, and

wherein the selected region corresponds to one of the at least one indicator selected based on the input.

10. The image processing method of claim 1, wherein the restoration data is obtained by scaling at least one of a region of the image and a region of the super resolution image corresponding to the region of the image, and comparing the region of the image and the region of the super resolution image after the scaling.

17. The image processing method of claim 1, wherein:

the restoration data includes a plurality of partial restoration data sets, each of the partial restoration data sets has at least one partial restoration data,

the at least one partial restoration data corresponds to the at least one region of the image, and

resolutions of the plurality of partial restoration data sets are different from each other.

18. The image processing method of claim 1, wherein:

the restoration data includes a plurality of partial restoration data,

each of the plurality of partial restoration data corresponds to each of the at least one region of the image, and

resolutions of at least a portion of the plurality of partial restoration data are different from each other.

19. The image processing method of claim 1, wherein:

the restoration data includes a plurality of partial restoration data for the at least one region of the image,

each of at least one region among the at least one region of the image corresponds multiple partial restoration data, and

resolutions of the multiple partial restoration data are different from each other.

20. The image processing method of claim 1, wherein the obtaining the target restoration data comprises obtaining the target restoration data from the restoration data according to a resolution of the enlarged image corresponding to the determined magnification.

21. The image processing method of claim 20, wherein the obtaining the target restoration data comprises determining the target restoration data among multiple partial restoration data corresponding to the selected region, according to the resolution of the enlarged image corresponding to the determined magnification, the multiple partial restoration data is included in the restoration data.

22. The image processing method of claim 20, wherein the obtaining the target restoration data comprises obtaining the target restoration data by processing a portion of the restoration data corresponding to the selected region according to the resolution of the enlarged image corresponding to the determined magnification.

23. The image processing method of claim 1, wherein the enlarged image is generated by:

generating a first partial image by upscaling a first portion of the image, the first portion corresponding to one of the at least one region of the image,

generating a second partial image by upscaling a second portion of the stored image, the second portion not corresponding to the one of the at least one region of the image,

generating a super resolution partial image by synthesizing the target restoration data with the first partial image, and

generating the enlarged image by combining the super resolution partial image with the second partial image.

2. An image processing apparatus comprising:

a storage configured to store an image and restoration data, wherein the restoration data indicates a difference between at least one region of the image and at least one region of a super resolution image corresponding to the at least one region of the image;

a display configured to display the stored image; and

at least one processor configured to select a region from among the at least one region of the image in response to an input, determine a magnification for the selected region, obtain target restoration data corresponding to the selected region from the restoration data based on the determined magnification, and generate an enlarged image according to the determined magnification based on the stored image and the target restoration data corresponding to the selected region,

wherein the display is further configured to display the enlarged image,

wherein the enlarged image is generated by scaling up the image and processing the scaled image with the target restoration data corresponding to the selected region,

wherein the enlarged image has at least one resolution which is greater than a resolution of the stored image, and

wherein the restoration data includes a first restoration data set having a first resolution, and a second restoration data set o having a second resolution different from the first resolution.

11. The image processing apparatus of claim 2, wherein the restoration data is obtained by scaling at least one of a region of the image and a region of the super resolution image corresponding to the region of the image, and comparing the region of the image and the region of the super resolution image after the scaling.

12. The image processing apparatus of claim 2, wherein the at least one processor is configured to determine the magnification for the selected region based on a resolution of a display and the input.

13. The image processing apparatus of claim 2, wherein the at least one processor is configured to generate the enlarged image by:

generating a partial image by upscaling a portion of the image to be displayed, and

generating the enlarged image by synthesizing the target restoration data with the partial image.

14. The image processing apparatus of claim 2, wherein the at least one processor is configured to generate the enlarged image by:

generating a first partial image by upscaling a first portion of the image, the first portion corresponding to one of the at least one region of the image,

generating a second partial image by upscaling a second portion of the stored image, the second portion not corresponding to the one of the at least one region of the image,

generating a super resolution partial image by synthesizing the target restoration data with the first partial image, and

generating the enlarged image by combining the super resolution partial image with the second partial image.

15. The image processing apparatus of claim 2, wherein the restoration data is obtained by comparing the at least one region of the image and the at least one region of the super resolution image corresponding to the at least one region of the image,

wherein the super resolution image is obtained based on a plurality of images.

16. The image processing apparatus of claim 2, wherein the display is configured to display at least one indicator on the image, wherein each indicator of the at least one indicator indicates one of the at least one region of the image, and

wherein the selected region corresponds to one of the at least one indicator selected based on the input.

24. The image processing apparatus of claim 2, wherein:

the restoration data includes a plurality of partial restoration data sets,

each of the partial restoration data sets has at least one partial restoration data,

the at least one partial restoration data corresponds to the at least one region of the image, and

resolutions of the plurality of partial restoration data sets are different from each other.

25. The image processing apparatus of claim 2, wherein:

the restoration data includes a plurality of partial restoration data,

each of the plurality of partial restoration data corresponds to each of the at least one region of the image, and

resolutions of at least portion of the plurality of partial restoration data are different from each other.

26. The image processing apparatus of claim 2, wherein:

the restoration data includes a plurality of partial restoration data for the at least one region of the image,

each of at least one region among the at least one region of the image corresponds multiple partial restoration data, and

wherein resolutions of the multiple partial restoration data are different from each other.

27. The image processing apparatus of claim 2, wherein the at least one processor is configured to obtain the target restoration data from the restoration data according to a resolution of the enlarged image corresponding to the determined magnification.

28. The image processing apparatus of claim 27, wherein the at least one processor is configured to determine the target restoration data among multiple partial restoration data corresponding to the selected region, according to the resolution of the enlarged image corresponding to the determined magnification, and the multiple partial restoration data is included in the restoration data.

29. The image processing apparatus of claim 27, wherein the at least one processor is configured to obtain the target restoration data by processing a portion of the restoration data corresponding to the selected region according to the resolution of the enlarged image corresponding to the determined magnification.