IMAGE PROCESSING APPARATUS AND IMAGE PROCESSING METHOD

• US 20160093023A1
• Filed: 09/25/2015
• Published: 03/31/2016
• Est. Priority Date: 09/26/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: PRASAD, BINDIGAN HARIPRASANNA PAWAN, PEDDIGARI, VENKAT RAMANA, RAMANUJAM, GAYATHRI, JAIN, ANURAG MITHALAL, ROA, PHANISH HANAGAL SRINIVASA, REDDY, LOKA SUDHARSAN, PATIDAR, DIVYA, BHAMIDIPATI, SREEVATSA DWARAKA
• Granted Patent: US 10,229,478 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

An image processing method, including preparing a first image; designating at least one region of interest (ROI) in the first image, generating restoration data; and generating a second image using the restoration data, the second image corresponding to the ROI, wherein a resolution of the second image is higher than a resolution of a region corresponding to the ROI in the first image.

Claims

1. An image processing method comprising:

preparing a first image;

designating at least one region of interest (ROI) in the first image;

generating restoration data; and

generating a second image using the restoration data, the second image corresponding to the ROI,

wherein a resolution of the second image is higher than a resolution of a region corresponding to the ROI in the first image.

2. The image processing method of claim 1, wherein the first image is a captured image of an object, and

wherein generating the restoration data further comprises:

generating restoration data associated with the region corresponding to the ROI in the first image; and

storing the first image and the restoration data.

3. The image processing method of claim 1, wherein the first image is a preview image generated using light reflected from an object,

wherein preparing the first image further comprises displaying the preview image on a display, and

wherein generating the restoration data further comprises:

capturing a third image of the object;

generating the restoration data based on a region corresponding to the ROI in the third image; and

storing the third image and the restoration data.

4. The image processing method of claim 3, wherein generating the restoration data further comprises:

capturing a plurality of images of the object; and

generating a super resolution image of the ROI using at least two images of the plurality of images.

5. The image processing method of claim 4, wherein generating the restoration data further comprises:

generating comparison data indicating a difference between a region corresponding to the ROI in the third image and a region corresponding to the ROI in the super resolution image.

6. The image processing method of claim 4, wherein generating the restoration data further comprises:

selecting at least one selected resolution, the selected resolution being greater than a resolution of a region corresponding to the ROI in the third image, and being less than or equal to a resolution of a region corresponding to the ROI in the super resolution image;

upscaling the region corresponding to the ROI in the third image to the selected resolution and downscaling the region corresponding to the ROI in the super resolution image to the selected resolution; and

generating comparison data associated with the selected resolution by comparing the region corresponding to the ROI in the third image and the region corresponding to the ROI in the super resolution image.

7. The image processing method of claim 3, wherein designating the at least one ROI further comprises:

dividing the preview image into a plurality of regions;

determining whether an object of interest (OOI) is included in the plurality of regions; and

designating at least one of the plurality of regions as the ROI according to the determination result.

8. The image processing method of claim 3, wherein designating the at least one ROI further comprises:

detecting a position of an object of interest (OOI) in the preview image by analyzing the preview image;

detecting a region corresponding to the OOI by performing image segmentation using the detected position; and

designating the detected region as the ROI.

9. The image processing method of claim 8, wherein detecting the position of the OOI comprises at least one from among:

detecting a position of a high frequency component in the preview image;

detecting a position of a face by performing face recognition on the preview image;

detecting a position of a text by performing optical character recognition (OCR) on the preview image; and

detecting a position of a focus in the preview image.

10. The image processing method of claim 3, wherein designating the at least one ROI further comprises:

receiving a drag input from a user;

detecting a first region corresponding to the drag input in the preview image;

receiving a tap input from a user;

detecting a position corresponding to the tap input in the preview image;

detecting a second region by performing image segmentation using the detected position; and

designating the first region and the second region as the ROI.

11. The image processing method of claim 3, wherein designating the at least one ROI further comprises:

detecting a position of an object of interest (OOI) in the preview image by analyzing the preview image;

detecting a region corresponding to the OOI by performing image segmentation by using the detected position;

displaying the detected region on the preview image;

receiving a tap input from a user at the detected region;

designating the detected region as a selected region according to the tap input;

receiving a drag input from a user;

detecting a third region corresponding to the drag input in the preview image; and

designating the selected region and the third region as the ROI.

12. The image processing method of claim 3, further comprising:

displaying the stored third image and the region corresponding to the ROI in the third image;

receiving an input from a user;

determining an enlargement magnification based on a resolution of the display using the received input;

generating an enlarged image of the third image using the enlargement magnification and the restoration data; and

displaying the enlarged image.

13. The image processing method of claim 12, wherein generating the enlarged image further comprises:

when the enlargement magnification is greater than 1 and less than a square root of a quotient obtained by dividing a resolution of the third image by a resolution of the display, generating the enlarged image by downscaling a portion of the third image to be displayed on the display; and

when the enlargement magnification is substantially equal to the square root of the quotient obtained by dividing the resolution of the third image by the resolution of the display, generating the enlarged image by cropping a portion of the third image to be displayed on the display.

14. The image processing method of claim 12, wherein generating the enlarged image further comprises:

when the enlargement magnification is greater than a square root of a first quotient obtained by dividing a resolution of the third image by a resolution of the display, and less than a product of the square root of the first quotient and a square root of a second quotient obtained by dividing a resolution of the region corresponding to the ROI in the super resolution image by a resolution of the region corresponding to the ROI in the third image, generating a partial image by upscaling a portion of the third image to be displayed on the display; and

generating the enlarged image by synthesizing the restoration data with the partial image.

15. The image processing method of claim 12, wherein the generating of the enlarged image comprises:

when the enlargement magnification is greater than a square root of a quotient obtained by dividing a resolution of the third image by a resolution of the display and less than a product of the square root of the quotient obtained by dividing the resolution of the third image by the resolution of the display and a square root of a quotient obtained by dividing a resolution of the region corresponding to the ROI in the super resolution image by a resolution of the region corresponding to the ROI in the third image, generating a first partial image by upscaling a first portion of the third image, the first portion corresponding to the ROI;

generating a second partial image by upscaling a second portion of the third image, the second portion not corresponding to the ROI;

generating a super resolution partial image by synthesizing the restoration data with the first partial image; and

generating the enlarged image by combining the super resolution partial image with the second partial image.

16. The image processing method of claim 12, wherein the generating of the enlarged image comprises:

when the enlargement magnification is greater than a product of a square root of a quotient obtained by dividing a resolution of the third image by a resolution of the display and a square root of a quotient obtained by dividing a resolution of the region corresponding to the ROI in the super resolution image by a resolution of the region corresponding to the ROI in the third image, generating a partial image by upscaling a portion of the third image;

generating a synthesized image by synthesizing the restoration data with the partial image; and

generating the enlarged image by upscaling the synthesized image.

17. A non-transitory computer-readable recording medium storing a program for executing the method of claim 1 on a computer.

18. An image processing method comprising:

displaying a stored image, the stored image including a region corresponding to a region of interest (ROI);

receiving an input from a user;

determining an enlargement magnification using the received input;

generating an enlarged image corresponding to the enlargement magnification using restoration data associated with the region corresponding to the ROI in the image; and

displaying the enlarged image,

wherein the enlarged image corresponds to the ROI, and

wherein a resolution of the enlarged image is higher than a resolution of the region corresponding to the ROI in the stored image.

19. An image processing apparatus comprising:

a light receiver configured to receive light reflected from an object; and

a controller configured to prepare a first image using the received light, to designate at least one region of interest in the first image, to generate restoration data, and to generate a second image using the restoration data, the second image corresponding to the ROI,

wherein a resolution of the second image is higher than a resolution of a region corresponding to the ROI in the first image.

20. The image processing apparatus of claim 19, further comprising a storage,

wherein the first image is a captured image of the object,

wherein the controller is configured to generate restoration data associated with the region corresponding to the ROI in the first image, and

wherein the storage is configured to store the first image and the restoration data.

21. The image processing apparatus of claim 19, further comprising:

an image signal processor configured to generate a preview image using the received light;

a display configured to display the preview image; and

a storage,

wherein the preview image is the first image,

wherein the controller is further configured to control the light receiver and the image signal processor to capture a third image of the object, and to generate the restoration data based on a region corresponding to the ROI in the third image, and

wherein the storage is configured to store the third image and the restoration data.

22. The image processing apparatus of claim 21, wherein the restoration data is a super resolution image of the ROI.

23. The image processing apparatus of claim 21, wherein the restoration data is comparison data indicating a difference between the region corresponding to the ROI in the third image and a region corresponding to the ROI in a super resolution image corresponding to the third image.

24. The image processing apparatus of claim 21, wherein the controller is further configured to select at least one selected resolution, the selected resolution being greater than a resolution of the region corresponding to the ROI in the third image, and less than or equal to a resolution of a region corresponding to the ROI in a super resolution image corresponding to the third image, to upscale the region corresponding to the ROI in the third image to the selected resolution, to downscale the region corresponding to the ROI in the super resolution image to the selected resolution, and to generate the restoration data associated with the selected resolution by comparing the region corresponding to the ROI in the third image and the region corresponding to the ROI in the super resolution image.

25. The image processing apparatus of claim 21, wherein the controller is configured to divide the preview image into a plurality of regions, to determine whether an object of interest (OOI) is included in the plurality of regions, and to designate at least one region of the plurality of regions as the ROI according to the determination result.

26. The image processing apparatus of claim 21, wherein the controller is further configured to detect a position of an object of interest (OOI) in the preview image by analyzing the preview image, to detect a region corresponding to the OOI by performing image segmentation by using the detected position, and to designate the detected region as the ROI.

27. The image processing apparatus of claim 21, further comprising an input interface configured to receive a drag input,

wherein the controller is further configured to detect a first region corresponding to the drag input in the preview image,

the input interface is further configured receive a tap input, and

the controller is further configured to detect a position corresponding to the tap input in the preview image, to detect a second region by performing image segmentation by using the detected position, and to designate the first region and the second region as the ROI.

28. The image processing apparatus of claim 21, further comprising an input interface configured to receive an input,

wherein the controller is further configured to determine an enlargement magnification based on a resolution of the display by using the received input, and to generate an enlarged image of the third image by using the enlargement magnification and the restoration data, and

the display is further configured to display the generated enlarged image.

29. An image processing method, comprising:

capturing a plurality of burst images using a camera;

selecting a preview image from among the plurality of burst images;

designating a region of interest in the preview image;

generating detail information corresponding to the region of interest, the detail information being generated from the plurality of burst images;

generating a detail image based on the detail information;

displaying the detail image on a display in response to a user input associated with the region of interest.

30. The method according to claim 29, wherein designating the region of interest in the preview image further comprises receiving a selection input from the user and designating the region of interest based on the selection input.

31. The method according to claim 29, wherein designating the region of interest in the preview image further comprises detecting an object of interest in the preview image and designating the region of interest based on the object of interest.

32. The method according to claim 29, wherein generating detail information corresponding to the region of interest further comprises selecting a comparison image from among the plurality of burst images, selecting a comparison region in the comparison image, the comparison region corresponding to the region of interest, performing a comparison between the comparison region and the region of interest, and generating the detail information based on the comparison.

33. An image processing method comprising:

preparing a first image;

designating at least one region of interest (ROI) in the first image;

generating restoration data corresponding to the ROI using the first image;

generating a compressed image by encoding the first image;

storing the compressed image and the restoration data.

34. The method according to claim 33, further comprising:

generating a restored image by restoring a region corresponding to the ROI in the compressed image using the restoration data, the restored image corresponding to the ROI; and

displaying the restored image,

wherein a resolution of the restored image is higher than a resolution of the region corresponding to the ROI in the compressed image.