Image processing device and image processing method for image correction

• US 10,089,770 B2
• Filed: 04/10/2017
• Issued: 10/02/2018
• Est. Priority Date: 01/31/2014
• Status: Active Grant

• Current Assignee: Morpho Incorporated
• Original Assignee: Morpho Incorporated
• Inventors: Kobayashi, Michihiro
• Primary Examiner(s): Hung, Yubin
• Application Number:
  US15/482,818
• Publication Number:
  US 20170213375A1
• Time in Patent Office: 540 Days
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/269 using gradient-based methods

Abstract

The present disclosure is to generate a high-quality image by correcting a predetermined correction target image based on a plurality of input images. In an image processing device 3, an image correcting section 160 detects a user tap gesture on a touch panel 250. When the position of the tap gesture is within foreground candidate areas detected by a foreground candidate area detecting section 140, the image correcting section 160 corrects a base image set by a base image setting section 120 in the areas corresponding to the foreground candidate areas.

Claims

1. An image processing device, comprising:

a display unit;

a detecting unit configured to detect a correction candidate area in a correction target image based on a plurality of input images;

a reliability determining unit configured to determine a correction reliability of the correction candidate area;

a display controlling unit configured to control the display unit to display the correction target image, the correction candidate area and the determined correction reliability; and

an image correcting unit configured to correct the correction candidate area based on a user operation,

wherein the reliability determining unit determines the correction reliability in a one-to-one correspondence with the correction candidate area, and

the display controlling unit controls the display unit to display whether to correct the correction candidate area in a user-recognizable manner on a basis of the determined correction reliability.

2. The image processing device according to claim 1,

wherein the display controlling unit perform one of the following display control:

the display control to control the display unit to display the correction candidate area on the correction target image, in a manner that reflects the determined correction reliability to indicate that correcting the correction candidate area is more appropriate as the correction reliability is higher, and

the display control to display an image in which at least one of a contour of the correction candidate area or a center of gravity of the correction candidate area on the correction target image, in a manner that reflects the determined correction reliability to indicate that correcting the correction candidate area is more appropriate as the correction reliability is higher.

3. The image processing device according to claim 2,

wherein the display controlling unit further perform the display control so that a correction target is contained in the correction candidate area in the user-recognizable manner.

4. The image processing device according to claim 1,

wherein the detecting unit detects a plurality of correction candidate areas in the correction target image,

the reliability determining unit determines the correction reliability of each of the correction candidate areas in the one-to-one correspondence with each of the plurality of correction candidate areas, and

the display controlling unit controls the display unit to display the correction target image, the plurality of correction candidate areas and the determined respective correction reliability,

the display controlling unit controls the display unit to display whether to correct each of the correction candidate areas in the user-recognizable manner on the basis of the determined correction reliability for each of the plurality of correction candidate areas.

5. The image processing device according to claim 4,

wherein the display controlling unit controls the display unit to display the plurality of correction candidate areas on the correction target image, in a manner that reflects the determined respective correction reliability for each of the plurality of correction candidate areas to indicate that correcting the correction candidate area is more appropriate as the correction reliability is higher.

6. The image processing device according to claim 4,

wherein the image correcting unit sets a correction candidate area from the plurality of correction candidate areas specified by a first user operation and to correct the set correction candidate area based on a second user operation.

7. The image processing device according to claim 6, further comprising:

a notifying unit configured to notify a predetermined caution when the determined correction reliability of the correction candidate area specified by the first user operation does not satisfy a predetermined condition.

8. The image processing device according to claim 1,

wherein the display controlling unit controls the display unit to inhibit displaying of the correction candidate area when the determined correction reliability of the correction candidate area does not satisfy a predetermined condition.

9. The image processing device according to claim 1,

wherein the image correcting unit inhibits correcting of the correction candidate area when the determined correction reliability of the correction candidate area does not satisfy a predetermined condition, regardless of the user operation.

10. The image processing device according to claim 1,

wherein the image correcting unit corrects the correction candidate area of the correction target image by using image correction processing that is selected based on the user operation or the determined correction reliability of the correction candidate area.

11. The image processing device according to claim 10,

wherein the image correcting unit performs the image correction processing that replaces an image data of the correction candidate area of the correction target image with an image data of an area corresponding to the correction candidate area of a correction image that is at least one of the plurality of input images.

12. The image processing device according to claim 10,

wherein the image correcting unit performs the image correction processing that extracts an image data that satisfies a predetermined relationship with image data of the correction candidate area from a correction image that is at least one of the plurality of input images, and pastes the extracted image data to the correction candidate area of the correction target image.

13. The image processing device according to claim 1,

wherein the display controlling unit controls the display unit to display an expected corrected image of the correction candidate area on the correction target image, in a manner that reflects the determined correction reliability.

14. The image processing device according to claim 13,

wherein the image correcting unit corrects the correction candidate area based on the user operation while the expected corrected image is displayed.

15. The image processing device according to claim 13,

wherein the display controlling unit controls the display unit to display the expected corrected image on the correction target image with a predetermined opacity.

16. The image processing device according to claim 1, further comprising:

a preferential correction area determining unit, configured to determine a preferential correction area specified by the user operation, the preferential correction area being an area in the correction target image that is preferentially corrected,

wherein the display controlling unit controls the display unit to display the preferential correction area on the correction target in a user-recognizable manner, and the image correcting unit preferentially corrects the preferential correction area of the correction target image.

17. The image processing device according to claim 1,

wherein the image correcting unit inhibits correcting of a correction candidate area in an inhibited correction area specified by the user operation, regardless of the determined correction reliability of the correction candidate area.

18. The image processing device according to claim 1,

wherein the image correcting unit forcibly corrects a correction candidate area in a forced correction area specified by the user operation, regardless of the determined correction reliability of the correction candidate area.

19. An image processing method, comprising:

detecting a correction candidate area in a correction target image based on a plurality of input images;

determining a correction reliability of the correction candidate area;

displaying the correction target image, the correction candidate area and the determined correction reliability; and

correcting the correction candidate area based on a user operation,

wherein the correction reliability is determined in a one-to-one correspondence with the correction candidate area, and

whether to correct the correction candidate area in a user-recognizable manner is displayed on a basis of the determined correction reliability.

20. A non-transitory computer readable recording medium storing a program to make a computer execute the steps of:

a detecting step of detecting a correction candidate area in a correction target image based on a plurality of input images;

a reliability determining step of determining a correction reliability of the correction candidate area;

a display step of displaying the correction target image, the correction candidate area and the determined correction reliability; and

an image correcting step of correcting the correction candidate area based on a user operation,

wherein the reliability determining step determines the correction reliability in a one-to-one correspondence with the correction candidate area, and

the display step displays whether to correct the correction candidate area in a user-recognizable manner on a basis of the determined correction reliability.