Image Enhancement and Repair Using Sample Data from Other Images

• US 20130279809A1
• Filed: 04/24/2012
• Published: 10/24/2013
• Est. Priority Date: 04/24/2012
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Dellinger, Richard R., Sabatelli, Alessandro F.
• Granted Patent: US 9,626,747 B2
• Time in Patent Office: Days
• US Class Current: 382/195
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/166 using acquisition arrangements
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image

Abstract

A image manipulation technique allows a user to correct an image using samples obtained from other images. These samples may be obtained from one or more other images in a library of images. Matching techniques may identify an image that best matches the image to be corrected, or may aggregate or average multiple images that are identified as containing an area corresponding to the area to be corrected. Identification of the image or images to use as the source of the samples may be automatic or manual. The images may be from a library of images under the control of the user or from a library of images maintained by another person or service provider. Application of the samples to correct the image may be manually or automatically directed.

Claims

1. An image modification method, comprising:

selecting a first region of a first image;

identifying automatically, from a database of images, one or more second images, wherein each of the one or more second images has a region corresponding to the first region; and

modifying automatically the first region based, at least in part, on the corresponding region from at least one of the one or more second images.

2. The method of claim 1, wherein the act of identifying automatically one or more second images comprises:

ascertaining a first location identifier of the first image; and

identifying each of the one or more second images based, at least in part, on a match between the first location identifier and a location identifier of each of the one or more second images.

3. The method of claim 1, wherein the act of selecting a first region of a first image further comprises analyzing automatically the first region to identify an object.

4. The method of claim 3, wherein the act of identifying automatically one or more second images comprises selecting one or more second images having represented therein the object.

5. The method of claim 1, wherein the act identifying automatically, from a database of images, one or more second images comprises:

searching the database of images for images having a region corresponding the first region,

wherein the act of searching the database is constrained by one or more predetermined criteria.

6. The method of claim 5, wherein the act of searching the database is constrained by geodata information associated with the first image.

7. The method of claim 5, wherein the act of searching the database is constrained by lighting information associated with the first image.

8. The method of claim 5, wherein the act of searching the database is constrained by a temporal window, wherein the first image was captured during the temporal window.

9. The method of claim 5, wherein the act of searching the database is constrained by label information associated with the first region.

10. The method of claim 1, wherein the act of modifying automatically the first region comprises:

combining automatically each of the corresponding regions from the one or more second images to generate a combined region; and

modifying automatically the first region based, at least in part, on the combined region.

11. The method of claim 10, wherein the act of combining automatically each of the corresponding regions from the one or more second images to generate a combined region comprises:

averaging the corresponding regions from the one or more second images to generate the combined region.

12. The method of claim 1, wherein the act of selecting a first region comprises:

detecting and selecting a facial region of the first image.

13. A non-transitory program storage device, readable by a programmable control device and comprising instructions stored thereon to cause the programmable control device to:

receive an indication that identifies a first region of a first image;

identify, from a database of images, one or more second images, wherein each of the one or more second images has a region corresponding to the first region; and

modify the first region based, at least in part, on the corresponding region from at least one of the one or more second images.

14. The non-transitory program storage device of claim 13, wherein the instructions to cause the programmable control device to identify, from a database of images, one or more second images, comprise instructions to cause the programmable control device to:

ascertain a first location identifier of the first image; and

identify each of the one or more second images based, at least in part, on a match between the first location identifier and a location identifier of each of the one or more second images.

15. The non-transitory program storage device of claim 13, wherein the instructions to cause the programmable control device to receive an indication that identifies a first region of a first image, further comprise instructions to cause the programmable control device to analyze the first region to identify an object.

16. The non-transitory program storage device of claim 15, wherein the instructions to cause the programmable control device to identify one or more second images comprise instructions to cause the programmable control device to select one or more second images having represented therein the object.

17. The non-transitory program storage device of claim 16, wherein the instructions to cause the programmable control device to modify the first region comprise instructions to cause the programmable control device to:

combine each of the corresponding regions from the one or more second images to generate a combined region; and

modify the first region based, at least in part, on the combined region.

18. An programmable device, comprising:

a programmable control device;

a memory coupled to the programmable control device; and

software stored in the memory, the software comprising instructions causing the programmable control device to;

receive an indication that identifies a first region of a first image;

identify automatically one or more second images from a collection of images, wherein each of the one or more second images has a region corresponding to the first region; and

modify the first region automatically based, at least in part, on the corresponding region from at least one of the one or more second images.

19. The programmable device of claim 18, wherein the instructions that cause the programmable control device to identify automatically one or more second images from a collection of images comprises instructions that cause the programmable control device to:

determine a location corresponding to the first image; and

automatically select one or more second images from the collection of images corresponding to the location.

20. The programmable device of claim 18, wherein the instructions that cause the programmable control device to receive an indication that identifies a first region of a first image comprise instructions that cause the programmable control device to:

identify a first object represented in the first image; and

automatically select the first region as a region of the first image including a representation of the first object.

21. The programmable device of claim 20, wherein the instructions that cause the programmable control device to identify automatically one or more images from a collection of images comprise instructions that cause the programmable control device to:

automatically select one or more image from the collection of images containing a representation of at least a part of the first object.

22. The programmable device of claim 18 wherein the instructions that cause the programmable control device to modify the first region comprise instructions that cause the programmable control device to:

automatically combine the corresponding regions from a plurality of the one or more second images to generate a combined region; and

automatically modify the first region based, at least in part, on the combined region.