Object-Level Image Editing

• US 20090096808A1
• Filed: 02/08/2007
• Published: 04/16/2009
• Est. Priority Date: 02/28/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Winn, John, Rother, Carsten
• Granted Patent: US 9,135,732 B2
• Time in Patent Office: Days
• US Class Current: 345/594
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text

Abstract

Systems and methods for editing digital images using information about objects in those images are described. For example, the information about objects comprises depth ordering information and/or information about the class each object is a member of. Examples of classes include sky, building, aeroplane, grass and person. This object-level information is used to provide new and/or improved editing functions such as cut and paste, filling-in image regions using tiles or patchworks, digital tapestry, alpha matte generation, super resolution, auto cropping, auto colour balance, object selection, depth of field manipulation, and object replacement. In addition improvements to user interfaces for image editing systems are described which use object-level information.

Claims

1. A method of editing a source digital image comprising the steps of:

(i) accessing automatically generated object-level information about the source digital image, that information comprising segmentation of the image into one or more objects and, for each of the objects, details of a class that the object is a member of; and

(ii) automatically processing the source digital image on the basis of the object-level information to form an output digital image.

2. A method as claimed in claim 1 wherein the object-level information comprises any of:

information about depth ordering of at least some of the objects;

object attributes; and

information about the position of the object parts.

3. A method as claimed in claim 1 which further comprises combining at least part of the source digital image with one or more reference digital images to form the output digital image and wherein the method comprises accessing automatically generated object-level information about the reference digital images, that information comprising segmentation of the reference images into one or more objects and, for each of the objects, details of a class that the object is a member of.

4. A method as claimed in claim 1 which is carried out for a plurality of source digital images to create a single output image and wherein the output image is formed automatically on the basis of object-level information about each of the source images.

5. A method as claimed in claim 4 which comprises selecting parts of the source images for inclusion in the output image on the basis of the object-level information.

6. A method as claimed in claim 4 which comprises selecting locations in the output image to position parts of the source images on the basis of the object-level information.

7. A method as claimed in claim 1 wherein the source image is of a low resolution compared with the output image.

8. A method as claimed in claim 7 wherein the step (ii) of processing the source image comprises accessing a database of images, having higher resolution than the source image, and selecting images from that database to use in forming the output image, on the basis of the object-level information.

9. A method as claimed in claim 1 wherein the output image is cropped from the source image and wherein the step (ii) of processing the source image comprises automatically cropping the source image on the basis of the object level information.

10. A method as claimed in claim 9 which further comprises automatically cropping the source image on the basis of a template, rule or criteria selected from a plurality of specified templates rules of criteria.

11. A method as claimed in claim 1 wherein the step (ii) of processing the source image comprises adjusting the colour balance of that source image on the basis of the object-level information.

12. A method as claimed in claim 1 wherein said step (ii) of processing the source image comprises receiving only a single user input for selecting an object in the source image and presenting an indication to a user via a user interface as to which object in the source image is selected as a result of that single user input, the selection having been made by the user with a single action.

13. A method as claimed in claim 12 which further comprises arranging the user interface such that editing options presented to a user via the interface are selected on the basis of the object-level information associated with the selected object.

14. A method as claimed in claim 12 which further comprises modifying any of the colour, focus or depth order of the selected object.

15. A method as claimed in claim 1 wherein said step (ii) of processing the source image comprises replacing an object or part of an object in the source image with an image of a selected object or object part from a database and wherein selection of the object or object part is made on the basis of the object-level information.

16. An apparatus for editing a source digital image comprising:

(i) an interface arranged to access automatically generated object-level information about the source digital image, that information comprising segmentation of the image into one or more objects and, for each of the objects, details of a class that the object is a member of; and

(ii) a processor arranged to automatically process the source digital image on the basis of the object-level information to form an output digital image.

17. An apparatus as claimed in claim 16 wherein the object-level information comprises any of:

information about depth ordering of at least some of the objects;

object attributes; and

information about the position of the object parts.

18. An apparatus as claimed in claim 16 wherein the processor is arranged to combine at least part of the source digital image with one or more reference digital images to form the output digital image and wherein the interface is arranged to access automatically generated object-level information about the reference digital images, that information comprising segmentation of the reference images into one or more objects and, for each of the objects, details of a class that the object is a member of.

19. A computer readable medium containing a computer program comprising computer program code means adapted to perform the method comprising:

(i) accessing automatically generated object-level information about the source digital image, that information comprising segmentation of the image into one or more objects and, for each of the objects, details of a class that the object is a member of; and

(ii) automatically processing the source digital image on the basis of the object-level information to form an output digital image.

21. The computer readable medium of claim 19, wherein the object-level information comprises any of:

information about depth ordering of at least some of the objects;

object attributes; and

information about the position of the object parts.

20. (canceled)