Object oriented image editing

• US 10,198,820 B1
• Filed: 01/31/2017
• Issued: 02/05/2019
• Est. Priority Date: 02/19/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: McNerney, Peter Joseph, Miller, Nolan Andrew
• Primary Examiner(s): Shah, Utpal D
• Application Number:
  US15/420,652
• Time in Patent Office: 735 Days
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 2207/20104 Interactive definition of region of interest [ROI]
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions

Abstract

Implementations generally relate to object based image editing. In some implementations, a method includes segmenting an image into object data by identifying one or more object classifications in the image and storing at least one locator for one or more regions of the image corresponding to each instance of the object classification. The method further includes receiving a selection of a representative portion of the segmented image from a user, and matching the representative portion with the object data to determine at least one matched object classification associated with the representative portion. The method further includes presenting the user with one or more of the matched object classifications for the user to instruct one or more edit operations to be applied to at least one object represented by the matched object classification.

Claims

1. A computer implemented method to edit an image, the method comprising:

segmenting the image into regions depicting objects;

identifying two or more different classifications of respective objects of the objects in the image;

storing locator information for each of the regions of the image corresponding to each instance of the respective objects of the two or more different classifications and a representation of each of the respective two or more different classifications;

providing to a user device, the stored representation of each of the respective two or more different classifications;

receiving a user selection indicating a select classification of the two or more different classifications;

receiving a user request to apply one or more edit operations to at least one object in the image corresponding to the select classification; and

applying the one or more edit operations to the at least one object in the image corresponding to the select classification by locating the at least one object with the location information for the at least one object.

2. The computer implemented method of claim 1, further including identifying a generic classification of a region of the image.

3. The method of claim 1, wherein the two or more different classifications include at least a broad category and a subcategory of increase specificity.

4. The method of claim 1, wherein at least one of the two or more classifications includes more than one instance of an object and the user selection includes a select number of the instances of the select classification.

5. The method of claim 1, wherein the representation is a semantic term of the respective two or more different classifications and wherein the user selection is by the user selecting the provided semantic term of the select classification.

6. The method of claim 1, wherein the user selection is a region of the image including a portion of the object corresponding to the select classification and further including determining the select classification by matching the region with location information to determine at least one matched classification of the object in the region.

7. The method of claim 6, wherein the user request is to apply the one or more edit operation to more than one instance of the at least one object.

8. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

segmenting an image into regions depicting objects;

identifying two or more different classifications of respective objects of the objects in the image;

storing locator information for each of the regions of the image corresponding to each instance of the respective objects of the two or more different classifications and a semantic term of each of the respective two or more different classifications;

providing to a user device, the stored semantic term of each of the respective two or more different classifications;

receiving a user selection indicating a select classification of the two or more different classifications;

providing at least one recommendation for one or more edit operations, wherein the at least one recommendation is based on the select classification;

receiving a user request to apply at least one of the one or more edit operations to at least one object in the image corresponding to the select classification; and

applying the one or more edit operations to the at least one object in the image corresponding to the select classification by locating the at least one object with the location information for the at least one object.

9. The computer-readable medium of claim 8, wherein the operations further comprise identifying a generic classification of a region of the image.

10. The computer-readable medium of claim 8, wherein the two or more different classifications include at least a broad category and a subcategory of increase specificity.

11. The computer-readable medium of claim 8, wherein at least one of the two or more classifications includes more than one instance of an object and the user selection includes a select number of the instances of the select classification.

12. The computer-readable medium of claim 8, wherein the user selection is a region of the image including a portion of the object corresponding to the select classification and further including determining the select classification by matching the region with location information to determine at least one matched classification of the object in the region.

13. The computer-readable medium of claim 12, wherein the user request is to apply the one or more edit operation to more than one instance of the at least one object.

14. A system comprising:

one or more processors; and

one or more computer-readable media having instructions stored thereon that, when executed by the one or more processors, cause performance of operations comprising;

segmenting an image into regions depicting objects;

identifying two or more different classifications of respective objects of the objects in the image;

storing locator information for each of the regions of the image corresponding to each instance of the respective objects of the two or more different classifications and a representation of each of the respective two or more different classifications;

providing to a user device, the stored representation of each of the respective two or more different classifications;

receiving a user selection indicating a select classification of the two or more different classifications;

receiving a user request to apply one or more edit operations to at least one object in the image corresponding to the select classification; and

applying the one or more edit operations to the at least one object in the image corresponding to the select classification by locating the at least one object with the location information for the at least one object.

15. The system of claim 14, wherein the operations further comprise identifying a generic classification of a region of the image.

16. The system of claim 14, wherein the two or more different classifications include at least a broad category and a subcategory of increase specificity.

17. The system of claim 14, wherein at least one of the two or more classifications includes more than one instance of an object and the user selection includes a select number of the instances of the select classification.

18. The system of claim 14, wherein the user selection is a region of the image including a portion of the object corresponding to the select classification and further including determining the select classification by matching the region with location information to determine at least one matched classification of the object in the region.

19. The system of claim 18, wherein the user request is to apply the one or more edit operation to more than one instance of the at least one object.

20. The system of claim 18, wherein the representation is a semantic term of the respective two or more different classifications and wherein the user selection is by the user selecting the provided semantic term of the select.