System and method for use of images with recognition analysis

• US 7,542,610 B2
• Filed: 10/03/2006
• Issued: 06/02/2009
• Est. Priority Date: 05/09/2005
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Like.Com (Alphabet Inc.)
• Inventors: Shah, Munjal, Gokturk, Salih Burak, Khan, Azhar
• Primary Examiner(s): KIM, CHONG R
• Application Number:
  US11/543,758
• Publication Number:
  US 20070081744A1
• Time in Patent Office: 973 Days
• Field of Search: None
• US Class Current: 382/209
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 40/107 Static hand or arm
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching

Abstract

An index is provided that holds information about each image content item in a collection of items, For each image content item, a first information item identifying the image content item and its location on a network, and at least one of (i) a second information item identifying a signature value of an object in the image content, or (ii) identification of a recognized object in the image content.

Claims

1. A method for enabling selection of image content items, the method comprising:

using a processor to perform steps comprising;

performing image analysis on a collection of image content items to obtain information about each image content item;

wherein performing image analysis includes (i) identifying one or more objects in individual image content items, (ii) determining a category of each identified object, the category of each object being one of a plurality of possible object categories;

(iii) identifying a set of features that are specific to the determined category of the identified object in each image content item, and (iv) determining information based on the set of features for the determined category that characterizes the identified object, the information characterizing the object to be separately identifiable from at least some other identified objects in the determined category;

storing the information obtained from the image analysis in one or more data stores;

identifying one or more criteria determined from a text or image content provided in connection with either a user or programmatically identified input; and

performing a search operation to identify one or more image content items that satisfy the one or more criteria of the input using the stored information in the one or more data stores;

wherein at least some of the collection of image content items correspond to images of merchandise objects;

wherein performing the search operation includes selecting one or more image content items of merchandise objects for display with a document in response to said input specifying a merchandise object in the document;

wherein selecting one or more images of merchandise objects for display with the document includes selecting the one or more images based on a determination that the one or more images of merchandise objects are similar to the specified merchandise object;

providing a link with each of the selected one or more images of the merchandise objects, wherein the link is selectable to enable a user to purchase the merchandise objects from a network site operated by the merchant.

2. The method of claim 1, wherein performing image analysis is performed programmatically and prior to identifying one or more criteria.

3. The method of claim 1, wherein storing the information includes storing the information in web based index that also associates information about each image content item with a location of the image content item.

4. The method of claim 1, wherein identifying one or more criteria includes programmatically analyzing a web document that is identified in the input to determine the one or more criteria from one or both of a text portion or image portion of the web document;

and wherein the method further comprises presenting the identified one or more image content items to the user.

5. The method of claim 1, identifying one or more criteria includes:

detecting an image content item that is selected or being viewed by the user;

identifying text data associated with the image content item; and

forming at least a portion of the one or more criteria based on the text data associated with the image content item.

6. The method of claim 1, wherein identifying one or more criteria includes:

detecting an input image content item from a web document;

identifying a given object of the input image content item, including a corresponding category of the given object, the corresponding category being associated with a specific set of features;

determining a signature value of the object in the input image content item based on a characterization of one or more features in the specific set of features for the corresponding category of the identified object;

forming at least one of the criteria from the signature value of the input image content item.

7. The method of claim 6, wherein

performing image analysis on individual image content items in the collection by determining information based on the set of features for the determined category that characterizes the identified object includes determining a signature value for the identified object, the signature value being based at least in part on a quantitative characterization of at least some features in the set of features for the determined category of the identified object;

and

performing a search operation to identify one or more image content items that includes using the signature of the one or more criteria to perform a similarity comparison with a signature value of other objects identified from image content items in the collection.

8. The method of claim 1, wherein at least some of the collection of image content items correspond to advertisement media, and wherein performing a search operation to identify one or more image content items includes selecting one or more advertisement media from the collection for display with a web document that is presented to the user.

9. The method of claim 8, wherein selecting one or more advertisement media is performed in response to the user making a selection of a particular content on the web document.

10. The method of claim 8, wherein selecting one or more advertisement media is performed in response to detecting the text or image content provided in the web document.

11. The method of claim 1, wherein performing a search operation includes configuring the search operation based on a category of the one or more criteria.

12. The method of claim 11, wherein configuring the search operation includes weighting how the one or more criteria compare to one or more corresponding features in the set of features for the category of the search.

13. The method of claim 1, wherein:

performing image analysis includes identifying one or more apparel items in each image content item,

determining the category of each object identified in each image content item includes identifying a category of the apparel item.

14. The method of claim 13, wherein identifying the set of features that are specific to the determined category includes identifying a buckle or zipper for one or more apparel items.

15. A computer system that operates to enable selection of image content items, the computer system comprising:

an image analysis sub-system that is configured to analyze individual image content items that form part of a collection of image content items, the image analysis sub-system being configured to (i) identify one or more objects from individual image content items, (ii) determine one or more of a plurality of categories for each identified object, (iii) associate a set of features that are specific to each of the plurality of categories, so that each category is assigned a set of features that are different from the set of features for at least some of the other categories;

wherein for each identified object, the image analysis sub-system is configured to determine a signature value for the object by characterizing individual features in the set of features that are specific to the category of the identified object;

a search component that is configured to (i) identify, from an input that specifies or includes an image, a given object and a corresponding category of the given object, (ii) determine a criteria from the image of the input, wherein the criteria is based at least in part on a specific set of features that are associated with the corresponding category of the object identified from the object in the image of the input, and (iii) use the signature value determined for each identified object in the collection to select one or more image content items from the collection that satisfies the criteria;

wherein at least some of the collection of image content items correspond to images of merchandise objects;

wherein the search component is configured to;

select one or more image content items of merchandise objects for display with a document in response to said input specifying a merchandise object in the document, the one or more images being selected based on a determination that the one or more images of merchandise objects are similar to the specified merchandise object;

provide a link with each of the selected one or more images of the merchandise objects, wherein the link is selectable to enable a user to purchase the merchandise objects from a network site operated by the merchant.

16. The computer system of claim 15, further comprising:

a data store that stores information that identifies or corresponds to each image content item in the collection, and for each image content item, the signature value of one or more objects identified from each image content item in the collection.

17. The computer system of claim 15, wherein the image analysis sub-system is configured to identify anyone of a plurality of apparel items as the object in individual image content items that form at least a portion of the collection.

18. The computer system of claim 17, wherein the set of features for an item of apparel includes a shape of the apparel item at a specific region of the object.

19. The computer system of claim 17, wherein the set of features for an item of apparel includes a style of the apparel item.

20. The computer system of claim 17, wherein the set of features for an item of apparel includes at least one feature that indicates presence of a buckle, zipper or shoe heel.

21. The computer system of claim 15, wherein the input further includes a user selection to view a web page or resource.

22. The computer system of claim 21, wherein the one or more components include a component that supplement the web page or resource being viewed by the user with the selected one or more image content items.

23. The computer system of claim 15, wherein the image analysis sub-system is configured to determine the signature value for each object of the individual image content items by quantitatively characterizing individual features in the set of features for the determined category of that object.

24. The computer system of claim 15, wherein the search component operates to determine a signature value of the given object in the image of the input based on a characterization of one or more features in the specific set of features for the corresponding category of the given object.

25. The computer system of claim 24, wherein the search component operates to form at least one of the criteria from the signature value of the image of the input.