System and method for enabling image recognition and searching of images

• US 7,657,100 B2
• Filed: 11/07/2007
• Issued: 02/02/2010
• Est. Priority Date: 05/09/2005
• Status: Expired due to Fees

• Current Assignee: Google LLC (Alphabet Inc.), Palm, Inc. (TCL Technology Group Corp.)
• Original Assignee: Like.Com (Alphabet Inc.)
• Inventors: Dalal, Navneet, Shah, Munjal, Torresani, Lorenzo, Gokturk, Salih Burak, Vu, Diem, Anguelov, Dragomir, Vanhoucke, Vincent, Khan, Azhar, Yang, Danny, Lin, Xiaofan, Sumengen, Baris
• Primary Examiner(s): Kim; Charles
• Application Number:
  US11/936,705
• Publication Number:
  US 20080152231A1
• Time in Patent Office: 818 Days
• Field of Search: None
• US Class Current: 382/209
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/5862 using texture
• CPC Class Codes: G06Q 30/08 Auctions
• CPC Class Codes: G06V 10/40 Extraction of image or video features
• CPC Class Codes: Y10S 707/915 Image

Abstract

Embodiments programmatically analyze each of a plurality of images in order to determine one or more visual characteristics about an item shown in each of the plurality of images. Data is stored corresponding to the one or more visual characteristics. An interface in is provided for which a user is able to specify one or more search criteria. In response to receiving the one or more search criteria, a search operation is performed to identify one or more items that have a visual characteristic that satisfies at least some of the one or more search criteria.

Claims

1. A method for enabling searching of focus items in images, the method comprising:

using one or more processors to perform steps comprising;

enabling search operations to be performed to determine images in a collection that depict objects having portions that are visually similar and not identical to a focus item that is included as part of an item of an input image, wherein the enabling search operations includes;

programmatically analyzing each of the plurality of images in order to determine a signature of an item shown in each of the plurality of images;

storing data corresponding to the signature of each item shown in the plurality of images;

providing an interface in which a user is able to specify the focus item that is included as part of the item of the input image;

determining a signature of at least a portion of the item of the input image that includes the focus item;

determining search criteria that is based at least on the signature of at least the portion of the item of the input image that includes the focus item;

in response to determining the search criteria, performing a search operation to identify a result that comprises one or more items that have a signature that satisfies at least some of the one or more search criteria so that one or more of the identified items includes portions that are visually similar and not identical to the focus item included in the search criteria;

wherein performing the search operation includes performing an initial search operation on the collection using a shortened or truncated version of the signature of the input image in order to identify an initial result, then performing, on the initial result, one or more subsequent search operations that each use a longer or more complete version of the signature;

wherein programmatically analyzing each of the plurality of images in order to determine the signature of the item includes;

(i) programmatically determining a category of the item, (ii) based on the determined category, programmatically determining the one or more focus items within each image of the plurality of images, and (iii) determining the signature based, in part, on the focus item.

2. The method of claim 1, further comprising displaying a search result to the user that includes the identified one or more items.

3. The method of claim 2, wherein providing an interface includes enabling the user to specify an image from a previous search result as a search criterion.

4. The method of claim 1, further comprising storing the information that is determined from programmatically analyzing each of the plurality of images as a quantitative value.

5. The method of claim 4, wherein the quantitative value corresponds to one or more signatures that are descriptive or identifying of an individual item.

6. The method of claim 5, further comprising indexing the one or more signatures for each of the plurality of images.

7. The method of claim 1, wherein storing data corresponding to the signatures includes storing text terms that are descriptive of the signatures, the text terms being determined from programmatically analyzing the individual images.

8. The method of claim 1, wherein programmatically analyzing each of the plurality of images in order to determine information about an item further includes identifying a quantitative value from analyzing individual images in the plurality of images, and convening the quantitative value into one or more text terms that are descriptive of a global or local feature of the individual images.

9. The method of claim 1, further comprising procuring at least some of the plurality of images by crawling one or more network sites.

10. The method of claim 1, wherein performing a search operation includes presenting one or more of the plurality of items in prominence based at least in pan on a determination that the one or more of the plurality of items better satisfy the threshold than other items of the plurality of items.

11. The method of claim 10, wherein presenting the one or more of the plurality of items in prominence is based on the determination that the one or more of the plurality of items more resemble an appearance of the focus item.

12. The method of claim 11, wherein presenting the one or more of the plurality of items in prominence includes ranking multiple items based on determining a degree of resemblance between each of the multiple items and the search criteria.

13. The method of claim 11, further comprising enabling the user to specify one or more parameters that correspond to a color, a shape, or a pattern, and wherein identifying a plurality of item that have the signature that satisfies a threshold includes defining the threshold based on a combination of the signature of the image of the item and the specified one or more parameters.

14. The method of claim 10, wherein providing an interface includes enabling the user to select a portion of an image that shows a corresponding portion of an object as input.

15. The method of claim 14, further comprising comparing image data of the one or more criteria, as determined from the corresponding portion of the object, to image data from the portion of the objects shown in the plurality of images, in order to determine whether the image data of individual objects in the plurality of images satisfies the threshold.

16. A system for enabling searching of images, the system comprising:

an image analysis module that analyzes individual images in a collection of images in order to determine a signature of an item shown in each of the individual images;

one or more memory mediums that store data corresponding to (i) each analyzed image, and (ii) signatures of each item shown in each analyzed image;

a user interface configured to be operable by a user in order to enable the user to provide an input that corresponds to a focus item with an item of the an input image;

a search module configured to (i) determine or identify a signature of at least a portion of the item of the input image that includes the focus item;

(ii) determine search criteria that is based at least on the signature of at least the portion of the item of the input image that includes the focus item;

(iii) perform an initial search operation on the collection using a shortened or truncated version of the signature of the input image in order to identify an initial result, then perform on the initial result, one or more subsequent search operations that each use a longer or more complete version of the signature;

wherein the image analysis module operates to;

(i) programmatically determine a category of the item, (ii) based on the determined category, programmatically determine one or more focus items within each image of the collection of images, and (iii) determine the signature based, in part, on the focus item.

17. The system of claim 16, wherein the user-interface is configured to be operable by a user in order to enable the user to provide non-textual input for use in determining the search criteria.

18. The system of claim 16, further comprising an image procurement module that is configured to procure images in the collection, and metadata about individual images in the collection, from a plurality of network sites.

19. The system of claim 16, wherein the search module is configured to perform a search operation that identifies, as a result of the search operation, a set of images that each include a signature that is within a range that is determined by the search criteria.

20. The system of claim 16, wherein the search module is configured to perform a search operation that identifies, as a result of the search operation, a set of images that is deemed to be a best match for the search criteria.

21. The system of claim 16, wherein the image analysis module is configured to analyze individual images to determine a classification about one or more items shown in each of the images in the collection.