System and method for search portions of objects in images and features thereof

• US 7,657,126 B2
• Filed: 11/07/2007
• Issued: 02/02/2010
• Est. Priority Date: 05/09/2005
• Status: Expired due to Fees

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Like.Com (Alphabet Inc.)
• Inventors: Shah, Munjal, Torresani, Lorenzo, Vu, Diem, Yang, Danny, Lin, Xiaofan, Sumengen, Baris, Dalal, Navneet, Gokturk, Salih Burak, Anguelov, Dragomir, Vanhoucke, Vincent, Khan, Azhar
• Primary Examiner(s): Kim; Charles
• Application Number:
  US11/936,734
• Publication Number:
  US 20080212899A1
• Time in Patent Office: 818 Days
• Field of Search: None
• US Class Current: 382/305
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06Q 30/0603 Catalogue ordering
• CPC Class Codes: Y10S 707/949 Inventory

Abstract

Embodiments enable searching of portions of objects in images, including programmatically analyzing each image in a collection in order to determine image data that, for individual images in the collection, represents one or more visual characteristics of a portion of an object shown in that image. A user is enabled to specify one or more search criteria that includes image data, and a search result may be determined based on one or more images in the collection that show a corresponding object that has a portion that satisfies a threshold. The threshold is defined at least in part by the one or more search criteria.

Claims

1. A method for enabling searching of objects in images, the method comprising:

using one or more processors to perform steps comprising;

programmatically analyzing each image in a collection of images, including determining at least a local feature for individual objects that appear in each image of the collection, the local feature of each object having a visual characteristic;

determining, from user-input, one or more search criteria;

corresponding to at least a portion of an object in a user selected image;

determining a similarity measurement between (i) the one or more search criteria and (ii) the local feature of objects individually depicted in at least some of the images in the collection;

making a determination, from the similarity measurements, as to N images representing a portion of the collection that display corresponding objects that (i) individually have one or more local features with visual characteristics that are most similar to at least the portion of the object in the user selected image, (ii) but which are not uniquely identifiable from the one or more search criteria;

wherein determining, from user-input, one or more search criteria further comprises;

selecting the portion of the object in the user selected image from user-input,

determining a value of a visual characteristic of the selected portion of the object in the user selected image, wherein the visual characteristic corresponds to at least one of a color, shape, pattern, or texture of the of the object,

selecting, from user-input, an alternative value for the visual characteristic of the selected portion of the object,

specifying at least a color and a shape criteria for the selected portion of the object in the user selected image and assigning weights for the specified color and shape criteria, including specifying or weighting the color or shape criteria using the alternative value.

2. The method of claim 1, wherein determining the one or more search criteria includes determining a recognition signature of at least the portion of object in the user selected image.

3. The method of claim 1, further comprising enabling a user to select a stored entry image as the user selected image.

4. A method for enabling searching of objects in images, the method comprising:

using one or more processors to perform steps comprising;

obtaining an image of an object for a collection of images;

programmatically analyzing the image in order to determine one or more visual characteristics of (i) the object, and (ii) one or more local features or portions of the object;

storing the visual characteristics determined of each object, including the visual characteristics of the one or more local feature or portions of the object;

determining, from user-input, one or more search criteria corresponding to at least a portion of an object in a user selected image; and

in response to receiving the one or more search criteria, referencing the search criteria against the stored visual characteristics, including visual characteristics corresponding to the one or more local features or portions of the individual objects, in order to determine one or more objects that have one or more local features or portions that satisfy the one or more search criteria;

wherein determining, from user-input, one or more search criteria further comprises;

selecting the portion of the object in the user selected image,

determining a value of a visual characteristic of the selected portion of the object in the user selected image, wherein the visual characteristic corresponds to at least one of a color, shape, pattern, or texture of the portion of the object,

selecting, from user-input, an alternative value for the visual characteristic of the selected portion of the object,

specifying at least a color and a shape criteria for the selected portion of the object in the user selected image and assigning weights for the specified color and shape criteria, including specifying or weighting the color or shape criteria using the alternative value.

5. The method of claim 4, wherein determining one or more search criteria includes determining a recognition signature of at least the portion of object in the user selected image.

6. The method of claim 4, further comprising enabling a user to select a stored entry image as the user selected image.