System and method for enabling image recognition and searching of remote content on display

• US 8,712,862 B2
• Filed: 09/14/2012
• Issued: 04/29/2014
• Est. Priority Date: 05/09/2005
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Gokturk, Salih Burak, Chiao, Dan, Phillips, Jacquie, Moran, Mark, Vanhoucke, Vincent, Khan, Azhar, Lin, Xiaofan, Shah, Munjal, Miller, Andrew, Dalal, Navneet, Vu, Diem
• Primary Examiner(s): Dunham, Jason
• Assistant Examiner(s): Civan, Ethan D
• Application Number:
  US13/618,653
• Publication Number:
  US 20130132236A1
• Time in Patent Office: 592 Days
• Field of Search: 705 261- 272
• US Class Current: 705/26.1
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06Q 30/0623 Item investigation
• CPC Class Codes: G06Q 30/0625 Directed, with specific intent or strategy
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06Q 30/08 Auctions

Abstract

Images are analyzed by programmatic mechanisms for accessing one or more remote web pages to retrieve content on display at the remote web pages. The retrieved images may be analyzed to determine information about an object shown in a corresponding image of the content on display. At least a portion of the object shown in the corresponding image of the content on display may be made selectable and associated with the determined information. This determined information may subsequently be used, in for example, search applications.

Claims

1. A method for analyzing images, the method being implemented by one or more hardware processors and comprising:

using the one or more hardware processors to perform steps comprising;

analyzing content, the content including image content and text content,

wherein the image content includes an image, and wherein analyzing the content includes analyzing the text content to determine one or more assumptions about a location or shape of an object of interest in the image;

determining a classification of the object of interest, wherein determining the classification of the object of interest is based at least in part on the text content;

performing image analysis on the image based at least in part on the classification and the one or more assumptions to detect the object of interest in the image;

determining information about the object of interest, based on the image analysis, using one or more sources other than the analyzed content;

providing data with the image in order to make the object of interest selectable apart from a remainder of the image; and

associating the information with the image so that at least some of the information is displayed when the object of interest is selected.

2. The method of claim 1, wherein analyzing the content is performed in response to receiving a trigger signal from a site of a web page on which the content is provided.

3. The method of claim 2, wherein providing data with the image includes providing a programmatic element for the web page to cause the trigger signal to generate in response to a download of that web page.

4. The method of claim 1, wherein determining the classification of an object of interest includes identifying one or more key words in the text content.

5. The method of claim 4, wherein the classification correlates the object of interest to one or more classes of merchandise items.

6. The method of claim 1, wherein performing image analysis includes determining a signature of at least a portion of the object of interest.

7. The method of claim 1, wherein performing image analysis includes identifying one or more visual characteristics of the object of interest.

8. The method of claim 7, wherein the one or more visual characteristics correspond to one or more of a color, a pattern, or a shape of at least a portion of the object of interest.

9. The method of claim 1, further comprising determining a search criterion from the information, obtaining a search result from the search criterion, and associating at least a portion of the search result with the object of interest so that at least the portion of the search result is displayed when the object of interest is selected.

10. The method of claim 9, wherein obtaining the search result includes obtaining one or more records that each contain one or more images of objects that satisfy the search criterion.

11. The method of claim 9, wherein obtaining the result includes obtaining a set of records that each include an object of the classification as the object of interest.

12. The method of claim 1, wherein the object of interest corresponds to an item of clothing or apparel.

13. The method of claim 12, wherein determining the classification includes determining, from the text content, a type of clothing or apparel that is depicted in the image.

14. The method of claim 13, wherein determining the type of clothing includes determining that the image content pertains to a celebrity.

15. A computer system comprising:

a memory that stores a set of instructions;

one or more processors that are coupled to the memory and use the set of instructions to;

analyze content, the content including image content and text content, wherein the image content includes an image, and wherein analyzing the content includes analyzing the text content to determine one or more assumptions about a location or shape of an object of interest in the image;

determine a classification of the object of interest, wherein determining the classification of the object of interest is based at least in part on the text content;

perform image analysis on the image based at least in part on the classification and the one or more assumptions to detect the object of interest in the image;

determine information about the object of interest, based on the image analysis, using one or more sources other than the analyzed content;

provide instructions with the image in order to make the object of interest selectable apart from a remainder of the image; and

associate the information with the image so that at least some of the information is displayed when the object of interest is selected.

16. The computer system of claim 15, wherein the one or more processors determine the classification of the object of interest by identifying one or more key words in the text content.

17. The computer system of claim 15, wherein the one or more processors perform the image analysis by identifying one or more visual characteristics of the object of interest.

18. The computer system of claim 17, wherein the one or more visual characteristics correspond to one or more of a color, a pattern, or a shape of at least a portion of the object of interest.

19. The computer system of claim 15, wherein the one or more processors determine a search criterion from the information, obtain a search result from the search criterion, and associate at least a portion of the search result with the object of interest so that at least the portion of the search result is displayed when the object is selected.

20. A non-transitory computer-readable medium that stores a set of instructions, the set of instructions being executable by one or more processors to cause the one or more processors to perform operations that include:

analyze content, the content including image content and text content, wherein the image content includes an image, and wherein analyzing the content includes analyzing the text content to determine one or more assumptions about a location or shape of an object of interest in the image;

determine a classification of the object of interest, wherein determining the classification of the object of interest is based at least in part on the text content;

perform image analysis on the image based at least in part on the classification and the one or more assumptions to detect the object of interest in the image;

determine information about the object of interest, based on the image analysis,

using one or more sources other than the analyzed content;

provide instructions with the image in order to make the object of interest selectable apart from a remainder of the image; and

associate the information with the image so that at least some of the information is displayed when the object of interest is selected.