Image-based popularity prediction

• US 8,977,629 B2
• Filed: 05/02/2012
• Issued: 03/10/2015
• Est. Priority Date: 05/24/2011
• Status: Active Grant

• Current Assignee: eBay Inc.
• Original Assignee: eBay Inc.
• Inventors: Goswami, Anjan, Chung, Sung Hwan
• Primary Examiner(s): CHANNAVAJJALA, SRIRAMA T
• Application Number:
  US13/462,179
• Publication Number:
  US 20120303615A1
• Time in Patent Office: 1,042 Days
• Field of Search: 707705-710, 707/723, 707/758, 707765-770, 707780-782, 707804-805, 707/688, 707728-730, 707/748, 707/749, 709/203, 709/206, 709215-220, 705/14.41, 705/14.23, 705/27.1, 382/162, 382/164, 382/168, 382/171, 382/190, 382193-194, 382/219
• US Class Current: 707/748
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 16/24 Querying
• CPC Class Codes: G06F 16/242 Query formulation
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/2462 Approximate or statistical queries
• CPC Class Codes: G06F 16/2465 Query processing support for facilitating data mining operations in structured databases
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/332 Query formulation
• CPC Class Codes: G06F 16/338 Presentation of query results
• CPC Class Codes: G06F 16/345 Summarisation for human users
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 17/00 Digital computing or data processing equipment or methods, specially adapted for specific functions information retrieval, database structures or file system structures therefor G06F16/00
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/2115 by evaluating different subsets according to an optimisation criterion, e.g. class separability, forward selection or backward elimination
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06N 5/048 : Fuzzy inferencing
• CPC Class Codes: G06Q 30/02 : Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06V 10/10 : Image acquisition document image scanning and transmission H04N1/00; control of digital cameras H04N23/60
• CPC Class Codes: G06V 10/40 : Extraction of image or video features
• CPC Class Codes: G06V 10/507 : Summing image-intensity values; Histogram projection analysis
• CPC Class Codes: G06V 10/75 : Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
• CPC Class Codes: G06V 10/771 : Feature selection, e.g. selecting representative features from a multi-dimensional feature space
• CPC Class Codes: G06V 20/695 : Preprocessing, e.g. image segmentation

Abstract

A machine may be configured to access an image of an item described by a description of the item. The machine may determine an image quality score of the image based on an analysis of the image. A request for search results that pertain to the description may be received by the machine, and the machine may present a search result that references the item'"'"'s image, based on its image quality score. Also, the machine may access images of items and descriptions of items and generate a set of most frequent text tokens included in the item descriptions. The machine may identify an image feature exhibited by an item'"'"'s image and determine that a text token from the corresponding item description matches one of the most frequent text tokens. A data structure may be generated by the machine to correlate the identified image feature with the text token.

Claims

1. A method comprising:

accessing an image that corresponds to a description of an item depicted in the image;

determining an image quality score of the image that depicts the item and corresponds to the description of the item, the determining of the image quality score of the image including segmenting the image to identify a foreground of the image and a background of the image and determining a brightness difference between the segmented background of the image and the segmented foreground of the image, the determined image quality score representing a degree of clarity with which the image shows the item in the segmented foreground of the image, the determining of the image quality score being performed by a processor of a machine based on an analysis of the image that depicts the item;

receiving a request for search results of which at least some pertain to the description of the item depicted in the image; and

presenting a search result that is referential of the image of the item based on the image quality score of the image and in response to the request for the search results of which at least some pertain to the description of the item depicted in the image, the presenting of the search result being performed by a processor of a machine.

2. The method of claim 1, wherein:

the accessing of the image includes accessing a user-submitted listing that includes the description of the item and the image of the item.

3. The method of claim 1, wherein:

the presenting of the search result based on the image quality score includes ranking the search results based on the image quality score and displaying the search results according to at least one rank determined by the ranking of the search results.

4. The method of claim 1, wherein:

the determining of the image quality score based on the analysis of the image includes quantizing color pixel values from the foreground of the image that depicts the item into a color histogram.

5. The method of claim 1, wherein:

the determining of the image quality score based on the analysis of the image includes determining a texture histogram that indicates magnitudes of edge responses within the foreground of the image that depicts the item.

6. The method of claim 1, wherein:

the determining of the image quality score based on the analysis of the image includes computing a hidden unit response with a Restricted Boltzmann Machine.

7. The method of claim 1, wherein:

the determining of the image quality score based on the analysis of the image includes determining a global feature selected from a group consisting of;

an aspect ratio of the image that depicts the item,

a brightness value of the image,

a dynamic range of the image, and

a contrast value of the image.

8. The method of claim 1, wherein:

the determining of the image quality score based on the analysis of the image includes determining a regional feature selected from a group consisting of;

a background lightness of the image that depicts the item,

a background uniformity of the image,

a foreground colorfulness of the image,

a ratio of background area to foreground area in the image,

and

a difference between a background contrast and a foreground contrast of the image.

9. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

accessing an image that corresponds to a description of an item depicted in the image;

determining an image quality score of the image that depicts the item and corresponds to the description of the item, the determining of the image quality score of the image including segmenting the image to identify a foreground of the image and a background of the image and determining a brightness difference between the segmented background of the image and the segmented foreground of the image, the determined image quality score representing a degree of clarity with which the image shows the item in the segmented foreground of the image, the determining of the image quality score being performed by the one or more processors of the machine based on an analysis of the image that depicts the item;

receiving a request for search results of which at least some pertain to the description of the item depicted in the image; and

presenting a search result that is referential of the image of the item based on the image quality score of the image and in response to the request for the search results of which at least some pertain to the description of the item depicted in the image.

10. A system comprising:

an access module configured to access an image that corresponds to a description of an item depicted in the image;

a processor configured by a quality module to determine an image quality score of the image that depicts the item and corresponds to the description of the item, the determining of the image quality score of the image including segmenting the image to identify a foreground of the image and a background of the image and determining a brightness difference between the segmented background of the image and the segmented foreground of the image, the determined image quality score representing a degree of clarity with which the image shows the item in the segmented foreground of the image, the determining of the image quality score being based on an analysis of the image that depicts the item;

a request module configured to receive a request for search results of which at least some pertain to the description of the item depicted in the image; and

a processor configured by a result module to present a search result that is referential of the image of the item based on the image quality score of the image and in response to the request for the search results of which at least some pertain to the description of the item depicted in the image.

11. The system of claim 10, wherein:

the result module is configured to rank the search results based on the image quality score and display the search results according to at least one rank determined by the ranking of the search results.

12. A method comprising:

accessing item images that include an item image illustrative of an item;

accessing item descriptions that correspond to the item images and include an item description inclusive of a text token and descriptive of the item depicted in the item image;

generating a set of most frequent text tokens included in the item descriptions based on the item descriptions;

identifying an image feature exhibited by the item image that depicts the item described by the item description, the identifying of the image feature including segmenting the image to identify a foreground of the image and a background of the image and determining a brightness difference between the segmented background of the image and the segmented foreground of the image as the identified image feature, the identifying of the image feature being performed by a processor of a machine;

determining that the text token included in the item description matches at least one of the generated set of most frequent text tokens included in the item descriptions; and

generating a data structure that correlates the e identified image feature exhibited by the item image with the text token included in the item description determined to match at least one of the generated set of most frequent text tokens, the generating of the data structure being performed by a processor of a machine.

13. The method of claim 12, wherein:

the generating of the data structure includes generating an average feature vector that corresponds to the text token and indicates the identified image feature is a component of the average feature vector of the text token.

14. The method of claim 13, wherein:

the generating of the average feature vector is based on multiple image features identified from multiple item images that correspond to multiple item descriptions that are each inclusive of the text token.

15. The method of claim 12, wherein:

the item description is a title of a listing that merchandises the item depicted in the item image.

16. The method of claim 12, wherein:

the text token is a reference text token correlated with the identified image feature by the generated data structure; and

the method further comprises;

receiving a query that includes a query text token;

determining that the query text token matches the reference text token correlated with the identified image feature; and

generating a relevance score that represents a degree of relevance between the query and the item image that depicts the item, the generating of the relevant score being based on the data structure that correlates the image feature exhibited by the item image with the text token included in the item description.

17. The method of claim 16, wherein:

the generating of the relevance score is based on an average feature vector that corresponds to the text token and indicates the identified image feature is a component of the average feature vector of the text token.

18. The method of claim 12, wherein:

the identifying of the image feature exhibited by the item image includes determining a global feature selected from a group consisting of;

an aspect ratio of the item image illustrative of the item,

a brightness value of the item image,

a dynamic range of the item image, and

a contrast value of the item image.

19. The method of claim 12, wherein:

the identifying of the image feature exhibited by the item image includes determining a regional feature selected from a group consisting of;

a background lightness of the item image illustrative of the item,

a background uniformity of the item image,

a foreground colorfulness of the item image,

a ratio of background area to foreground area in the item image,

and

a difference between a background contrast and foreground contrast of the item image.

20. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

accessing item images that include an item image illustrative of an item;

accessing item descriptions that correspond to the item images and include an item description inclusive of a text token and descriptive of the item depicted in the item image;

generating a set of most frequent text tokens included in the item descriptions based on the item descriptions;

identifying an image feature exhibited by the item image that depicts the item described by the item description, the identifying of the image feature including segmenting the image to identify a foreground of the image and a background of the image and determining a brightness difference between the segmented background of the image and the segmented foreground of the image as the identified image feature, the identifying of the image feature being performed by the one or more processors of the machine;

determining that the text token included in the item description matches at least one of the generated set of most frequent text tokens included in the item descriptions; and

generating a data structure that correlates the identified image feature exhibited by the item image with the text token included in the item description determined to match at least one of the generated set of most frequent text tokens.

21. A system comprising:

an access module configured to access item images that include an item image illustrative of an item; and

to

access item descriptions that correspond to the item images and include an item description inclusive of a text token and descriptive of the item depicted in the item image;

a key module configured to generate a set of most frequent text tokens included in the item descriptions based on the item descriptions;

a processor configured by a feature module to identify an image feature exhibited by the item image that depicts the item described by the item description, the identifying of the image feature including segmenting the image to identify a foreground of the image and a background of the image and determining a brightness difference between the segmented background of the image and the segmented foreground of the image as the identified image feature, the identifying of the image feature being performed by a processor of a machine;

a match module configured to determine that the text token included in the item description matches at least one of the generated set of most frequent text tokens included in the item descriptions; and

a processor configured by a generation module to generate a data structure that correlates the identified image feature exhibited by the item image with the text token included in the item description determined to match at least one of the generated set of most frequent text tokens.