Identification of items depicted in images

• US 9,495,386 B2
• Filed: 02/16/2009
• Issued: 11/15/2016
• Est. Priority Date: 03/05/2008
• Status: Expired due to Fees

• Current Assignee: eBay Inc.
• Original Assignee: eBay Inc.
• Inventors: Tapley, John, Farraro, Eric J., Gupta, Raghav, Grandhi, Roopnath
• Primary Examiner(s): ULRICH, NICHOLAS S
• Application Number:
  US12/371,882
• Publication Number:
  US 20090304267A1
• Time in Patent Office: 2,829 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/434 using image data, e.g. images, photos, pictures taken by a user
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 40/186 Templates
• CPC Class Codes: G06Q 30/0601 Electronic shopping [e-shopping]
• CPC Class Codes: G06Q 30/0623 Item investigation
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06V 10/56 relating to colour

Abstract

In an example embodiment, a method of identifying an item depicted in an image is provided. In this method, the image depicting the item is accessed; in addition, other images and their item identifiers are also accessed. A match of the image with one of the other images is identified. The match can be based on a variety of matching techniques, such as the application of an edge detection algorithm and the conversion of the images into color histograms. With a match, the image is then associated with an item identifier of the matched image. In one example, a template associated with one of the item identifiers can be accessed.

Claims

1. A method of identifying an item depicted in a first image, the method comprising:

receiving a request to identify the item, the request including the first image;

accessing a plurality of other images and a plurality of associated item data, the plurality of associated item data including a description in each of a plurality of item listings;

parsing the plurality of associated item data to identify a plurality of item identifiers associated with the plurality of other images;

calculating at least one statistical difference between a first color histogram of the first image and a color histogram of each of the plurality of other images;

determining a second color histogram of a second image as having a lowest calculated statistical difference among the at least one statistical difference between the first color histogram and the color histogram of each of the plurality of other images;

detecting a first edge in the first image and a second edge in the second image;

identifying a match of the first color histogram with the second color histogram responsive to determining the second color histogram as having the lowest calculated statistical difference among the at least one statistical difference, the identifying being based on the detection of the first edge in the first image and the second edge in the second image, the second image depicting a second item that corresponds to a barcode among the plurality of item identifiers;

associating the first image with the barcode that corresponds to the second item depicted in the second image based on the match of the first and second color histograms of the first and second images;

accessing a template associated with the barcode that corresponds to the second item depicted in the second image based on the match of the first and second color histograms of the first and second images; and

transmitting the template in a response to the request.

2. The method of claim 1, wherein the identification of the match of the first color histogram with the second color histogram comprises comparing the first image with the second image.

3. The method of claim 1, wherein the identification of the match comprises:

converting the first image into the first color histogram;

converting the second image into the second color histogram; and

comparing the first color histogram with the second color histogram.

4. A non-transitory, machine-readable medium that stores instructions that, when performed by one or more processors of a machine, cause the machine to perform operations comprising:

accessing a first image depicting an item;

accessing a plurality of other images and a plurality of associated item data, the plurality of associated item data including a description in each of a plurality of item listings;

parsing the plurality of associated item data to identify a plurality of item identifiers associated with the plurality of other images;

calculating at least one statistical difference between a first color histogram of the first image and a color histogram of each of the plurality of other images;

determine a second color histogram of the second image as having a lowest calculated statistical difference among the at least one statistical difference between the first color histogram and the color histogram of each of the plurality of other images;

detecting a first edge in the first image and a second edge in the second image;

identifying a match of the first color histogram with the second color histogram responsive to determining the statistical difference between the first color histogram and the second color histogram as having the lowest statistical difference among the at least one statistical difference, the identifying being based on the detection of the first edge in the first image and the second edge in the second image, the second image depicting a second item that corresponds to a barcode among the plurality of item identifiers;

associating the first image with the barcode that corresponds to the second item depicted in the second image based on the match of the first and second color histograms of the first and second images, the association identifying the item in the first image; and

accessing a template associated with the barcode that corresponds to the second item depicted in the second image based on the match of the first and second color histograms of the first and second images.

5. The non-transitory, machine-readable medium of claim 4, wherein the operation of identifying the match of the first color histogram with the second color histogram comprises comparing the first image with the plurality of other images.

6. The non-transitory, machine-readable medium of claim 4, wherein the operation of identifying the match comprises:

converting the first image into the first color histogram;

converting the second image into the second color histogram; and

comparing the first color histogram with the second color histogram.

7. The non-transitory, machine-readable medium of claim 4, wherein the instructions, when performed by a machine, cause the machine to perform operations further comprising:

receiving a request to identify the item depicted in the first image;

accessing a plurality of item data associated with the item identifier that corresponds to the second image; and

transmitting a response to the request, the response including the plurality of item data and the item identifier that corresponds to the second image.

8. The non-transitory, machine-readable medium of claim 7, wherein the plurality of item data associated with the item identifier that corresponds to the second image includes a description of the item.

9. The non-transitory, machine-readable medium of claim 7, wherein the plurality of item data associated with the item identifier that corresponds to the second image includes a location of the item.

10. The non-transitory, machine-readable medium of claim 4, wherein the edge detection algorithm is a Canny edge detector algorithm.

11. A processing system comprising:

at least one processor; and

a memory in communication with the at least one processor, the memory being configured to store an item recognition module that is executable by the at least one processor, the item recognition module having instructions, that when executed by the at least one processor, cause operations to be performed, the operations comprising;

accessing a first image depicting an item;

accessing a plurality of other images and a plurality of associated item data, the plurality of associated item data including a description in each of a plurality of item listings;

parsing the plurality of associated item data to identify a plurality of item identifiers associated with the plurality of other images;

calculating at least one statistical difference between a first color histogram of the first image and a color histogram of each of the plurality of other images;

determining a second color histogram of a second image as having a lowest calculated statistical difference among the at least one statistical difference between the first color histogram and the color histogram of each of the plurality of other images;

detecting a first edge in the first image and a second edge in the second image;

identifying a match of the first color histogram with the second color histogram responsive to determining the second color histogram as having the lowest calculated statistical difference among the at least one statistical difference, the identifying being based on the detection of the first edge in the first image and the second edge in the second image, the second image depicting a second item that corresponds to a barcode among the plurality of item identifiers;

associating the first image with the barcode that corresponds to the second item depicted in the second image based on the match of the first and second color histograms of the first and second images; and

accessing a template associated with the barcode that corresponds to the second item depicted in the second image based on the match of the first and second color histograms of the first and second images.

12. The processing system of claim 11, wherein the item recognition module includes a neural network module that is executable by the at least one processor, wherein the neural network module has instructions that, when executed by the at least one processor, cause operations to be performed, comprising to identifying the match of the first image with the second image based on a learning algorithm.

13. The processing system of claim 11, wherein the operation of identifying the match comprises:

converting the first image into the first color histogram;

converting the second image into the second color histogram;

comparing the first color histogram with the second color histogram.

14. An apparatus comprising:

one or more processors and executable instructions accessible on a computer-readable medium that, when executed, configure the one or more processors to at least;

receive a request to identify the item, the request including the first image, to access a plurality of other images and a plurality of associated item data, the plurality of item data including a description in each of a plurality of item listings, and to further parse the plurality of associated item data to identify a plurality of item identifiers associated with the plurality of other images;

calculate at least one statistical difference between a first color histogram of the first image and a color histogram of each of the plurality of other images, and to determine a second color histogram of a second image as having a lowest calculated statistical difference among the at least one statistical difference between the first color histogram and the color histogram of each of the plurality of other images;

detect a first edge in the first image and a second edge in the second image;

identify a match of the first color histogram with the second color histogram responsive to determining the second color histogram as having the lowest calculated statistical difference among the at least one statistical difference, the identifying being based on the detection of the first edge in the first image and the second edge in the second image, the second image depicting a second item that corresponds to a barcode among the plurality of item identifiers;

associate the first image with the barcode that corresponds to the second item depicted in the second image based on the match of the first and second color histograms, the association identifying the item in the image; and

access a template associated with the barcode that corresponds to the second item depicted in the second image based on the match of the first and second color histograms of the first and second images, and to transmit the template in a response to the request.

15. The apparatus of claim 14, wherein the plurality of other images are submitted by a plurality of users.