Inventory confirmation

• US 9,595,038 B1
• Filed: 05/18/2015
• Issued: 03/14/2017
• Est. Priority Date: 05/18/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Cavalcanti, Daniel Hafner, Mishra, Pragyana K.
• Primary Examiner(s): Le, Vu
• Assistant Examiner(s): Woldemariam, Aklilu
• Application Number:
  US14/715,505
• Time in Patent Office: 666 Days
• Field of Search: 382/100, 382/103, 382/112, 382/128, 382/129, 382/130, 382/131, 382/305
• US Class Current: 1/1
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06Q 30/0185 Product, service or business identity fraud
• CPC Class Codes: G06T 3/02 Affine transformations for image registration G06T3/147; for image mosaicing G06T3/4038
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/42 Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
• CPC Class Codes: G06V 10/46 Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features colour feature extraction G06V10/56
• CPC Class Codes: G06V 10/7515 Shifting the patterns to accommodate for positional errors
• CPC Class Codes: G06V 2201/06 Recognition of objects for industrial automation
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks
• CPC Class Codes: G06V 30/424 Postal images, e.g. labels or addresses on parcels or postal envelopes

Abstract

Images of items may be evaluated to identify one or more attributes of the items, including points, contours or shapes corresponding to such items or their packaging, in order to determine whether such items are authentic or counterfeit. Such attributes may be compared to corresponding attributes of items that are known to be authentic. If the attributes identified in an image of an item arriving at a fulfillment center are determined to be consistent with attributes identified in an image of an authentic item, then the item arriving at the fulfillment center may be deemed authentic. If, however, the attributes identified in the image of the item arriving at the fulfillment center are not determined to be consistent with any attributes identified in images of any authentic items, then the item arriving at the fulfillment center may be deemed counterfeit.

Claims

1. A system comprising:

an imaging device;

a conveyor defining a direction of travel along an axis, and conveying a first item in the direction of travel;

a memory storing one or more computer-executable instructions; and

a computing device in communication with the memory, the imaging device and the conveyor, wherein the computing device is configured with the one or more computer-executable instructions to at least;

cause, using the imaging device, capture of at least a first image of a first packaging of the first item on the conveyor;

extract data regarding a first plurality of features provided on the first packaging of the first item from at least the first image;

define a first compact representation of a first portion of the first packaging of the first item based at least in part on the extracted data regarding the first plurality of features, wherein the first compact representation comprises information regarding a first shape provided on the first portion of the first packaging of the first item and information regarding a second shape provided on the first portion of the first packaging of the first item, the information regarding the first shape comprising information regarding at least one of a first contour, a first line segment, a first color or a first location of the first shape;

identify a second item designated as associated with the first item;

identify a second compact representation of a second portion of a second packaging of the second item, wherein the second compact representation comprises information regarding a third shape provided on the second portion of the second packaging of the second item and information regarding a fourth shape provided on the second portion of the second packaging of the second item, the information regarding the third shape comprising information regarding at least one a third contour, a third line segment, a third color or a third location of the third shape;

determine whether the information regarding the first shape provided on the first portion of the first packaging of the first item is consistent with the information regarding the third shape provided on the second portion of the second packaging of the second item using at least one processor;

in response to determining that the information regarding the first shape is consistent with the information regarding the third shape,

determine whether the information regarding the second shape provided on the first portion of the first packaging of the first item is consistent with the information regarding the fourth shape provided on the second portion of the second packaging of the second item using the at least one processor; and

in response to determining that the information regarding the second shape is consistent with the information regarding the fourth shape,

store an indication that the first item is associated with the second item in at least one data store.

2. The system of claim 1, wherein the computing device is further configured with the one or more computer-executable instructions to:

in response to determining that the information regarding the first shape is not consistent with the information regarding the third shape, or

in response to determining that the information regarding the second shape is not consistent with the information regarding the fourth shape,

store an indication that the first item is not associated with the second item in at least one data store.

3. The system of claim 1, wherein the computing device is further configured with the one or more computer-executable instructions to:

extract data regarding a first boundary of the first packaging of the first item from at least the first image,

wherein each of the first plurality of features is located within the first boundary of the first packaging of the first item, and

wherein the data regarding the first boundary of the first packaging of the first item is extracted according to at least one of a Canny edge detector or a Sobel operator.

4. The system of claim 1, wherein the computing device is further configured with the one or more computer-executable instructions to:

interpret a first marking provided on the first packaging of the first item based at least in part on the first image of the first packaging of the first item,

wherein the second item is identified based at least in part on the interpreted first marking.

5. A computer-implemented method comprising:

identifying at least a first image of at least a first surface of a first packaging of a first item using at least one processor;

extracting first information regarding at least one first shape provided on the first surface of the first packaging of the first item within the first image using the at least one processor by at least identifying a first boundary of the first packaging of the first item within the first image using the at least one processor, the at least one first shape being provided on the first surface of the first packaging of the first item within the first boundary;

identifying second information regarding at least one second shape provided on a second surface of a second packaging of a second item within a second image using the at least one processor, wherein the second item is distinct from the first item and is known to comply with at least one standard;

determining whether at least a first portion of the first information is consistent with a second portion of the second information; and

in response to determining that the first portion of the first information is not consistent with the second portion of the second information, storing a first indication that the first item does not comply with the at least one standard in at least one data store.

6. The computer-implemented method of claim 5, further comprising:

in response to determining that the first portion of the first information is consistent with the second portion of the second information,

storing a second indication that the first item complies with the at least one standard in the at least one data store.

7. The computer-implemented method of claim 5, wherein extracting the first information regarding the at least one first shape provided on the first surface of the first packaging of the first item further comprises:

performing a geometric transformation of the first image,

wherein the geometric transformation is at least one of a translation, a scaling, a reflection, a rotation or a shear of at least a portion of the first image comprising the first surface of the first packaging of the first item.

8. The computer-implemented method of claim 5, wherein the first boundary of the first packaging of the first item is identified according to at least one of a Canny edge detection algorithm or a Sobel operator.

9. The computer-implemented method of claim 5, wherein the at least one first shape is identified within the first boundary of the first packaging of the first item according to at least one of a Canny edge detection algorithm or a Sobel operator.

10. The computer-implemented method of claim 5, wherein the at least one first shape is at least a portion of at least one of a label, a character, a trademark or an indicium provided on the first surface of the first packaging of the first item, and

wherein the portion of the first image comprises the first surface.

11. The computer-implemented method of claim 5, wherein the at least one first shape comprises at least one of an arc, a circle, an ellipse, a hyperbola, a line segment, a parabola or a polygon.

12. The computer-implemented method of claim 11, wherein the first portion of the first information comprises at least one of:

an angle, a length or a radius of the arc;

at least one coordinate of a center of the circle;

a radius, a diameter, a circumference or an area of the circle;

at least one coordinate of at least one focus of the ellipse;

a sum of distances between the at least one focus and the ellipse;

a coordinate of at least one focal point of the hyperbola;

an equation defining at least one axis of the hyperbola;

a length of the line segment;

at least one coordinate of at least one end of the line segment;

at least one coordinate of a focus of the parabola;

an equation defining a directrix of the parabola;

at least one distance between one of the focus of the parabola or the directrix of the parabola and the parabola;

at least one coordinate of at least one vertex of the polygon;

or

a length of at least one side of the polygon.

13. The computer-implemented method of claim 5, wherein the first information comprises at least one of:

a type of the at least one first shape;

at least one dimension of the at least one first shape;

or

at least one coordinate of the at least one first shape, and

wherein the second information comprises at least one of;

a type of the at least one second shape;

at least one dimension of the at least one second shape;

or

at least one coordinate of the at least one second shape.

14. The computer-implemented method of claim 13, further comprising:

identifying at least one tolerance associated with one of the type, the at least one dimension or the at least one coordinate of the at least one second shape,

wherein determining whether at least the first portion of the first information is consistent with at least the second portion of the second information comprises;

determining whether one of the type, the at least one dimension or the at least one coordinate of the at least one first shape is within the at least one tolerance of the one of the type, the at least one dimension or the at least one coordinate of the at least one second shape.

15. The computer-implemented method of claim 5, further comprising:

receiving a designation of the first item as the second item from a source of the first item; and

identifying the second information regarding the at least one second shape provided on the second surface of the second packaging of the second item within the second image based at least in part on the designation.

16. The computer-implemented method of claim 15, wherein receiving the designation of the first item as the second item from the source of the first item further comprises:

interpreting at least one marked identifier provided on at least the first surface of the first packaging of the first item,

wherein the at least one marked identifier is associated with the second item.

17. The computer-implemented method of claim 5, further comprising:

identifying information regarding a plurality of items, wherein each of the plurality of items is known to comply with the at least one standard, and wherein the second item is one of the plurality of items;

identifying a plurality of images, wherein each of the plurality of images is of at least one surface of a respective packaging of the plurality of items, and wherein the second image is one of the plurality of images;

extracting information regarding shapes provided on surfaces of the respective packaging of the items within the plurality of images using the at least one processor, wherein the at least one second shape is one of the shapes; and

storing at least some of the information regarding the shapes provided on the surfaces of the respective packaging of the items within the plurality of images in the at least one data store, wherein the second information regarding the at least one second shape provided on the second surface of the second packaging of the second item within the second image is included in the at least some of the information regarding the shapes provided on the surfaces of the respective packaging of the items within the plurality of images, and

wherein identifying the second information regarding the at least one second shape provided on the second surface of the second packaging of the second item within the second image further comprises;

comparing the first information to the at least some of the information regarding the shapes provided on the surfaces of the respective packaging of the items within the plurality of images in the at least one data store.

18. The computer-implemented method of claim 5, wherein identifying at least the first image of at least the first surface of the first packaging of the first item further comprises:

capturing the first image of at least the first surface of the first packaging of the first item using at least one imaging device.

19. The computer-implemented method of claim 18, further comprising:

causing the first item to travel along a first conveyor,

wherein the first image of at least the first surface of the first packaging of the first item is captured while the first item travels along the first conveyor.

20. A method comprising:

capturing a first image of at least one surface of a first packaging of a first item traveling on a conveying system at a fulfillment center, wherein the conveying system includes at least one conveyor;

normalizing the first image according to a first transformation using at least one processor;

identifying a first boundary on the at least one surface of the first packaging of the first item within the first image using the at least one processor;

identifying at least one first shape included in the normalized first image using the at least one computer processor, the at least one first shape being provided on the at least one surface of the first packaging of the first item within the first boundary;

defining a first set of attributes of the at least one first shape included in the normalized first image using the at least one processor;

identifying a plurality of sets of attributes of shapes included in images of compliant items using the at least one computer processor, wherein each of the images includes a shape provided on a surface of a respective packaging of one of the compliant items;

comparing the first set of attributes of the at least one first shape included in the normalized first image to the plurality of sets of attributes of the shapes included in the images of the respective packaging of the compliant items using the at least one processor;

determining whether the first set of attributes of the at least one first shape included in the normalized first image corresponds to at least one of the plurality of sets of attributes of the shapes included in the images of the respective packaging of the compliant items using the at least one processor;

in response to determining that the first set of attributes of the at least one first shape included in the normalized first image does not correspond to any of the plurality of sets of attributes of the shapes included in the images of the respective packaging of the compliant items,

identifying a first storage facility associated with items that are not compliant using the at least one processor; and

causing the first item to be transported to the first storage facility by the conveying system, wherein each of the compliant items is determined based at least in part on a comparison of the plurality of sets of attributes with at least one standard.

21. The method of claim 20, further comprising:

in response to determining that the first set of attributes of the at least one first shape included in the normalized first image corresponds to the at least one of the plurality of sets of attributes of the shapes included in the images of the respective packaging of the compliant items,

identifying a second item associated with the at least one of the plurality of sets of attributes of the shapes included in the images of the respective packaging of the compliant items using the at least one processor;

identifying a second storage facility associated with the second item using the at least one processor; and

causing the first item to be transported to the second storage facility by the conveying system.

22. The method of claim 21, further comprising:

in response to determining that the first set of attributes of the at least one first shape included in the normalized first image corresponds to the at least one of the plurality of sets of attributes of the shapes included in the images of the respective packaging of the compliant items,

determining a type of item associated with the second item using the at least one processor; and

storing an indication that one of the type of item has arrived at the fulfillment center.