Three-dimensional indexing protocol

• US 10,460,287 B2
• Filed: 07/25/2016
• Issued: 10/29/2019
• Est. Priority Date: 08/14/2013
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Pavani, Sri Kaushik, Garg, Anshul
• Primary Examiner(s): Wilder, A. Hunter
• Application Number:
  US15/218,904
• Publication Number:
  US 20160335591A1
• Time in Patent Office: 1,191 Days
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06Q 40/12 Accounting
• CPC Class Codes: G06T 7/0004 Industrial image inspection

Abstract

The disclosure includes a system and method for indexing synthetically modified images of a high quality image. An image recognition application receives images of a product, crops background regions from the images, scales the image based on a minimum value among width and height of the image and generates multiple image sizes, blurs the images, brightens the image and indexes the images as being associated with the product. The images can be of box-shaped packages that include four or six images or cylindrical packages that include, for example, eight images of the packages. The images can be indexed in a k-dimensional tree for faster retrieval.

Claims

1. A method comprising:

generating, using one or more processors, a planogram of indexed products;

receiving, by the one or more processors, a captured image of a plurality of products;

identifying, by the one or more processors, the plurality of products in the captured image using the indexed products by;

computing a first set of features for the captured image including a location, an orientation, and an image descriptor for the first set of features;

comparing the first set of features for the captured image to features of a first indexed product to determine whether the first set of features for the captured image can be transformed to the features of the first indexed product by a combination of translation, rotation, and scaling;

determining the captured image matches the first indexed product responsive to determining that the first set of features for the captured image can be transformed to the features of the first indexed product by the combination of translation, rotation, and scaling;

blurring a region of a first product in the captured image to generate a partially blurred image, the first product matching the first indexed product;

computing a second set of features for the partially blurred image and comparing the second set of features for the partially blurred image to features of a second indexed product to determine whether the second set of features can be transformed to the features of the second indexed product by a combination of translation, rotation, and scaling;

determining the partially blurred image matches the second indexed product responsive to determining that the second set of features for the partially blurred image can be transformed to the features of the second indexed product by the combination of translation, rotation, and scaling; and

returning matches found for the plurality of products in the captured image;

analyzing, by the one or more processors, the plurality of products using the captured image and the planogram of indexed products; and

notifying, by the one or more processors, a user of a condition based on the analysis.

2. The method of claim 1, wherein analyzing the plurality of products further comprises determining, by the one or more processors, whether a change in inventory of the first product satisfies a threshold.

3. The method of claim 2, wherein determining whether the change in inventory of the first product satisfies a threshold includes determining whether the first product is out of stock.

4. The method of claim 3, wherein notifying the user includes an instruction to restock the first product.

5. The method of claim 1, wherein analyzing the plurality of products further comprises checking whether locations of the plurality of products in the captured image match expected locations based on the planogram of indexed products.

6. The method of claim 1, wherein analyzing the plurality of products further comprises generating sales statistics for the first product.

7. The method of claim 1, further comprising:

receiving, by the one or more processors, an image of the first product;

receiving, by the one or more processors, information relating to the first product; and

indexing, by the one or more processors, the image of the first product and the information relating to the first product as being associated with the first product.

8. A system comprising:

one or more processors; and

a memory, the memory storing instructions, which when executed cause the one or more processors to;

generate a planogram of indexed products;

receive a captured image of a plurality of products;

identify the plurality of products in the captured image using the indexed products by;

computing a first set of features for the captured image including a location, an orientation, and an image descriptor for the first set of features;

comparing the first set of features for the captured image to features of a first indexed product to determine whether the first set of features for the captured image can be transformed to the features of the first indexed product by a combination of translation, rotation, and scaling;

determining the captured image matches the first indexed product responsive to determining that the first set of features for the captured image can be transformed to the features of the first indexed product by the combination of translation, rotation, and scaling;

blurring a region of a first product in the captured image to generate a partially blurred image, the first product matching the first indexed product;

computing a second set of features for the partially blurred image and comparing the second set of features for the partially blurred image to features of a second indexed product to determine whether the second set of features can be transformed to the features of the second indexed product by a combination of translation, rotation, and scaling;

determining the partially blurred image matches the second indexed product responsive to determining that the second set of features for the partially blurred image can be transformed to the features of the second indexed product by the combination of translation, rotation, and scaling; and

returning matches found for the plurality of products in the captured image;

analyze the plurality of products using the captured image and the planogram of indexed products; and

notify a user of a condition based on the analysis.

9. The system of claim 8, wherein to analyze the plurality of products, the instructions further cause the one or more processors to determine whether a change in inventory of the first product satisfies a threshold.

10. The system of claim 9, wherein the change in inventory of the first product satisfying the threshold indicates the first product is out of stock.

11. The system of claim 10, wherein to notify the user the instructions further cause the one or more processors to include an instruction to restock the first product.

12. The system of claim 8, wherein to analyze the plurality of products the instructions further cause the one or more processors to check whether locations of the plurality of products in the captured image match expected locations based on the planogram of indexed products.

13. The system of claim 8, wherein to analyze the plurality of products the instructions further cause the one or more processors to generate sales statistics for the first product.

14. The system of claim 8, wherein the instructions further cause the one or more processors to:

receive, by the one or more processors, an image of the first product;

receive, by the one or more processors, information relating to the first product; and

index, by the one or more processors, the image of the first product and the information relating to the first product as being associated with the first product.

15. A computer program product comprising a non-transitory computer usable medium including a computer readable program, wherein the computer readable program, when executed on a computer causes the computer to:

generate a planogram of indexed products;

receive a captured image of a plurality of products;

identify the plurality of products in the captured image using the indexed products by;

computing a first set of features for the captured image including a location, an orientation, and an image descriptor for the first set of features;

comparing the first set of features for the captured image to features of a first indexed product to determine whether the first set of features for the captured image can be transformed to the features of the first indexed product by a combination of translation, rotation, and scaling;

determining the captured image matches the first indexed product responsive to determining that the first set of features for the captured image can be transformed to the features of the first indexed product by the combination of translation, rotation, and scaling;

blurring a region of a first product in the captured image to generate a partially blurred image, the first product matching the first indexed product;

computing a second set of features for the partially blurred image and comparing the second set of features for the partially blurred image to features of a second indexed product to determine whether the second set of features can be transformed to the features of the second indexed product by a combination of translation, rotation, and scaling;

determining the partially blurred image matches the second indexed product responsive to determining that the second set of features for the partially blurred image can be transformed to the features of the second indexed product by the combination of translation, rotation, and scaling; and

returning matches found for the plurality of products in the captured image;

analyze the plurality of products using the captured image and the planogram of indexed products; and

notify a user of a condition based on the analysis.

16. The computer program product of claim 15, wherein to analyze the plurality of products, the computer readable program further causes the computer to determine whether a change in inventory of the first product satisfies a threshold.

17. The computer program product of claim 16, wherein the change in inventory of the first product satisfying the threshold indicates the first product is out of stock.

18. The computer program product of claim 17, wherein to notify the user the computer readable program further causes the computer to include an instruction to restock the first product.

19. The computer program product of claim 15, wherein to analyze the plurality of products the computer readable program further causes the computer to check whether locations of the plurality of products in the captured image match expected locations based on the planogram of indexed products.

20. The computer program product of claim 15, wherein to analyze the plurality of products the computer readable program further causes the computer to generate sales statistics for the first product.