Method and a system for object recognition

• US 10,402,777 B2
• Filed: 06/07/2017
• Issued: 09/03/2019
• Est. Priority Date: 06/18/2014
• Status: Active Grant

• Current Assignee: Trax Technology Solutions Pte. Ltd. (Trax Technologies (SG) Pte Ltd.)
• Original Assignee: Trax Technology Solutions Pte. Ltd. (Trax Technologies (SG) Pte Ltd.)
• Inventors: Cohen, Daniel Shimon, Adato, Yair, Pomeranz, Dolev
• Primary Examiner(s): Shah, Utpal D
• Application Number:
  US15/616,474
• Publication Number:
  US 20170270475A1
• Time in Patent Office: 818 Days
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 20/68 Food, e.g. fruit or vegetables
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

The present disclosure provides a method of image processing comprising: obtaining by an imaging device a low resolution version and a high resolution version of a retail image, the high resolution version of the retail image being a temporary file to be erased automatically after a predetermined time period; transmitting to a server the low resolution version of the retail image; upon receipt of a request from the server, the request including data representative of a contour of an unidentified item in the low resolution version of the retail image, cropping a high resolution item image from the high resolution version of the retail image, the high resolution item image corresponding to the contour of the unidentified item; and transmitting the high resolution item image to the server thereby enabling updating an item database.

Claims

1. An image processing system for analyzing retail items displayed on store shelves, the system comprising:

a database configured to store a set of visual identifiers corresponding to a plurality of retail items;

at least one server configured to;

receive from a first electronic device at least one image of a plurality of products displayed on a store shelf;

determine a region of interest in the at least one image, wherein the region of interest includes a plurality of representations of similar products;

attempt to recognize at least one of the plurality of products to a degree of certainty by searching for a visual identifier in the region of interest of the at least one image;

send to a second electronic device a request about the at least one product, when the attempt at least partially fails, wherein the request asks a user operating the second electronic device to identify if the similar products in the region of interest are of a same type;

receive from the second electronic device a response; and

identify the at least one product based on the at least one image and the received response.

2. The image processing system of claim 1, wherein the request further asks the second electronic device to transmit an additional image of the at least one product.

3. The image processing system of claim 2, wherein the additional image of the at least one product has an image resolution higher than the received at least one image.

4. The image processing system of claim 2, wherein the additional image of the at least one product is captured from a different capturing angle than the received at least one image.

5. The image processing system of claim 2, wherein the additional image of the at least one product is captured by a different electronic device than the first electronic device.

6. The image processing system of claim 2, wherein the additional image of the at least one product includes an SKU bar code for the at least one product.

7. The image processing system of claim 1, wherein the request further asks a user operating the second electronic device to provide additional information regarding the least one product.

8. The image processing system of claim 7, wherein the additional information includes at least one detail characterizing the least one product.

9. The image processing system of claim 8, wherein the at least one detail characterizing the least one product is not visually shown on the least one product.

10. The image processing system of claim 1, wherein the at least one server is configured to update a product database based on the received response.

11. The image processing system of claim 1, wherein the request includes data representative of a contour of the at least one product, and the received response includes portion of an image that corresponds with the contour of the at least one product and with an image resolution higher than the at least one image received from the first electronic device.

12. The image processing system of claim 11, wherein the data representative of the contour includes a position and size of the at least one product in the at least one image received from the first electronic device.

13. The image processing system of claim 1, wherein the at least one product is identified through recognizing the visual identifier.

14. The image processing system of claim 1, wherein the database is further configured to store a set of high level identifiers corresponding to the plurality of retail items, and the at least one product is identified through recognizing a high level identifier.

15. The image processing system of claim 1, wherein the at least one product is identified without recognizing an SKU bar code on the at least one product.

16. The image processing system of claim 1, wherein the at least one server is configured to update a visual identifier of a retail item stored in the database based on the received response.

17. The image processing system of claim 1, wherein the at least one server is configured to add a new visual identifier of a new retail item to the database based on the at least one image and the received response.

18. The image processing system of claim 17, wherein the at least one server is further configured to recognize retail items using the new visual identifier added to the database.

19. The image processing system of claim 1, wherein the at least one server is further configured to highlight the region of interest in the at least one image received from the first electronic device, and to send the highlighted at least one image to the second electronic device along with the request.

20. The image processing system of claim 1, wherein the first electronic device is at least one of a digital camera, a fixed camera with communication layers, an electronic handheld device, an autonomous robotic device, and a drone.

21. The image processing system of claim 1, wherein the first electronic device and the second electronic device are a same device.

22. A non-transitory computer readable medium for image processing, the computer readable medium including instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising:

receiving from a first electronic device at least one image of a plurality of products displayed on a store shelf;

determining a region of interest in the at least one image, wherein the region of interest includes a plurality of representations of similar products;

attempting to recognize at least one of the plurality of products to a degree of certainty by searching for a visual identifier in the region of interest of the at least one image;

sending to a second electronic device a request about the at least one product, when the attempt at least partially fails, wherein the request asks a user operating the second electronic device to identify if the similar products in the region of interest are of a same type;

receiving from the second electronic device a response; and

identifying the at least one product based on the at least one image and the received response.

23. An image processing system for analyzing retail items displayed on store shelves, the system comprising:

a database configured to store a set of visual identifiers corresponding to a plurality of retail items;

at least one server configured to;

receive from a first electronic device at least one image of a plurality of products displayed on a store shelf;

attempt to recognize at least one of the plurality of products to a degree of certainty by searching for a visual identifier in the at least one image;

send to a second electronic device a request about the at least one product, when the attempt at least partially fails, wherein the request includes data representative of a contour of the at least one product;

receive from the second electronic device a response including portion of an image that corresponds with the contour of the at least one product and with an image resolution higher than the at least one image received from the first electronic device; and

identify the at least one product based on the at least one image and the received response.

24. The image processing system of claim 23, wherein the data representative of the contour includes a position and size of the at least one product in the at least one image received from the first electronic device.