Object recognition for exception handling in automatic machine-readable symbol reader systems

• US 9,239,943 B2
• Filed: 05/29/2014
• Issued: 01/19/2016
• Est. Priority Date: 05/29/2014
• Status: Active Grant

• Current Assignee: Datalogic Adc, Inc. (Datalogic S.p.A.)
• Original Assignee: Datalogic Adc, Inc. (Datalogic S.p.A.)
• Inventors: Svetal, Michael
• Primary Examiner(s): LE, THIEN MINH
• Application Number:
  US14/289,950
• Publication Number:
  US 20150347801A1
• Time in Patent Office: 600 Days
• Field of Search: 235/454, 23547201-47203, 235/483, 235/484, 235/485
• US Class Current: 1/1
• CPC Class Codes: G06F 18/40 Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
• CPC Class Codes: G06K 7/10762 Relative movement
• CPC Class Codes: G06K 7/1447 extracting optical codes from image or text carrying said optical code
• CPC Class Codes: G06V 10/945 User interactive design; Environments; Toolboxes
• CPC Class Codes: G06V 10/987 with the intervention of an operator

Abstract

Systems and methods for exception handling in an automated machine-readable symbol reader system having a machine-readable symbol reader that captures machine-readable symbols within a view volume. One or more image capture devices obtain a plurality of images of an exception object in response to an exception generated in the view volume. A processor receives the images, identities visual object recognition features from each image, and compares the features to determine one or more descriptive measures indicative of a likelihood that an operator (e.g., store employee) will be able to identify the exception object by viewing the image. The processor displays at least one of the images (e.g., the most descriptive image) on a display device according to the descriptive measure so that an operator can rapidly identify the identity of the exception object and take steps to resolve the exception.

Claims

1. A method of operation for an automated machine-readable symbol reader system, the method comprising:

receiving a plurality of images of an exception object in at least one processor-readable medium, the exception object having not been successfully identified by a machine-readable symbol reader of the machine-readable symbol reader system;

extracting, by at least one processor, one or more visual features from each of the plurality of received images;

accessing a database of visual model data associated with a plurality of known objects;

comparing, by the at least one processor, the extracted one or more visual features of each of the plurality of images with the visual model data associated with the plurality of known objects;

determining, by the at least one processor, a descriptive measure for each of the plurality of images based on the comparison of the extracted one or more visual features with the visual model data; and

sequentially displaying on a display device two or more of the plurality of images in an order dependent on the determined descriptive measures.

2. The method of claim 1 wherein comparing the extracted one or more visual features of each of the plurality of images with the visual model data associated with the plurality of known objects comprises identifying matches between the extracted visual features and the visual model data associated with the plurality of known objects, and determining a descriptive measure comprises determining a descriptive measure for each of the plurality of images based on the number of identified matches.

3. The method of claim 2 wherein sequentially displaying on a display device two or more of the plurality of images in an order dependent on the determined descriptive measures comprises first displaying one of the plurality of images determined to have the highest number of identified matches.

4. The method of claim 1 wherein extracting one or more visual features from each of the plurality of images comprises extracting one or more geometric point, one or more scale-invariant features, one or more scale-invariant feature transform features, or one or more color features from each of the plurality of images.

5. The method of claim 1 wherein extracting one or more visual features from each of the plurality of images comprises extracting one or more color features, and determining a descriptive measure for each of the plurality of images comprises evaluating a quantity of different colors extracted from each of the plurality of images.

6. The method of claim 1, further comprising:

logically associating, in at least one processor-readable medium, the exception object with one of the known objects; and

automatically combining at least some of the visual features extracted from the plurality of images of the exception object with the visual model data for the one of the known objects to update the visual model data.

7. The method of claim 6, further comprising:

receiving, by the at least one processor, identification data for the exception object obtained by an operator via a machine-readable symbol reader operatively coupled to the at least one processor.

8. The method of claim 1, further comprising:

logically associating, in at least one processor-readable medium, the exception object with one of the known objects; and

automatically revising the visual model data for the one of the known objects to include at least some of the extracted visual features of at least one of the plurality of images of the exception object.

9. The method of claim 1 wherein extracting one or more visual features from each of the plurality of images comprises transforming each of the plurality of images into a collection of feature descriptors, each of which is invariant to at least one of image translation, scaling, and rotation.

10. The method of claim 1 wherein comparing the extracted one or more visual features of each of the plurality of visual images with visual model data stored in a database comprises determining whether a logo is present or absent in each of the plurality of images, and determining a descriptive measure for each of the plurality of images comprises determining a descriptive measure for each of the plurality of images based on the determined presence or absence of a logo in each of the plurality of images.

11. The method of claim 1, further comprising:

positioning a plurality of image capture devices proximate the automated machine-readable symbol reader, each of the plurality of image capture devices directed toward a different one of a plurality of views; and

activating each of the plurality of image capture devices to capture the plurality of images at the plurality of views.

12. The method of claim 1 wherein determining a descriptive measure for each of the plurality of images comprises determining, for each of the plurality of images, a descriptive measure indicative of a likelihood that an operator will identify the exception object by viewing the image.

13. The method of claim 12 wherein sequentially displaying on a display device two or more of the plurality of images in an order dependent on the determined descriptive measures comprises first displaying one of the plurality of images determined to have the highest likelihood that an operator will identify the exception object by viewing the image.

14. The method of claim 1 wherein sequentially displaying on a display device two or more of the plurality of images comprises autonomously, and without user intervention, sequentially displaying on the display device two or more of the plurality of images.

15. A method of operation for an automated machine-readable symbol reader system, the method comprising:

receiving a plurality of images of an exception object in at least one processor-readable medium, the exception object having not been successfully identified by a machine-readable symbol reader of the machine-readable symbol reader system;

extracting, by at least one processor, one or more visual features from each of the plurality of received images;

accessing a database of visual model data associated with a plurality of known objects;

identifying matches between the extracted visual features and the visual model data associated with the plurality of known objects;

determining a descriptive measure ranking for each of the plurality of images based on the number of identified matches; and

sequentially displaying on a display device at least two of the plurality of images in an order dependent on the respective ranking of each of the at least two of the plurality of images.

16. An automated machine-readable symbol reader system, comprising:

a machine-readable symbol reader that reads machine-readable symbols associated with objects transported through a view volume;

an exception identification subsystem that detects an exception object that has been transported into or through the view volume without being successfully identified by the machine-readable symbol reader;

an exception handling subsystem that includes;

a display device;

a database of stored visual features data of known objects;

at least one image capture device that obtains a plurality of images of the exception object; and

at least one processor operatively coupled to the display device, the at least one database and the at least one image capture device, the at least one processor;

extracts a plurality of visual features from each of the plurality of images of the exception object;

identifies matches between the extracted visual features and the stored visual features data of known objects;

determines a descriptive measure for each of the plurality of images based on the number of identified matches; and

causes two or more of the plurality of images of the exception object to be displayed sequentially on the display device in an order dependent on the determined descriptive measures.

17. The automated machine-readable symbol reader system of claim 16 wherein the exception handling subsystem further comprises a machine-readable symbol reader that reads a machine-readable symbol of the exception object to resolve an exception associated with the exception object.

18. The automated machine-readable symbol reader system of claim 16 wherein the display device is a handheld display device.

19. The automated machine-readable symbol reader system of claim 16, further comprising:

a conveyor subsystem that transports the object into and through the view volume.

20. The automated machine-readable symbol reader system of claim 19 wherein the at least one image capture device comprises a plurality of image capture devices positioned along the conveyor subsystem at different locations to provide different fields of view.

21. The automated machine-readable symbol reader system of claim 16 wherein the at least one processor:

causes the display device to first display one of the plurality of images determined to have the highest number of identified matches.

22. The automated machine-readable symbol reader system of claim 21 wherein the at least one processor:

determines a descriptive measure ranking for each of the plurality of images based on the number of identified matches; and

causes the display device to sequentially display at least two of the plurality of images in an order dependent on the respective ranking of each of the at least two of the plurality of images.

23. The automated machine-readable symbol reader system of claim 16 wherein the at least one processor:

extracts one or more geometric point features, one or more scale-invariant features, or one or more color features from each of the plurality of images.

24. The automated machine-readable symbol reader system of claim 16 wherein the at least one processor:

logically associates, in at least one processor-readable medium, the exception object with one of the known objects; and

automatically combines at least some of the visual features extracted from the plurality of images of the exception object with the visual features data for the one of the known objects to update the visual features data for the one of the known objects.

25. The automated machine-readable symbol reader system of claim 24, further comprising:

a machine-readable symbol reader operatively coupled to the at least one processor, wherein the at least one processor receives identification data for the exception object obtained by an operator via the machine-readable symbol reader.

26. The automated machine-readable symbol reader system of claim 16 wherein the at least one processor:

logically associates, in at least one processor-readable medium, the exception object with one of the known objects; and

automatically revises the visual features data for the one of the known objects to include at least some of the extracted visual features of at least one of the plurality of images of the exception object.

27. The automated machine-readable symbol reader system of claim 16 wherein the at least one processor:

transforms each of the plurality of images into a collection of feature descriptors, each of which is invariant to at least one of image translation, scaling, and rotation.

28. The automated machine-readable symbol reader system of claim 16 wherein the at least one processor:

determines whether a logo is present or absent in each of the plurality of images; and

determines a descriptive measure for each of the plurality of images based on the determined presence or absence of a logo in each of the plurality of images.

29. The automated machine-readable symbol reader system of claim 16 wherein the at least one processor:

determines, for each of the plurality of images, a descriptive measure indicative of a likelihood that an operator will identify the exception object by viewing the image.

30. The automated machine-readable symbol reader system of claim 29 wherein the at least one processor:

causes the display device to first display one of the plurality of images determined to have the highest likelihood that an operator will identify the exception object by viewing the image.

31. An automated machine-readable symbol reader system, comprising:

a machine-readable symbol reader that reads machine-readable symbols associated with objects transported through a view volume;

an exception identification subsystem that detects an exception object that has been transported into or through the view volume without being successfully identified by the machine-readable symbol reader;

an exception handling subsystem that includes;

a display device;

a database of stored visual features data of known objects;

at least one image capture device that obtains a plurality of images of the exception object; and

at least one processor operatively coupled to the display device, the at least one database and the at least one image capture device, the at least one processor;

extracts a plurality of visual features from each of the plurality of images of the exception object;

identifies matches between the extracted visual features and the stored visual features data of known objects to determine whether a logo is present or absent in each of the plurality of images;

determines a descriptive measure for each of the plurality of images based on the determined presence or absence of a logo in each of the plurality of images; and

causes at least one of the plurality of images of the exception object to be displayed on the display device based on the determined descriptive measure.

32. The automated machine-readable symbol reader system of claim 31, further comprising:

a conveyor subsystem that transports the object into and through the view volume,

wherein the at least one image capture device comprises a plurality of image capture devices positioned along the conveyor subsystem at different locations to provide different fields of view.

33. The automated machine-readable symbol reader system of claim 31 wherein the at least one processor:

extracts one or more geometric point features, one or more scale-invariant features, or one or more color features from each of the plurality of images.

34. The automated machine-readable symbol reader system of claim 31 wherein the exception handling subsystem further comprises a machine-readable symbol reader that reads a machine-readable symbol of the exception object to resolve an exception associated with the exception object.

35. The automated machine-readable symbol reader system of claim 31 wherein the display device is a handheld display device.