Image-based inventory control system using advanced image recognition

• US 9,922,413 B2
• Filed: 08/13/2015
• Issued: 03/20/2018
• Est. Priority Date: 08/08/2008
• Status: Active Grant

• Current Assignee: Snap-On Incorporated
• Original Assignee: Snap-On Incorporated
• Inventors: Glickman, Stephen L., Jackson, David A., Lipsey, Matthew J., Phillips, Preston, Rogers, Frederick J.
• Primary Examiner(s): Mikels, Matthew
• Application Number:
  US14/826,106
• Publication Number:
  US 20160098831A1
• Time in Patent Office: 950 Days
• Field of Search: 235375, 235385
• CPC Class Codes: B25H 3/00 Storage means or arrangements for workshops facilitating access to, or handling of, work tools or instruments containers or packages with special means for dispensing contents B65D83/00; nail dispensers B25C3/00
• CPC Class Codes: B25H 3/028 by sliding extraction from within a common frame
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06T 2207/30108 Industrial image inspection
• CPC Class Codes: G06T 7/0008 checking presence/absence
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/26 Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 20/66 Trinkets, e.g. shirt buttons or jewellery items recognising microscopic objects G06V20/69
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

Systems for monitoring an inventory condition of objects based on captured images are described. An exemplary system includes at least one storage drawer, each storage drawer including a plurality of storage locations for storing objects; and a data storage device storing reference data for each storage drawer. The reference data includes, for each region of interest corresponding to each respective storage drawer, at least one of a reference image signature representing attributes of the region of interest when a stored object is present, and a reference image signature representing attributes of the region of interest when the stored object is not present. A data processor of the system is configured to derive an image signature representing attributes of a region of interest corresponding to a drawer using a captured image of the drawer; and determine an inventory condition of the drawer according to the stored reference data corresponding to the region of interest and the derived image signature representing attributes of the captured image of the region of interest.

Claims

1. An inventory control system for determining an inventory condition of objects stored in the system, the system comprising:

a plurality of storage drawers, each storage drawer including a plurality of storage locations for storing objects;

an image sensing device configured to capture images of each of the storage drawers;

a data storage device storing reference data for each of a plurality of objects, wherein the reference data includes, for each respective object, an image signature comprising a pixel number distribution representing one or more attributes of an image of the respective object; and

a data processor configured to;

derive an image signature representing attributes of a region of interest of a drawer of the plurality of drawers using an image of the drawer captured by the image sensing device;

compare the derived image signature representing attributes of the region of interest of the drawer to the image signature of the reference data comprising the pixel number distribution for each of the plurality of objects stored in the data storage device; and

determine the identity of an object stored in the region of interest of the drawer based on a result of the comparison of the derived image signature to the stored image signature of the reference data comprising the pixel number distribution for each of the plurality of objects.

2. The inventory control system of claim 1, wherein the data storage device further stores reference data for each storage drawer, wherein the reference data includes, for each respective storage drawer, at least one of a reference image signature representing attributes of the storage drawer when a stored object is present and a reference image signature representing attributes of the storage drawer when the stored object is not present, and

the data processor is configured to determine an inventory condition of the drawer according to the stored reference data corresponding to the drawer and the derived image signature representing attributes of the captured image of the region of interest of the drawer.

3. The inventory control system of claim 2, wherein the data processor is configured to:

determine that an incorrect object is located in the region of interest of the drawer based on a result of a comparison of the derived image signature to the stored reference data corresponding to the drawer; and

in response to determining that the incorrect object is located in the region of interest of the drawer, generate an alert to a user or system administrator of the inventory control system.

4. The inventory control system of claim 3, wherein the alert generated by the data processor includes the determined identity of the object stored in the region of interest of the drawer.

5. The inventory control system of claim 1, wherein the deriving of the image signature comprises deriving a pixel number distribution with respect to one or more attributes of the region of interest using the captured image of the drawer.

6. The inventory control system of claim 5, wherein the one or more attributes with respect to which the pixel number distribution is derived are selected from among intensity, hue, and saturation values.

7. The inventory control system of claim 5, wherein the pixel number distribution includes a histogram.

8. The inventory control system of claim 1, wherein the pixel number distribution for each of the plurality of objects uniquely characterizes the one or more attributes of the image of the object.

9. The inventory control system of claim 8, wherein the one or more attributes with respect to which the pixel number distribution is derived are selected from among intensity, hue, and saturation values.

10. The inventory control system of claim 8, wherein the pixel number distribution includes a histogram.

11. A method for using in an inventory control system for determining an inventory condition of objects stored in the system, the system comprising a plurality of storage drawers, each storage drawer including a plurality of storage locations for storing objects, an image sensing device configured to capture images of each of the storage drawers, a data storage device storing reference data for each of a plurality of objects, wherein the reference data includes, for each respective object, an image signature comprising a pixel number distribution representing one or more attributes of an image of the respective object,

the method comprising:

deriving an image signature representing attributes of a region of interest of a drawer of the plurality of drawers using an image of the drawer captured by the image sensing device;

comparing the derived image signature representing attributes of the region of interest of the drawer to the image signature of the reference data comprising the pixel number distribution for each of the plurality of objects stored in the data storage device; and

determining the identity of an object stored in the region of interest of the drawer based on a result of the comparison of the derived image signature to the reference data for each of the plurality of objects.

12. The method of claim 11, wherein the data storage device further stores reference data for each storage drawer, wherein the reference data includes, for each respective storage drawer, at least one of a reference image signature representing attributes of the storage drawer when a stored object is present and a reference image signature representing attributes of the storage drawer when the stored object is not present, and

the method further comprising:

determining an inventory condition of the drawer according to the stored reference data corresponding to the drawer and the derived image signature representing attributes of the captured image of the region of interest of the drawer.

13. The method of claim 12, further comprising:

determining that an incorrect object is located in the region of interest of the drawer based on a result of a comparison of the derived image signature to the stored reference data corresponding to the drawer; and

in response to determining that the incorrect object is located in the region of interest of the drawer, generating an alert to a user or system administrator of the inventory control system.

14. The method of claim 13, wherein the alert generated by the data processor includes the determined identity of the object stored in the region of interest of the drawer.

15. The method of claim 11, wherein the deriving of the image signature comprises deriving a pixel number distribution with respect to one or more attributes of the region of interest using the captured image of the drawer.

16. The method of claim 15, wherein the one or more attributes with respect to which the pixel number distribution is derived are selected from among intensity, hue, and saturation values.

17. The method of claim 15, wherein the pixel number distribution includes a histogram.

18. The method of claim 11, wherein the pixel number distribution for each of the plurality of objects uniquely characterizes the one or more attributes of the image of the object.

19. The method of claim 18, wherein the one or more attributes with respect to which the pixel number distribution is derived are selected from among intensity, hue, and saturation values.

20. The method of claim 18, wherein the pixel number distribution includes a histogram.