System and method for performing production line product identification

• US 10,303,987 B2
• Filed: 04/19/2018
• Issued: 05/28/2019
• Est. Priority Date: 04/26/2017
• Status: Active Grant

• Current Assignee: Sensors, Inc.
• Original Assignee: Sensors, Inc.
• Inventors: Kotula, David J.
• Primary Examiner(s): Randazzo, Thomas
• Application Number:
  US15/956,813
• Publication Number:
  US 20180314867A1
• Time in Patent Office: 404 Days
• CPC Class Codes: B65G 2201/0235 Containers
• CPC Class Codes: B65G 2203/041 Camera
• CPC Class Codes: B65G 47/34 Devices for discharging articles or materials from conveyor B65G47/256 takes precedence
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 2218/08 Feature extraction
• CPC Class Codes: G06K 7/10445 the record carriers being fixed to further objects, e.g. RFIDs fixed to packages, luggage, mail-pieces or work-pieces transported on a conveyor belt
• CPC Class Codes: G06K 7/10722 Photodetector array or CCD scanning
• CPC Class Codes: G06K 7/10861 sensing of data fields affixed to objects or articles, e.g. coded labels postal sorting B07C3/14, conveying articles B65G47/48
• CPC Class Codes: G06K 7/1413 1D bar codes
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30128 Food products
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/761 Proximity, similarity or dissimilarity measures
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 20/68 Food, e.g. fruit or vegetables
• CPC Class Codes: G06V 2201/06 : Recognition of objects for industrial automation
• CPC Class Codes: G06V 30/242 : Division of the character sequences into groups prior to recognition; Selection of dictionaries

Abstract

In an illustrative embodiment, a system for identifying products on a production line includes image capturing devices that acquire images of containers moving along a production line at an inspection location. The system also includes a rejection device and a controller that configures the image capturing devices for image acquisition based on properties of the containers, identifies a product associated with each of the containers based on a portion of a product identification code and a portion of additional features detected in the images, and determines whether the identified product matches predetermined properties or characteristics, resulting in a pass result, otherwise a non-pass result occurs. When a non-pass result occurs, the controller outputs a signal to actuate the rejection device that removes the container from the production line.

Claims

1. A system comprising:

a plurality of image capturing devices situated near an inspection location on a continuously moving production line, wherein each of the plurality of image capturing devices is oriented to acquire images of a plurality of items moving along the production line from a different orientation angle;

a rejection device; and

a controller communicatively coupled to the plurality of image capturing devices and the rejection device, wherein the controller is configured to, for each item of the plurality of items,

identify, from a plurality of images of the respective item, the plurality of images comprising at least one image captured from each of the plurality of image capturing devices, a subset of the plurality of images including at least one of a portion of a product identification code and a portion of additional features of the respective item,

assess quality of each image of the subset of images to identify at least one candidate image portraying one or more features useful for positively identifying the respective item in further analysis, the one or more features including the at least one of the portion of the product identification code and the portion of the additional features of the respective item,

apply at least one matching algorithm to the at least one candidate image to determine whether the at least one of the portion of the product identification code and the portion of the additional features matches product information corresponding to the item, and

responsive to application of the at least one matching algorithm resulting in no matching product information, cause actuation of the rejection device to remove the item from the production line.

2. The system of claim 1, wherein assessing the quality of each image of the subset of images comprises, after identifying a first candidate image of the at least one candidate image, ceasing to assess the quality of further images of the subset of images, wherein applying the at least one matching algorithm comprises applying the at least one matching algorithm to the first candidate image.

3. The system of claim 1, wherein the plurality of image capturing devices are arranged surrounding the inspection location to obtain images from all vertical sides of each of the plurality of items.

4. The system of claim 1, wherein causing actuation of the rejection device comprises causing actuation of the rejection device to divert the respective item onto a rejection conveyor.

5. The system of claim 1, wherein assessing quality comprises determining a quality score for each of the subset of images of the respective item, wherein each image of the at least one candidate image has a respective quality score at or above a threshold value.

6. The system of claim 5, wherein applying the at least one matching algorithm comprises applying pattern matching to a highest quality image of the respective item having a highest quality score of the subset of images of the respective item.

7. The system of claim 6, wherein applying pattern matching comprises applying the at least one of the portion of the product identification code and the portion of the additional features to a plurality of packaging patterns stored in a non-transitory computer readable memory of the controller, wherein each of the plurality of packaging patterns is associated with a type of item processed on the production line.

8. The system of claim 7, wherein the plurality of packaging patterns comprises one or more templates for labels associated with the types of items processed on the production line, wherein each template of the one or more templates includes at least one of a respective product identification code, a respective brand logo, a respective product logo, a respective ingredient list, respective nutrition information, and respective quantity information.

9. The system of claim 1, wherein the controller is further configured to automatically adjust one or more settings of at least one of the plurality of image capturing devices based on at least one of item size, item shape, label dimensions, and label distribution of the plurality of items.

10. The system of claim 9, wherein automatically adjusting the one or more settings comprises deactivating at least one of the plurality of image capturing devices.

11. The system of claim 9, wherein the one or more settings of the at least one of the plurality of image capturing devices comprises at least one of an orientation aspect angle and a position.

12. A method, comprising:

determining, by a controller, a type of container expected on a continuous production line;

automatically adjusting, by the controller, one or more settings of each image capturing device of one or more of a plurality of image capturing devices to configure each of the one or more image capturing devices to acquire images from a different respective orientation angle, such that a 360-degree rotational range of features of each of a plurality of containers is captured by the plurality of image capturing devices, wherein

the one or more settings of each image capturing device of the one or more image capturing devices are based at least in part on the type of container;

acquiring, with the plurality of image capturing devices, images of the plurality of containers moving along the continuous production line;

by the controller for a given container of the plurality of containers,

applying a first extraction algorithm configured to extract a machine-readable information code from at least one of the plurality of images of the respective container acquired by the plurality of image capturing devices;

responsive to determining the first extraction algorithm failed to identify the machine-readable information code, applying a second extraction algorithm configured to extract a feature pattern from each of the plurality of images of the respective container;

applying a matching algorithm to the feature pattern of at least one of the plurality of images to match the feature pattern to product information; and

responsive to failing to identify a match by the matching algorithm, causing actuation of a rejection device to remove the respective container from the continuous production line.

13. The method of claim 12, wherein automatically adjusting the one or more settings of the one or more image capturing devices comprises automatically adjusting a position of a first image capturing device of the one or more image capturing devices.

14. The method of claim 12, further comprising:

determining, with the controller, a quality score for each of the plurality of images, wherein each respective quality score provides an indication a quality of at least one of the machine-readable information code and the feature pattern.

15. The method of claim 12, wherein applying the matching algorithm comprises matching at least one of a component alignment and a shape pattern with one or more pattern templates representative of product labeling.

16. The method of claim 12, further comprising, responsive to failing to identify a match by the matching algorithm, issuing, by the controller, an alarm.

17. The method of claim 12, further comprising, responsive to failing to identify a match:

determining, by the controller, whether a predetermined number of product mismatches have been identified; and

responsive to determining the predetermined number has been identified, causing, by the controller, a conveyor of the continuous production line to halt.

18. The method of claim 12, wherein automatically adjusting the one or more settings of the one or more image capturing devices comprises determining a number of selected image capturing devices of the plurality of image capturing devices based on at least one of a packaging shape of the plurality of containers, a packaging size of the plurality of containers, and label characteristics of the plurality of containers.

19. The method of claim 18, wherein automatically adjusting the one or more settings of the one or more image capturing devices comprises at least one of activating one or more of the number of selected image capturing devices and deactivating one or more unselected image capturing devices of the plurality of image capturing devices.

20. The method of claim 12, wherein each container of at least a portion of the plurality of containers has a cylindrical shape.