Self checkout with visual recognition
First Claim
1. An automated checkout system for point-of-sale, comprisinga data reader section including an optical code reader having a read volume and configured to read an optical code on an item being passed through the read volume and generate a product identifier of the item;
- a collection section within which items read by the optical code reader are collected after the items have been passed through the read volume;
at least one camera disposed with a field of view of the collection section for capturing one or more images of the item within the collection section;
a database of features and images of known objects;
an image processor configured toa) extract a plurality of visual features from the one or more images of the item,b) identify matches between the extracted visual features and the features of known objects,c) generate a geometric transform between the extracted visual features and the features of known objects for a subset of known objects corresponding to the matches, andd) identify one of the known objects based on a best match of the geometric transform; and
a transaction processor configured to execute at least one of a predetermined set of actions if the known object that has been identified is different than the item corresponding to the product identifier.
2 Assignments
0 Petitions
Accused Products
Abstract
A system and method is disclosed for using object recognition/verification and weight information to confirm the accuracy of a UPC scan, or to provide an affirmative recognition where no UPC scan was made. In the preferred embodiment, the checkout system comprises: a universal product code (UPC) scanner configured to generate a product identifier; at least one camera for capturing one or more image of an item; a database of features and images of known objects; an image processor configured to: extract a plurality of geometric point features from the one or more images; identify matches between the extracted geometric point features and the features of known objects; generate a geometric transform between the extracted geometric point features and the features of known objects for a subset of known objects corresponding to matches; and identify one of the known objects based on a best match of the geometric transform; and a transaction processor configured to execute one of a predetermined set of actions if the identified object is different than the product identifier.
65 Citations
19 Claims
-
1. An automated checkout system for point-of-sale, comprising
a data reader section including an optical code reader having a read volume and configured to read an optical code on an item being passed through the read volume and generate a product identifier of the item; -
a collection section within which items read by the optical code reader are collected after the items have been passed through the read volume; at least one camera disposed with a field of view of the collection section for capturing one or more images of the item within the collection section; a database of features and images of known objects; an image processor configured to a) extract a plurality of visual features from the one or more images of the item, b) identify matches between the extracted visual features and the features of known objects, c) generate a geometric transform between the extracted visual features and the features of known objects for a subset of known objects corresponding to the matches, and d) identify one of the known objects based on a best match of the geometric transform; and a transaction processor configured to execute at least one of a predetermined set of actions if the known object that has been identified is different than the item corresponding to the product identifier. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A method of checkout at point of sale station, the station having (1) a data reader section including an optical code reader with a read volume and configured to read an optical code on an item being passed through the read volume and generate a product identifier of the item and (2) a collection section within which items read by the optical code reader are collected after having been passed them through the read volume, the method comprising the steps of
passing an item bearing an optical code through the read volume of the optical code reader within the data reader section; -
reading the optical code with the optical code reader, the optical code reader generating a product identifier of the item; transferring the item into the collection section; by means of at least one camera disposed with a field of view of the collection section, capturing one or more images of the item that has been transferred into the collection section; and by means of a processor, (a) accessing a database of features and images of known objects, (b) extracting a plurality of visual features from the one or more images of the item, (c) identifying matches between the extracted visual features and the features of known objects, (d) generating a geometric transform between the extracted visual features and the features of known objects for a subset of known objects corresponding to the matches, (e) identifying one of the known objects based on a best match of the geometric transform; and executing one of a predetermined set of actions if the known object that has been identified from the extracted visual features is different than the item corresponding to the product identifier. - View Dependent Claims (11, 12, 13, 14, 15, 16, 17)
-
-
18. A method of checkout at point of sale station, the station having (1) a data reader section including an optical code reader with a read volume and configured to read an optical code on an item being passed through the read volume and generate a product identifier of the item and (2) a downstream collection section within which items read by the optical code reader are collected after having been passed through the read volume, the method comprising the steps of
via the optical code reader, identifying items by attempting to read the optical code on each item as it is passed through the read volume; -
moving the item into the collection section; by means of at least one camera disposed with a field of view of the collection section, capturing one or more images of the item moved into the collection section; by means of a processor, (a) accessing a database of features and images of known objects, (b) extracting a plurality of visual features from the one or more images of the item, (c) identifying matches between the extracted visual features and the features of known objects, (d) generating a geometric transform between the extracted visual features and the features of known objects for a subset of known objects corresponding to the matches, (e) identifying one of the known objects based on a best match of the geometric transform; determining whether the known object identified in the collection section does not correspond to any item having been identified by the optical code reader in a current transaction; if the known object identified in the collection section is determined to not correspond to any item having been identified by the optical code reader in a current transaction, taking a remedial action selected from the group consisting of;
adding the known object identified to a list of items being purchased, and inquiring of a user or operator whether the known object identified is desired to be added the list of items being purchased. - View Dependent Claims (19)
-
Specification