Systems and methods for merchandise checkout

• US 20060283943A1
• Filed: 08/22/2006
• Published: 12/21/2006
• Est. Priority Date: 02/27/2004
• Status: Active Grant

• Current Assignee: Datalogic Adc, Inc. (Datalogic S.p.A.)
• Original Assignee: Evolution Robotics Retail Incorporated (Datalogic ADC Incorporated)
• Inventors: Cremean, Micheal, Hudnut, Alec, Goncalves, Luis, Ostrowski, Jim, Simonini, Alex
• Granted Patent: US 8,267,316 B2
• Time in Patent Office: Days
• US Class Current: 235/383
• CPC Class Codes: A47F 9/045 Handling of baskets or shopping trolleys at check-out counters, e.g. unloading, checking
• CPC Class Codes: G07F 7/02 by keys or other credit registering devices
• CPC Class Codes: G07G 1/0036 Checkout procedures
• CPC Class Codes: G07G 1/0063 with means for detecting the geometric dimensions of the article of which the code is read, such as its size or height, for the verification of the registration
• CPC Class Codes: G07G 1/0081 the reader being a portable scanner or data reader G07G1/009 takes precedence
• CPC Class Codes: G07G 3/00 Alarm indicators, e.g. bells
• CPC Class Codes: G07G 3/003 Anti-theft control theft alarms G08B13/00
• CPC Class Codes: G08B 13/1961 Movement detection not involving frame subtraction, e.g. motion detection on the basis of luminance changes in the image
• CPC Class Codes: G08B 13/19671 Addition of non-video data, i.e. metadata, to video stream

Abstract

Systems and methods for recognizing and identifying items located on the lower shelf of a shopping cart in a checkout lane of a retail store environment for the purpose of reducing or preventing loss or fraud and increasing the efficiency of a checkout process. The system includes one or more visual sensors that can take images of items and a computer system that receives the images from the one or more visual sensors and automatically identifies the items. The system can be trained to recognize the items using images taken of the items. The system relies on matching visual features from training images to match against features extracted from images taken at the checkout lane. Using the scale-invariant feature transformation (SIFT) method, for example, the system can compare the visual features of the images to the features stored in a database to find one or more matches, where the found one or more matches are used to identify the items.

Claims

1. A system for checking out merchandise, comprising:

at least one visual sensor for capturing an image of an object on a moveable structure; and

a subsystem coupled to the at least one visual sensor, wherein the subsystem is configured to;

extract one or more visual features from the image of the object on the moveable structure;

compare the one or more extracted visual features to a plurality of visual features associated with a plurality of known objects, and

identify matching visual features to find a match between the object on the moveable structure and one of the plurality of known objects.

2. The system of claim 1, wherein the at least one visual sensor is a digital camera with a charge-coupled-device (CCD) imager, a complementary metal-oxide semiconductor (CMOS) imager, an infrared imager, or any combination thereof.

3. The system of claim 1, wherein the subsystem comprises:

a checkout subsystem configured to receive visual data from the at least one visual sensor;

a server configured to receive visual data from the checkout subsystem, recognize the object, and send match data to the checkout subsystem; and

an object database configured to store the plurality of visual features associated with the plurality of known objects.

4. The system of claim 3, wherein the object database is spread over a plurality of storage devices connected via a network.

5. The system of claim 3, wherein the checkout subsystem is coupled to one or more input devices, each of the one or more input devices including a barcode scanner, a scale, a keyboard, a keypad, a touch screen, a card reader or any combination thereof.

6. The system of claim 3, wherein the checkout subsystem comprises a checkout terminal used by a cashier or a self-service checkout terminal.

7. The system of claim 1, wherein the one or more extracted visual features, the plurality of visual features associated with the plurality of known objects, and the matching visual features common to the one or more extracted visual features and plurality of visual features associated with the plurality of known objects are scale-invariant feature transform (SIFT) features.

8. The system of claim 1, wherein the moveable structure comprises a cart.

9. The system of claim 8, wherein the cart comprises a shopping cart.

10. The system of claim 9, wherein the shopping cart comprises a bottom of basket, and wherein the at least one visual sensor comprises one or more cameras directed to the bottom of basket when the cart is in a checkout lane.

11. The system of claim 1, wherein the at least one visual sensor comprises one or more cameras directed to a checkout lane.

12. A system for checking out merchandise, comprising:

at least one visual sensor for capturing an image of an object in a moveable structure;

a checkout subsystem adapted to receive visual data from the at least one visual sensor and analyze the visual data with a scale invariant feature transform (SIFT);

a server adapted to receive the analyzed visual data comprising one or more SIFT features from the checkout system, recognize the object from among a plurality of known objects based on the SIFT features, and send match data to the checkout subsystem.

13. The system of claim 12, wherein the checkout subsystem is coupled to one or more input devices, each of the one or more input devices including a barcode scanner, a scale, a keyboard, a keypad, a touch screen, a card reader or any combination thereof, and wherein the checkout subsystem is a checkout terminal used by a cashier or a self-service checkout terminal.

14. The system of claim 12, wherein the checkout subsystem is connected to a checkout terminal operable by a cashier.

15. The system of claim 14, wherein the server is adapted to transmit information of the recognized object to the checkout terminal for display to the cashier.

16. The system of claim 14, wherein the checkout subsystem is adapted to automatically transmit a price of the recognized object to the checkout terminal.

17. The system of claim 14, wherein the checkout subsystem is adapted to add a price of the recognized object to a list of merchandise being purchased.

18. A system for checking out merchandise in a shopping cart, comprising:

at least one visual sensor for capturing an image of the merchandise in the shopping cart;

a checkout subsystem adapted to receive visual data from the at least one visual sensor; and

a server adapted to receive the visual data from the checkout system, extract one or more scale-invariant feature transform (SIFT) features from the image of the merchandise in the cart, recognize the merchandise based on the SIFT features, and send match data to the checkout subsystem.

19. A computer readable medium in a merchandise checkout system embodying program code with instructions for recognizing an object, said computer readable medium comprising:

program code for receiving a visual image data of the an object on a cart, the visual data comprising one or more scale-invariant feature transform (SIFT) features of the object;

program code for comparing the SIFT features of the object with SIFT features of a plurality of known objects to find a set of matches;

program code for identifying the object on the cart as one of the plurality of known object based on the set of matches; and

program code for sending a recognition alert to a checkout terminal.

20. A system for checking out merchandise, comprising:

at least one visual sensor for capturing an image of one or more objects on a cart; and

a subsystem coupled to the at least one visual sensor and configured to detect and recognize the object from a plurality of known objects by analyzing the image of the one or more objects and cart using a scale-invariant feature transform (SIFT) to extract visual features from the image of the one or more objects on the cart.

21. A system for checking out merchandise, comprising:

at least one visual sensor for capturing an image of at least a portion of one or more objects on a moveable structure;

a database of known scale-invariant features associated with a plurality of known objects; and

a subsystem coupled to the at least one visual sensor, wherein the subsystem is adapted to;

detect at least one scale-invariant feature for each of the one or more objects from the image of the objects on the moveable structure; and

identify each of the one or more objects from the image by matching the at least one detected scale-invariant feature with the scale-invariant features associated with the plurality of known objects from the database.

22. A method for checking out merchandise, the method comprising:

capturing an image of one or more objects on a moveable structure;

extracting at least one scale-invariant feature from the image of the objects on the moveable structure; and

comparing the at least one extracted scale-invariant feature to a plurality of known scale-invariant features associated with a plurality of known objects;

identifying one or more matches between the at least one extracted scale-invariant feature and the plurality of known scale-invariant features; and

identifying each of the one or more objects on the moveable structure based on the one or more matches.

23. The method of claim 22, wherein the moveable structure is a shopping cart.

24. The method of claim 22, wherein the at least one extracted scale-invariant feature is a scale-invariant feature transform (SIFT) feature.