System and methods for merchandise checkout

• US 7,100,824 B2
• Filed: 12/27/2004
• Issued: 09/05/2006
• Est. Priority Date: 02/27/2004
• Status: Active Grant

• Current Assignee: Datalogic Adc, Inc. (Datalogic S.p.A.)
• Original Assignee: Evolution Robotics Inc. (iRobot Corporation)
• Inventors: Cremean, Michael, Hudnut, Alec, Goncalves, Luis, Ostrowski, Jim, Simonini, Alex
• Primary Examiner(s): Le, Thien M.
• Assistant Examiner(s): Kim, Tae W.
• Application Number:
  US11/023,004
• Publication Number:
  US 20050189411A1
• Time in Patent Office: 617 Days
• Field of Search: 382/100, 382/103, 186/61, 186/66, 235/383, 235/462.12, 235/462.06
• US Class Current: 235/383
• CPC Class Codes: A47F 9/045 Handling of baskets or shopping trolleys at check-out counters, e.g. unloading, checking
• CPC Class Codes: G07F 7/02 by keys or other credit registering devices
• CPC Class Codes: G07G 1/0036 Checkout procedures
• CPC Class Codes: G07G 1/0063 with means for detecting the geometric dimensions of the article of which the code is read, such as its size or height, for the verification of the registration
• CPC Class Codes: G07G 1/0081 the reader being a portable scanner or data reader G07G1/009 takes precedence
• CPC Class Codes: G07G 3/00 Alarm indicators, e.g. bells
• CPC Class Codes: G07G 3/003 Anti-theft control theft alarms G08B13/00
• CPC Class Codes: G08B 13/1961 Movement detection not involving frame subtraction, e.g. motion detection on the basis of luminance changes in the image
• CPC Class Codes: G08B 13/19671 Addition of non-video data, i.e. metadata, to video stream

Abstract

Systems and methods for recognizing and identifying items located on the lower shelf of a shopping cart in a checkout lane of a retail store environment for the purpose of reducing or preventing loss or fraud and increasing the efficiency of a checkout process. The system includes one or more visual sensors that can take images of items and a computer system that receives the images from the one or more visual sensors and automatically identifies the items. The system can be trained to recognize the items using images taken of the items. The system relies on matching visual features from training images to match against features extracted from images taken at the checkout lane. Using the scale-invariant feature transformation (SIFT) method, for example, the system can compare the visual features of the images to the features stored in a database to find one or more matches, where the found one or more matches are used to identify the items.

Claims

1. A method of checking out merchandise, comprising:

(a) receiving visual image data of an object in a cart;

(b) analyzing the visual image data to extract one or more visual features;

(c) comparing the visual image data of the object and cart with data stored in a database to find a set of matches wherein the data comprises a plurality of scale-invariant features associated with a plurality of known objects, wherein the step of comparing is based on a scale invariant feature transform (SIFT) method;

(d) determining if the set of matches is found; and

(e) sending a recognition alert.

2. The method of claim 1, wherein the step of comparing comprises:

finding a match for each of the one or more features.

3. The method of claim 2, wherein the step of comparing further comprises:

associating a quality measure with the match.

4. The method of claim 3, wherein the step of comparing further comprises:

if the associated quality measure exceeds a predetermined threshold, including the match in the set of matches.

5. The method of claim 3, wherein the quality measure is a match confidence that ranges from 0 to 100%.

6. The method of claim 3, wherein the step of comparing further comprises:

selecting a particular match associated with a highest quality measure.

7. The method of claim 6, wherein the step of comparing comprises a step of including the particular match in the set of matches.

8. The method of claim 1, further comprising, prior to the step of sending a recognition alert:

computing a statistical probability that each of the one or more visual features can be recognized.

9. The method of claim 1, further comprising, prior to the step of sending a recognition alert:

(e) checking if each element of the set of matches is reliable.

10. The method of claim 9, further comprising:

(f) if all elements of the set of matches are unreliable, repeating the steps (a)–

(e).

11. The method of claim 9, wherein the step of checking comprises:

recognizing each element of the set of matches for a plurality of process cycles.

12. The method of claim 9, wherein the step of receiving an image comprises:

capturing a plurality of images.

13. The method of claim 12, wherein the step of receiving an image further comprises:

comparing two consecutive ones of the plurality of images to detect a motion; and

if the motion is detected, taking later one of the two consecutive images.

14. A system for checking out a merchandise, comprising:

at least one visual sensor for capturing an image of a portion of an object on a movable structure; and

a subsystem coupled to the at least one visual sensor and configured to detect and recognize the object by analyzing the image of the portion of the object and movable structure;

wherein the subsystem is adapted to identify scale-invariant features using a scale-invariant feature transform (SIFT) applied to said image of the object and moveable structure.

15. The system of claim 14, wherein the moveable structure comprises a shopping cart.