Optical flow for object recognition

• US 20060147087A1
• Filed: 01/03/2006
• Published: 07/06/2006
• Est. Priority Date: 01/04/2005
• Status: Active Grant

• Current Assignee: Datalogic Adc, Inc. (Datalogic S.p.A.)
• Original Assignee: Evolution Robotics Retail Incorporated (Datalogic ADC Incorporated)
• Inventors: Goncalves, Luis, Gremmer, Isaac
• Granted Patent: US 7,646,887 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 20/68 Food, e.g. fruit or vegetables

Abstract

Disclosed is a system and method for using optical flow to detect objects moving past a camera and to select images of the moving objects. A shopping cart, for example, may be detected by subdividing an image into a plurality of image blocks; comparing the blocks to a preceding image to determine the motion of the portion of the object pictured; associating the most common motion with the shopping cart. The motion of the cart may also be integrated over time for purposes of tracking cart motion and selecting a subset of the captured images for object recognition processing. Detection of the cart and image selection improves computational efficiency an increase merchandise throughput.

Claims

1. A method for determining optical flow of a region of interest, the method comprising the steps of:

capturing a plurality of images of one or more objects on a moveable structure, the plurality of images comprising a first image and a second image;

partitioning the second image into a plurality of data blocks;

comparing a set of one or more of said blocks with at least a portion of the first image;

determining the displacement between the first image and block for each of the blocks in which a match is detected;

identifying the non-zero displacement common to the most blocks; and

associating the most common displacement with the moveable structure and one or more objects.

2. The method of claim 1, wherein the movable structure is a shopping cart.

3. The method of claim 2, wherein one or more of the steps are executed by a processor operatively coupled to a checkout terminal.

4. The method of claim 1, wherein the plurality of images are captured by a video camera.

5. The method of claim 4, wherein the video camera is directed to a cart bottom of basket.

6. The method claim 1, wherein the second image is down-sampled prior to the partitioning step.

7. The method of claim 2, wherein the step of comparing comprises the step correlating the set of blocks with an associated portion of the first image.

8. The method of claim 7, wherein the associated portion of the first image is a search field.

9. The method of claim 8, wherein the search field has a width sized to capture substantially all horizontal motion of the cart.

10. The method of claim 9, wherein the search field has a height sized to exclude substantially all human motion.

11. The method of claim 1, further comprising the step of determining a measure of texture in each block after the step of partitioning.

12. The method of claim 11, wherein the set of blocks consists of those blocks for which the measure of texture exceeds a first threshold.

13. The method claim 1, wherein the method further comprises the step of generating a histogram for binning the displacement of each of the set of blocks.

14. The method of claim 1, further comprising the step of identifying the one or more objects from the blocks associated with the most common displacement.

15. The method of claim 14, wherein the step of identifying the one or more objects comprises:

extracting one or more visual features from the blocks associated with the most common displacement; and

comparing the one or more visual features to a database of known objects.

16. The method of claim 15, wherein the visual features are scale-invariant feature transformation (SIFT) features.

17. A system for measuring optical flow of a region of interest, the system comprising:

a visual sensor for capturing a plurality of images of one or more objects on a moveable structure, the plurality of images comprising a first image and a second image;

a processor adapted to;

a) partition the second image into a plurality of blocks;

b) compare a set of one or more of said blocks with at least a portion of the first image;

c) determine a displacement between the first image and block for each of the blocks in which a match is detected;

d) identify the non-zero displacement common to a set of blocks; and

e) identifying one or more of the objects from the blocks associated with the most common displacement.

18. The system of claim 17, wherein the identified non-zero displacement is common to the greatest number of blocks of the second image.

19. A method of selecting images of interest, the method comprising:

capturing a plurality of images of one or more objects on a moveable structure;

partitioning each of the plurality of images into a plurality of blocks;

generating a displacement measurement for one or more of the blocks based on at least one preceding image;

identifying, for each image, the displacement measurement common to the most blocks;

adding the identified displacement measurement to a cumulative displacement measurement; and

selecting each image for which the cumulative displacement measurement exceeds a predetermined value with respect to a previously selected image.

20. The method of claim 19, wherein the predetermined value corresponds to approximately one third a field of view of a camera for capturing the plurality of images.

21. The method of claim 19, further comprising the step of identifying the one or more objects from the selected images.

22. The method of claim 21, wherein the step of identifying the one or more objects from the identified images is based on scale-invariant feature transformation (SIFT) features.

23. The method of claim 19, further comprising the step of down-sample each of the plurality of images prior to the partitioning step.

24. The method of claim 23, further comprising the step of identifying the one or more objects from the captured images associated with the selected images.

25. A method of selecting images of interest, the method comprising:

a visual sensor for capturing a plurality of images of one or more objects on a moveable structure;

a processor adapted to;

a) partition each of the plurality of images into a plurality of blocks;

b) generate a displacement measurement for one or more of the blocks based on at least one preceding image;

c) identify, for each image, the displacement measurement common to the most blocks;

d) add the identified displacement measurement to a cumulative displacement measurement; and

e) select each image for which the cumulative displacement measurement exceeds a predetermined value.