Imaging systems and methods for tracking objects

• US 10,936,881 B2
• Filed: 09/23/2016
• Issued: 03/02/2021
• Est. Priority Date: 09/23/2015
• Status: Active Grant

• Current Assignee: Datalogic USA, Inc.
• Original Assignee: Datalogic USA, Inc.
• Inventors: Olmstead, Bryan, Gremmer, Isaac, Macchia, Michele
• Primary Examiner(s): Tran, Trang U
• Application Number:
  US15/762,997
• Publication Number:
  US 20180218224A1
• Time in Patent Office: 1,621 Days
• Field of Search: 348169, 348 36, 348 39, 348143
• CPC Class Codes: G06Q 30/0601 Electronic shopping [e-shopping]
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G07G 1/0045 with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader
• CPC Class Codes: G08B 13/19643 wherein the cameras play different roles, e.g. different resolution, different camera type, master-slave camera
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A first imager has a relatively high resolution and a relatively narrow first field-of-view. Information about objects in an environment is detected or captured, and used to steer the first field-of-view of the first imager. The sensor(s) may take the form of a second imager with a relatively lower resolution and relatively wider second field-of-view. Alternatively, other types of sensors, for instance presence/absence sensors may be employed. The first field-of-view may be directed toward an object that satisfies one or more conditions, for instance matching a particular SKU. The first field-of-view may track a moving object, for instance via a tracking mirror and actuator. This approach may be employed in retail locations, for example in grocery or convenience stores, for instance to reduce various forms of theft or in industrial environments.

Claims

1. An imaging system to image objects in retail environment, the imaging system comprising:

a first imager having a first imager field-of-view to capture images of the environment, wherein the first imager is a rolling shutter imager;

a steering component selectively operable to steer at least a portion of the first imager field-of-view relative to one or more objects in the environment; and

an object tracking subsystem that includes at least one hardware processor, the object tracking subsystem communicatively coupled to cause the steering component to steer the first imager field-of-view based at least in part on information indicative of at least one of a presence, a position, a speed or a direction of at least one object in the environment to at least partially track objects in the environment;

wherein the object tracking subsystem scans a plurality of objects that enter the environment, distinguishes between objects of the plurality of objects as being either an object of interest or an object not of interest, and responsive to identification of an object of interest causes the steering component to steer the first imager field-of-view to an initial scan position at a first speed such that the identified object of interest is positioned within the first imager field-of-view and then to follow a rolling shutter exposure by panning the first field of view of the first imager at a second speed, the second speed slower than the first speed.

2. The imaging system of claim 1 wherein the rolling shutter imager includes a complementary metal oxide semiconductor (CMOS) image sensor and at least one of a mechanical or electronic rolling shutter.

3. The imaging system of claim 1 wherein the steering component comprises an electro-actuated component that moves the image sensor to track the object movement.

4. The imaging system of claim 1 wherein the steering component comprises a steering mirror interposed along a first optical path between the first imager and the environment, the steering mirror selectively operable to steer at least a portion of the first imager field-of-view relative to one or more objects in the environment, and wherein the steering mirror is a fast scanning mirror.

5. The imaging system of claim 4 wherein the object tracking subsystem comprises:

a second imager having a second imager field-of-view to capture images of the environment, the first imager field-of-view relatively more narrow than the second imager field-of-view; and

a control subsystem, the control subsystem communicatively coupled the second imager to receive information directly or indirectly therefrom, and communicatively coupled to cause the steering mirror to steer the first imager field-of-view based at least in part on information received via the second imager.

6. The imaging system of claim 4 wherein the at least one processor identifies at least one region-of-interest in an image captured by the second imager, and determines a set of coordinates to steer the first imager field-of-view to capture images of the region-of-interest.

7. The imaging system of claim 6 wherein the at least one processor generates a path to steer the first imager field-of-view to capture images of multiple regions-of-interest.

8. The imaging system of claim 7 wherein the at least one processor prioritizes image capture by the first imager between multiple regions-of-interest.

9. The imaging system of claim 4 wherein the second imager is a time-of-flight camera.

10. The imaging system of claim 5 wherein the object tracking subsystem detects an entrance of at least a first object into the second field-of-view.

11. The imaging system of claim 5 wherein the object tracking subsystem identifies an object entering the second field-of-view as corresponding to a defined type of object.

12. The imaging system of claim 5 wherein the object tracking subsystem detects an entrance of at least a first object into the first field-of-view.

13. The imaging system of claim 5 wherein the object tracking subsystem identifies an object entering the first field-of-view as corresponding to a defined type of object.

14. The imaging system of claim 10 wherein the object tracking subsystem determines a position of the object, an estimate of a speed of the object, a direction of the object, or any combination thereof.

15. The imaging system of claim 10, further comprising:

a steering mirror actuator drivingly coupled to the steering mirror and responsive to signals from the control subsystem of the object tracking subsystem to steer the first imager field-of-view.

16. The imaging system of claim 10 wherein the first imager has a higher resolution than the second imager, and further comprising:

a steering mirror actuator drivingly coupled to the steering mirror and responsive to signals from the control subsystem of the object tracking subsystem to concurrently steer an entire field-of-view of the image sensor of the first imager.

17. The imaging system of claim 1 wherein the retail environment is:

a) a checkout stand with a point-of-sale terminal, and the first and the second fields-of-view each encompass at least a portion of the checkout stand;

b) a checkout stand with a conveyor belt, and the first and the second fields-of-view encompass at least a portion of the conveyor belt;

or

c) an aisle spaced remotely from a checkout stand, and the first and the second fields-of-view encompass at least a portion of the aisle.

18. The imaging system of claim 17 wherein the object tracking subsystem identifies a first object in the first field-of-view as corresponding to a defined stock keeping unit (SKU) of a retail object.

19. The imaging system of claim 17 wherein the object tracking subsystem identifies a second object in at least one of the first or the second field-of-view as corresponding to at least one of a shopping cart or a shopping basket in which one or more retail objects reside.

20. The imaging system of claim 17 wherein the object tracking subsystem determines whether one or more retail objects are retained in a shopping cart or a shopping basket at a defined point in a retail transaction.

21. The imaging system of claim 5 wherein the environment is an industrial environment with a conveyor belt and the first field of view encompasses at least a portion of the conveyor belt.

22. The imaging system of claim 1, further comprising:

a variable focus lens positioned between the first imager and the environment;

an illumination source positioned and oriented to illuminate at least a portion of the environment in the first field-of-view; and

a polarizer positioned between the first imager and the environment.

23. The imaging system of claim 1 wherein the at least one processor performs compensation to recover blur and motion induced artifacts.

24. The imaging system of claim 1 wherein response to identification of an object not of interest the object tracking subsystem refrains from causing the steering component to steer the first imager field-of-view such that the object not of interest is positioned within the first imager field-of-view.

25. The imaging system of claim 1 wherein, to identify the at least one of the plurality of objects as an object of interest, the object tracking subsystem employs:

a) image processing to identify graphical elements, text, symbols, or any combination thereof;

b) machine-vision techniques;

c) parcel dimensioning techniques;

or

d) any combination thereof.

26. The imaging system of claim 1 wherein the object tracking system is communicatively coupled to a database such that the object tracking system queries the database during identification of the at least one of the plurality of objects as an object of interest.

27. The imaging system of claim 26 wherein the database includes information specific to objects of interest, the information including:

price, manufacturer, model, stock keeping unit, age restriction on sale, special promotion, tax status, color, logo, text, symbol, or any combination thereof.