IMAGING SYSTEMS AND METHODS FOR TRACKING OBJECTS

• US 20180218224A1
• Filed: 09/23/2016
• Published: 08/02/2018
• Est. Priority Date: 09/23/2015
• Status: Active Grant

• Current Assignee: Datalogic USA, Inc.
• Original Assignee: Datalogic Adc, Inc. (Datalogic S.p.A.)
• Inventors: Olmstead, Bryan, Gremmer, Isaac, Macchia, Michele
• Granted Patent: US 10,936,881 B2
• Time in Patent Office: Days
• CPC Class Codes: G06Q 30/0601 Electronic shopping [e-shopping]
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G07G 1/0045 with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader
• CPC Class Codes: G08B 13/19643 wherein the cameras play different roles, e.g. different resolution, different camera type, master-slave camera
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A first imager has a relatively high resolution and a relatively narrow first field-of-view. Information about objects in an environment is detected or captured, and used to steer the first field-of-view of the first imager. The sensor(s) may take the form of a second imager with a relatively lower resolution and relatively wider second field-of-view. Alternatively, other types of sensors, for instance presence/absence sensors may be employed. The first field-of-view may be directed toward an object that satisfies one or more conditions, for instance matching a particular SKU. The first field-of-view may track a moving object, for instance via a tracking mirror and actuator. This approach may be employed in retail locations, for example in grocery or convenience stores, for instance to reduce various forms of theft or in industrial environments.

Claims

1. An imaging system to image objects in an environment, the imaging system comprising:

a first imager having a first imager field-of-view to capture images of the environment;

a steering component selectively operable to steer at least a portion of the first imager field-of-view relative to one or more objects in the environment; and

an object tracking subsystem that includes at least one hardware processor, the object tracking subsystem communicatively coupled to cause the steering component to steer the first imager field-of-view based at least in part on information indicative of at least one of a presence, a position, a speed or a direction of at least one object in the environment to at least partially track objects in the environment.

2. The imaging system of claim 1 wherein the first imager is a rolling shutter imager.

3. The imaging system of claim 1 wherein the first imager is a rolling shutter imager having a complementary metal oxide semiconductor (CMOS) image sensor and at least one of a mechanical or electronic rolling shutter.

4. The imaging system of claim 1 wherein steering component comprises an electro-actuated component that moves the image sensor to track the object movement.

5. The imaging system of claim 2 wherein the steering component comprises a steering mirror interposed along a first optical path between the first imager and the environment, the steering mirror selectively operable to steer at least a portion of the first imager field-of-view relative to one or more objects in the environment, and wherein the steering mirror is a fast scanning mirror.

6. The imaging system of claim 5 wherein the object tracking subsystem comprises:

a second imager having a second imager field-of-view to capture images of the environment, the first imager field-of-view relatively more narrow than the second imager field-of-view; and

a control subsystem, the control subsystem communicatively coupled the second imager to receive information directly or indirectly therefrom, and communicatively coupled to cause the steering mirror to steer the first imager field-of-view based at least in part on information received via the second imager.

7. The imaging system of claim 6 wherein the object tracking subsystem detects an entrance of at least a first object into the second field-of-view.

8. The imaging system of claim 6 wherein the object tracking subsystem identifies an object entering the second field-of-view as corresponding to a defined type of object.

9. The imaging system of claim 6 wherein the object tracking subsystem detects an entrance of at least a first object into the first field-of-view.

10. The imaging system of claim 6 wherein the object tracking subsystem identifies an object entering the first field-of-view as corresponding to a defined type of object.

11. The imaging system of claim 7 wherein the object tracking subsystem determines a position of the object.

12. The imaging system of any of claim 7 wherein the object tracking subsystem determines at least an estimate of a speed of the object.

13. The imaging system of claim 12 wherein the object tracking subsystem determines a direction of the object.

14. The imaging system of any of claim 7 wherein the object tracking subsystem determines a direction of the object.

15. The imaging system of claim 7, further comprising:

a steering mirror actuator drivingly coupled to the steering mirror and responsive to signals from the control subsystem of the object tracking subsystem to steer the first imager field-of-view.

16. The imaging system of any of claim 7 wherein the first imager is a rolling shutter imager having an image sensor and at least one of a mechanical or electronic rolling shutter, the first imager having a higher resolution than the second imager, and further comprising:

a steering mirror actuator drivingly coupled to the steering mirror and responsive to signals from the control subsystem of the object tracking subsystem to concurrently steer an entire field-of-view of the image sensor of the first imager.

17. The imaging system of claim 16 wherein the control subsystem of the object tracking system causes the steering mirror actuator to move the steering mirror to an initial scan position at a first speed, then to immediately follow a rolling shutter exposure by panning the sensor at a second speed, the second speed slower than the first speed.

18. The imaging system of claim 6 wherein the environment is a retail environment.

19. The imaging system of claim 18 wherein the retail environment is a checkout stand with a point-of-sale terminal, and the first and the second fields-of-view each encompass at least a portion of the checkout stand.

20. The imaging system of claim 18 wherein the retail environment is a checkout stand with a conveyor belt, and the first and the second fields-of-view encompass at least a portion of the conveyor belt.

21. The imaging system of claim 18 wherein the retail environment is an aisle spaced remotely from a checkout stand, and the first and the second fields-of-view encompass at least a portion of the aisle.

22. The imaging system of any of claim 19 wherein the object tracking subsystem identifies a first object in the first field-of-view as corresponding to a defined stock keeping unit (SKU) of a retail object.

23. The imaging system of any of claim 19 wherein the object tracking subsystem identifies a second object in at least one of the first or the second field-of-view as corresponding to at least one of a shopping cart or a shopping basket in which one or more retail objects reside.

24. The imaging system of any of claim 19 wherein the object tracking subsystem determines whether one or more retail objects are retained in a shopping cart or a shopping basket at a defined point in a retail transaction.

25. The imaging system of claim 6 wherein the environment is an industrial environment with a conveyor belt and the first field of view encompasses at least a portion of the conveyor belt.

26. The imaging system of claim 1, further comprising:

a variable focus lens positioned between the first imager and the environment;

an illumination source positioned and oriented to illuminate at least a portion of the environment in the first field-of-view; and

a polarizer positioned between the first imager and the environment.

27. The imaging system of claim 1 wherein the at least one processor performs compensation to recover blur and motion induced artifacts.

28. The imaging system of any of claim 5 wherein the at least one processor identifies at least one region-of-interest in an image captured by the second imager, and determines a set of coordinates to steer the first imager field-of-view to capture images of the region-of-interest.

29. The imaging system of claim 28 wherein the at least one processor generates a path to steer the first imager field-of-view to capture images of multiple regions-of-interest.

30. The imaging system of claim 29 wherein the at least one processor prioritizes image capture by the first imager between multiple regions-of-interest.

31. The imaging system of any of claim 5 wherein the second imager is a time-of-flight camera.

32-54. -54. (canceled)