Method and system for tracking objects between cameras

• US 9,947,107 B2
• Filed: 09/16/2015
• Issued: 04/17/2018
• Est. Priority Date: 09/16/2015
• Status: Active Grant

• Current Assignee: AGT International GMBH
• Original Assignee: AGT International GMBH
• Inventors: Krauss, Stephan, Hirzel, Jan
• Primary Examiner(s): Kelley, Christopher S
• Assistant Examiner(s): Tarko, Asmamaw G
• Application Number:
  US14/855,762
• Publication Number:
  US 20170076468A1
• Time in Patent Office: 944 Days
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

A method, system and non-transitory computer readable storage medium, the computer-implemented method comprising: receiving a first image captured by a first capture device, the first images depicting first objects; obtaining a kinematic value for each of the first objects; receiving a second image captured by a second capture device, the second images depicting second objects; and analyzing the first and second images and the kinematic value, to determine transition parameters related to transition of objects from a first field of view of the first capture device to a second field of view of the second capture device, wherein the first capture device and the second capture device are calibrated, wherein a gap exists between the first and the second fields of view, and wherein said transition parameters are usable for associating further objects captured by the first capture device with corresponding objects captured by the second capture device.

Claims

1. A computer-implemented method comprising:

receiving at least one first image captured by a first capture device, the at least one first image depicting a first multiplicity of objects;

obtaining a kinematic value for each of the first multiplicity of objects;

receiving at least one second image captured by a second capture device, the at least one second image depicting a second multiplicity of objects; and

analyzing the at least one first image, the at least one second image, and the kinematic value, to determine transition parameters related to transition of objects from a first field of view of the first capture device to a second field of view of the second capture device,

wherein determining the transition parameters comprises;

selecting from a first initial multiplicity of objects appearing in the at least one first image a first potential multiplicity of objects and selecting from a second initial multiplicity of objects appearing in the at least one second image a second potential multiplicity of objects, such that each of the first potential multiplicity of objects can plausibly be associated with one of the second potential multiplicity of objects, and each of the second potential multiplicity of objects can plausibly be associated with one of the first potential multiplicity of objects;

responsive to determining the first potential multiplicity of objects and the second potential multiplicity of objects, creating a correspondence data structure between the first potential multiplicity of objects and the second potential multiplicity of objects;

selecting from the correspondence data structure a multiplicity of matches between objects captured by the first capture device and objects captured by the second capture device; and

determining the transition parameters based on locations of the objects in the multiplicity of matches, and kinematic parameters of the objects in the first potential multiplicity of objects,

wherein the first capture device and the second capture device are calibrated,

wherein a gap exists between the first field of view and the second field of view, and

wherein said transition parameters are usable for associating further objects captured by the first capture device with corresponding objects captured by the second capture device.

2. The method of claim 1, wherein the transition parameters comprise at least one parameter selected from the group consisting of:

lane offset parameter, lane offset parameter variance, speed correction and speed correction variance.

3. The method of claim 1 further comprising determining calibration between the first capture device and the second capture device.

4. The method of claim 1, further comprising enhancing the correspondence data structure with data relating to shape or color correspondence between the first potential multiplicity of objects and the second potential multiplicity of objects.

5. The method of claim 1, wherein determining whether a first object from the first potential multiplicity of objects can plausibly be associated with a second object from the second potential multiplicity of objects comprises:

determining a first distance as a distance between a location of the second object and a location of the first object;

estimating a second distance by combining a time difference between when the first object was last captured by the first capture device and when the second object was first captured by the second capture device with the kinematic value of the first object; and

subject to the first distance and the second distance differing by at most a predetermined value, determining that the first object can plausibly be associated with the second object.

6. The method of claim 1, wherein determining the transition parameters comprises calculating a speed correction factor and a speed correction factor variance, said calculating comprising:

retrieving a speed v_m measured for each object captured by the first capture device;

determining a speed v_e as

7. The method of claim 1, wherein determining the transition parameters comprises calculating a lane offset parameter and a lane offset variance parameter, said calculating comprising:

estimating the lane offset parameter as

8. A computerized system for determining transition parameters between objects appearing in a first image captured by a first capture device and objects appearing in a second image captured by a second capture device, the system comprising a processor configured to:

receive at least one first image captured by a first capture device, the at least one first image depicting a first multiplicity of objects;

obtain a kinematic value for each of the first multiplicity of objects;

receive at least one second image captured by a second capture device, the at least one second image depicting a second multiplicity of objects; and

analyze the at least one first image, the at least one second image, and the kinematic value to determine transition parameters related to transition of objects from a first field of view of the first capture device to a second field of view of the second capture device,

wherein as part of determining the transition parameters the processing unit is configured to;

determine whether each of an initial first multiplicity of objects captured by the first capture device can plausibly be associated with each of a second initial multiplicity of objects captured by the second capture device;

responsive to determining a first potential multiplicity of objects captured by the first capture device that can plausibly be associated with a second potential multiplicity of objects captured by the second capture device, create a correspondence data structure between the first potential multiplicity of objects and the second potential multiplicity of objects;

select from the correspondence data structure a multiplicity of matches between objects captured by the first capture device and objects captured by the second capture device; and

determine the transition parameters based on locations and kinematic values of the objects in the multiplicity of matches,

wherein the first capture device and the second capture device are calibrated,

wherein a gap exists between the first field of view and the second field of view, and

wherein said transition parameters are usable for associating further objects captured by the first capture device with corresponding objects captured by the second capture device.

9. The computerized system of claim 8, wherein the transition parameters comprise at least one parameter selected from the group consisting of:

lane offset parameter, lane offset parameter variance, speed correction and speed correction variance.

10. The computerized system of claim 8, wherein the processing unit is further configured to determine calibration between the first capture device and the second capture device.

11. The computerized system of claim 9, wherein the processing unit is further configured to enhance the correspondence data structure with data relating to shape or color of the first potential multiplicity of objects and the second potential multiplicity of objects.

12. The computerized system of claim 9, wherein as part of determining whether a first object captured by the first capture device can plausibly be associated with a second object captured by the second capture device, the processing unit is configured to:

determine a first distance as a distance between a location of the second object and a location of the first object;

estimate a second distance as a distance the first object has passed since leaving the field of view of the first capture device, by combining a time difference between when the first object was last captured by the first capture device and when the second object was first captured by the second capture device with the kinematic value of the first object; and

subject to the first distance and the second distance differing by at most a predetermined value, determine that the first object can plausibly be associated with the second object.

13. The computerized system of claim 9, wherein as part of determining the transition parameters the processing unit is configured to calculate a speed correction factor and a speed correction factor variance by:

retrieving a speed v_m measured for each object captured by the first capture device;

determining a speed v_e as

14. The computerized system of claim 8, wherein as part of determining the transition parameters the processing unit is configured to calculate a lane offset parameter and a lane offset variance parameter by:

estimating the lane offset parameter as

15. A non-transitory computer readable storage medium tangibly embodying a program of instructions executable by a machine to determine transition parameters between objects appearing in a first image captured by a first capture device and objects appearing in a second image captured by a second capture device, comprising the steps of:

receiving at least one first image captured by a first capture device, the at least one first image depicting a first multiplicity of objects;

obtaining a kinematic value for each of the first multiplicity of objects;

receiving at least one second image captured by a second capture device, the at least one second image depicting a second multiplicity of objects; and

analyzing the at least one first image, the at least one second image, and the kinematic value to determine transition parameters related to transition of objects from a first field of view of the first capture device to a second field of view of the second capture device,

wherein determining the transition parameters comprises;

selecting from a first initial multiplicity of objects appearing in the at least one first image a first potential multiplicity of objects and selecting from a second initial multiplicity of objects appearing in the at least one second image a second potential multiplicity of objects, such that each of the first potential multiplicity of objects can plausibly be associated with one of the second potential multiplicity of objects, and each of the second potential multiplicity of objects can plausibly be associated with one of the first potential multiplicity of objects;

responsive to determining the first potential multiplicity of objects and the second potential multiplicity of objects, creating a correspondence data structure between the first potential multiplicity of objects and the second potential multiplicity of objects;

selecting from the correspondence data structure a multiplicity of matches between objects captured by the first capture device and objects captured by the second capture device; and

determining the transition parameters based on locations of the objects in the multiplicity of matches, and kinematic parameters of the objects in the first potential multiplicity of objects,

wherein the first capture device and the second capture device are calibrated,

wherein a gap exists between the first field of view and the second field of view, and

wherein said transition parameters are usable for associating further objects captured by the first capture device with corresponding objects captured by the second capture device.