System and method for detecting and tracking objects

• US 10,108,863 B2
• Filed: 09/02/2016
• Issued: 10/23/2018
• Est. Priority Date: 09/03/2015
• Status: Active Grant

• Current Assignee: Miovision Technologies Incorporated
• Original Assignee: Miovision Technologies Incorporated
• Inventors: Miller, Nicholas, Mishra, Akshaya K., Eichel, Justin A.
• Primary Examiner(s): Fitzpatrick, Atiba O
• Application Number:
  US15/255,326
• Publication Number:
  US 20170068861A1
• Time in Patent Office: 781 Days
• Field of Search: None
• CPC Class Codes: G06F 18/2113 by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats

Abstract

A system and method are provided for mapping spatial and temporal measurements of motion constrained objects in a scene, e.g., vehicles. The method comprises determining a location parameter, and/or other interesting properties, for each of a plurality of objects at a plurality of points over time; generating a vector field over location and time using the location parameters, to specify the motion of each object over time; deriving measurements for each object using velocities from the vector field for that object, as the object moves through the scene over time; and outputting both individual and aggregate object and object property flow for the plurality of objects. Combining the generated map with a network graph and/or network model can provide network flow insights useful for historic event analysis, network flow monitoring, and planning purposes.

Claims

1. A method of mapping movement of objects through a known space, the method comprising:

determining a location parameter for each of a plurality of objects at a plurality of points in time and corresponding locations in the known space, the known space comprising a network of motion constrained pathways for the objects;

generating a vector field over location and time across the known space using the location parameters, to specify, for each object, motion of that object through the known space over time;

generating a motion path for each object using velocities from the vector field for that object, as the object moves through the known space over time; and

outputting a mapping of the motion paths for the plurality of objects over the known space.

2. The method of claim 1, further comprising:

estimating a location parameter for velocities for each of a plurality of objects at a given time;

or

incorporating measured velocity parameters into location parameters for each of the plurality of objects.

3. The method of claim 1, wherein unique object presence and spatial occupancy are obtained through vector field intersections.

4. The method of claim 1, further comprising removing at least one measurement associated with noise, clutter, or a spurious measurement.

5. The method of claim 1, wherein the location parameter is determined using at least one image sensor applied to an image or video.

6. The method of claim 1, wherein the location parameter is determined using a position sensor.

7. The method of claim 6, wherein the position sensor corresponds to an egocentric video-based localization.

8. The method of claim 1, further comprising:

generating at least one additional mapping of the motion paths in at least one additional vector field for the known space, or augmentation to at least one existing field, associated with at least one additional corresponding location in the known space; and

outputting at least one additional mapping.

9. The method of claim 8, further comprising interpolating the motion paths between locations to account for gaps between location sensors.

10. The method of claim 1, wherein the location parameter comprises two dimensions, and the mapping is a three dimensional vector field.

11. The method of claim 1, further comprising obtaining a video capturing at least one image of a scene in the known space, and determining the location parameter using the video.

12. The method of claim 11, wherein multiple videos, live or recorded, are used to map a larger spatial location.

13. The method of claim 11, wherein the video extends to a plurality of spectral bands selected to better detect objects of interest.

14. The method of claim 1, further comprising utilizing multiple camera registrations to improve data collection spatial and temporal synchronization issues using flow as a salient feature.

15. The method of claim 1, wherein the location parameter of a single object at a given time is determined using an object passing through a gate with a known position, the location parameter corresponding to the known position being incorporated into the vector field.

16. The method of claim 1, wherein at least one of the objects is a vehicle.

17. The method of claim 1, wherein the known space comprises at least one scene each scene comprising at least one constraint to motion of the objects through that scene.

18. The method of claim 17, wherein at least one constraint corresponds to a roadway and at least one of the objects corresponds to a vehicle.

19. The method of claim 1, further comprising analyzing the mapping of the motion paths, and using an output of the analyzing to perform at least one adaptive control operation.

20. The method of claim 19, wherein the adaptive control operation is applied to the known space or another known space in a traffic network.

21. The method of claim 1, further comprising storing the mapping for subsequent monitoring.

22. The method of claim 21, further comprising at least one of monitoring and reporting on normal traffic events or abnormal traffic events, wherein normal or abnormal traffic events comprise any one or more of:

vehicles entering a wrong way on a one-way roadway, collisions, traffic jam detection, and vehicle spacing measurements.

23. The method of claim 21, further comprising monitoring to automatically adjust data collection or video surveillance bandwidth based on density of object flow.

24. The method of claim 1, further comprising storing the mapping and performing at least one monitoring operation associated with the known space.

25. The method of claim 1, further comprising measuring locations of one or more objects in one, two, or three dimensions over time.

26. The method of claim 25, wherein three dimensional location measurements are obtained for aerial or submersible vehicles.

27. A non-transitory computer readable medium comprising computer executable instructions for mapping movement of objects through a known space, comprising instructions for:

determining a location parameter for each of a plurality of objects at a plurality of points in time and corresponding locations in the known space, the known space comprising a network of motion constrained pathways for the objects;

generating a vector field over location and time across the known space using the location parameters, to specify, for each object, motion of that object through the known space over time;

generating a motion path for each object using velocities from the vector field for that object, as the object moves through the known space over time; and

outputting a mapping of the motion paths for the plurality of objects over the known space.

28. A system comprising a processor and memory, the memory comprising computer executable instructions for mapping movement of objects through a known space, comprising instructions for:

determining a location parameter for each of a plurality of objects at a plurality of points in time and corresponding locations in the known space, the known space comprising a network of motion constrained pathways for the objects;

generating a vector field over location and time across the known space using the location parameters, to specify, for each object, motion of that object through the known space over time;

generating a motion path for each object using velocities from the vector field for that object, as the object moves through the known space over time; and

outputting a mapping of the motion paths for the plurality of objects over the known space.