Method and system for recapturing a trajectory of an object

• US 6,449,382 B1
• Filed: 04/28/1999
• Issued: 09/10/2002
• Est. Priority Date: 04/28/1999
• Status: Active Grant

• Current Assignee: LinkedIn Corporation (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Hobbs, Philip C. D., Sachar, Howard E., Ciccolo, Arthur Charles
• Primary Examiner(s): Johns, Andrew W.
• Assistant Examiner(s): NAKHJAVAN, SHERVIN K
• Application Number:
  US09/300,999
• Time in Patent Office: 1,231 Days
• Field of Search: 382/100, 382/103, 382/107, 382/291, 382/293, 382/294, 382/295, 348/169, 348/25, 348/48, 396/153, 352/39, 352/137, 346/516
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image

Abstract

The present invention is in the field of sensor fusion, and discloses in one embodiment a method for extracting the trajectories of moving objects from an assembly of low-resolution sensors, whose spatial relationships are initially unknown, except that their fields of view are known to overlap so as to form a continuous coverage region, which may be much larger than the field of view of any individual sensor. Segments of object trajectories may be extracted from the data of each sensor, and then stitched together to reconstruct the trajectories of the objects. The stitching process also allows determination of the spatial relationships between the sensors, so that from initially knowing little or nothing about the sensor arrangement or the paths of the objects, both may be reconstructed unambiguously.

Claims

1. A method for recapturing a trajectory of an object, comprising:

employing first and second sensors, respectively capable of gathering information about an object'"'"'s trajectory within first and second at least partially overlapping fields of view, thereby obtaining a sub-trajectory in each field of view;

self-configuring said first and second sensors by computing a spatial relationship from the offsets between sub-trajectories; and

stitching the sub-trajectories together in sequence to form a single estimated trajectory.

2. A method according to claim 1, wherein said sensors comprise multi-pixel infrared sensors.

3. A method according to claim 1, further comprising:

time stamping each sub-trajectory.

4. A method according to claim 3, further comprising:

using the time stamp for identifying sub-trajectories corresponding to the same object.

5. A method according to claim 3, further comprising:

using the time stamps for rejecting subtrajectories which if stitched together would give rise to a discontinuous estimated trajectory.

6. A method according to claim 1 wherein said stitching comprises choosing among sub-trajectories candidates for stitching, based on the sub-trajectories'"'"' spatial proximity in the overlap region.

7. A method according to claim 1, wherein each sub-trajectory is time stamped, and wherein said stitching comprises choosing among sub-trajectories candidates for stitching, based on the sub-trajectories'"'"' spatial proximity in the overlap region.

8. A method according to claim 7, further comprising:

using the time stamps for rejecting stitching candidates which, if stitched together, would give rise to a discontinuous estimated trajectory.

9. A method according to claim 1, further comprising:

employing a third sensor capable of gathering information about an object'"'"'s trajectory within a third field of view overlapping both the first and second fields of view, in which the mutual consistency of two estimated spatial relationships between sensors is improved by adjusting the estimates to reduce the error residuals from pairwise comparison of a trajectory in the overlap regions.

10. The method according to claim 1, wherein said data used for self-configuration of said first and second sensors is used to form said single estimated trajectory.

11. A method for recapturing a trajectory of an object, the method comprising the steps of:

1) employing first and second sensors, respectively capable of gathering information about an object'"'"'s trajectory in first and second fields of view, thereby obtaining a sub trajectory in each field of view, and so disposed that a known characteristic of the object'"'"'s motion places bounds on an error committed in extrapolating its subtrajectories between the edges of the fields of view; and

2) stitching the subtrajectories into a single estimated trajectory.

12. A method according to claim 11, comprising the step of extrapolating one or both subtrajectories so that they overlap in time, prior to stitching them together.

13. A system suitable for recapturing a trajectory of an object, the system comprising:

first and second sensors, respectively capable of gathering information about an object'"'"'s trajectory within first and second at least partially overlapping fields of view, for obtaining a sub-trajectory in each field of view; and

means for stitching the sub-trajectories together in sequence to form a single estimated trajectory,

wherein said first and second sensors are self-configured by computing a spatial relationship from the offsets between sub-trajectories.

14. A system according to claim 13, wherein at least one of the first and second sensors comprises a multi-pixel infrared sensor.

15. A method for determining a spatial relationship between the coordinate systems corresponding to two multi-pixel sensors, whose fields of view are known to overlap at least partially, the method comprising:

gathering sub-trajectory data for an object moving in the fields of view;

selecting sub-trajectories which in sequence form a trajectory of the object; and

computing the spatial relationship from the offsets between the selected sub-trajectories.

16. A method according to claim 15, wherein the object emits a recognizable signature that differentiates it from other objects.

17. A method according to claim 15, wherein said sensor has, at most, one object in its field of view.

18. A method according to claim 15, wherein said selecting comprises choosing among only those sub-trajectories whose time extents overlap.

19. A method according to claim 15, wherein said computing comprises applying the condition that the trajectory of a real object is continuous in time and space.

20. A method according to claim 15, further comprising:

employing a third multi-pixel infrared sensor, in which the mutual consistency of two spatial relationships between sensors is improved by the imposition of a priori geometrical constraints.

21. A method for determining a spatial relationship between the coordinate systems corresponding to two multi-pixel sensors, the method comprising the steps of:

1) gathering sub-trajectory data for an object moving in the fields of view of the sensors, where a known characteristic of the object'"'"'s motion places bounds on an error committed in extrapolating its subtrajectories between the edges of the fields of view;

2) selecting from the sub-trajectories those which correspond to the object;

3) extrapolating the subtrajectories until they overlap in time; and

4) computing the spatial relationship from the offsets between the selected sub-trajectories.

22. A system suitable for determining a spatial relationship between the coordinate systems corresponding to two multi-pixel sensors whose fields of view are known to overlap at least partially, the system comprising:

means for gathering sub-trajectory data for an object moving in the fields of view;

means for selecting sub-trajectories which in sequence form a trajectory of the object; and

means for computing the spatial relationships from the offsets between the selected sub-trajectories.