SYNCHRONIZATION OF MULTIPLE DATA SOURCES TO A COMMON TIME BASE

• US 20110228091A1
• Filed: 05/31/2011
• Published: 09/22/2011
• Est. Priority Date: 01/16/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Ponticelli, Martin Josef, Kimchi, Gur, Curlander, John Charles, Neuhold, Gerhard, Kroepfl, Michael, Pehserl, Joachim, Bernögger, Stefan
• Granted Patent: US 8,428,088 B2
• Time in Patent Office: Days
• US Class Current: 348/148
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks

Abstract

Systems and methods are described herein that cause data from asynchronous data sources to be provided with a timestamp that corresponds to a common time base. A trigger board can be used to control synchronized data sources, and can generate timestamps when data is collected by the synchronized data sources. Unsynchronized data sources can generate data independent of the trigger board. System timestamps are generated each time data from the synchronized data source and the unsynchronized data source is received. Values of the system timestamp can be modified, and can be replaced by timestamps that correspond to the time base used by the trigger board.

Claims

1. A method that facilitates synchronizing data from different data sources having different time bases, the method comprising:

receiving a plurality of images captured by a camera;

receiving, for the plurality of images captured by the camera, a respective first plurality of timestamps having a first time base;

receiving a plurality of data packets that are output by a sensor, the data packets having a second time base and comprising data that is indicative of position of the camera over time;

receiving a sampling frequency of the sensor; and

synchronizing the plurality of images captured by the camera with the plurality of data packets based at least in part upon the first time base and the sampling frequency of the sensor such that the plurality of data packets are assigned timestamps having the first time base.

2. The method of claim 1, the sensor being a Global Positioning System sensor.

3. The method of claim 1, further comprising:

transmitting a plurality of trigger signals to the camera that cause the camera to capture the plurality of images, the trigger signals having the plurality of timestamps having the first time base.

4. The method of claim 1 configured for execution on an automobile that comprises the camera and the sensor.

5. The method of claim 1, the data packets having a second plurality of timestamps corresponding to the second time base assigned thereto, and further comprising:

computing the sampling frequency of the sensor based at least in part upon the timestamps corresponding to the second time base that are assigned to the data packets.

6. The method of claim 5, further comprising:

assigning a third plurality of timestamps to the images captured by the camera, the third plurality of timestamps having a third time base;

assigning a fourth plurality of timestamps to the data packets output by the sensor, the fourth plurality of timestamps having the third time base; and

synchronizing the plurality of images captured by the camera with the plurality of data packets based at least in part upon the third plurality of timestamps assigned to the images captured by the camera and the fourth plurality of timestamps assigned to the data packets output by the sensor.

7. The method of claim 6, further comprising:

modifying at least one timestamp in the fourth plurality of timestamps based at least in part upon the sampling frequency of the sensor.

8. The method of claim 1, wherein the sensor is one of a distance measurement instrument, an inertial navigation system, or a velocity sensor.

9. The method of claim 1, further comprising:

transmitting a trigger signal that causes the camera and at least one other camera to simultaneously capture images, the simultaneously captured images assigned timestamps of the first time base having the same value.

10. The method of claim 1, wherein the plurality of images captured by the camera and the plurality of data packets output by the sensor are received by way of a serial interface of a computing device.

11. The method of claim 1, wherein the camera is a near infrared camera.

12. A system that facilitates synchronizing data from different data sources, comprising:

a trigger board that outputs trigger signals according to a first time base;

a camera in communication with the trigger board that captures a plurality of images having a respective plurality of timestamps of the first time base corresponding thereto, the camera capturing an image each time a trigger signal is received from the trigger board, and each image captured by the camera assigned a timestamp of the first time base;

a first sensor that outputs a plurality of data packets at a sampling frequency that corresponds to a second time base, the data packets output by the sensor being indicative of location of the camera over time; and

a computing apparatus that receives the plurality of images and corresponding timestamps of the first time base and the plurality of data packets output by the first sensor and assigns the plurality of data packets output by the first sensor timestamps of the first time base based at least in part upon the timestamps of the plurality of images and the sampling frequency of the first sensor.

13. The system of claim 12, wherein the camera is a near infrared camera.

14. The system of claim 12, wherein the first sensor is one of an inertial navigation system, a Global Positioning System sensor, a velocity sensor, an accelerometer.

15. The system of claim 12, wherein the computing apparatus receives the plurality of images and corresponding timestamps and the plurality of data packets output by the sensor by way of a serial interface.

16. The system of claim 12, wherein the computing apparatus assigns a first set of system timestamps to the plurality of images and a second set of system timestamps to the data packets output by the first sensor, and wherein the computing apparatus assigns the plurality of data packets output by the first sensor the timestamps of the first time base based at least in part upon the first set of system timestamps and the second set of system timestamps.

17. The system of claim 12, further comprising a plurality of cameras oriented in varying directions, each of the plurality of cameras capturing images responsive to receipt of trigger signals from the trigger board.

18. The system of claim 12, further comprising a second sensor that outputs data packets indicative of the location of the camera at a second sampling frequency, and wherein the computing apparatus assigns timestamps of the first time base to the data packets output by the second sensor based at least in part upon the timestamps of the plurality of images and the second sampling frequency.

19. The system of claim 12 coupled to an automobile.

20. A computer-readable medium comprising instructions that, when executed by a processor, cause the processor to perform acts comprising:

receiving a plurality of images with a respective plurality of timestamps of a first time base;

assigning a first plurality of system timestamps to the plurality of images, the first plurality of system timestamps being of a second time base;

receiving a plurality of data packets from a sensor that outputs the plurality of data packets at a sampling frequency;

assigning a second plurality of system timestamps to the plurality of data packets, the second plurality of system timestamps being of the second time base; and

assigning a third plurality of timestamps of the first time base to the plurality of data packets based at least in part upon the first plurality of system timestamps assigned to the plurality of images and the second plurality of system timestamps to the plurality of data packets.