Synchronization of multiple data sources to a common time base

• US 8,428,088 B2
• Filed: 05/31/2011
• Issued: 04/23/2013
• Est. Priority Date: 01/16/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Kroepfl, Michael, Neuhold, Gerhard, Ponticelli, Martin Josef, Pehserl, Joachim, Kimchi, Gur, Curlander, John Charles, Berngger, Stefan
• Primary Examiner(s): Hailu, Kibrom T
• Assistant Examiner(s): LAMONT, BENJAMIN S
• Application Number:
  US13/149,203
• Publication Number:
  US 20110228091A1
• Time in Patent Office: 693 Days
• Field of Search: 370/395.62, 370503-509, 375/356, 375/371, 375/373, 375/376
• US Class Current: 370/503
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks

Abstract

Systems and methods are described herein that cause data from asynchronous data sources to be provided with a timestamp that corresponds to a common time base. A trigger board can be used to control synchronized data sources, and can generate timestamps when data is collected by the synchronized data sources. Unsynchronized data sources can generate data independent of the trigger board. System timestamps are generated each time data from the synchronized data source and the unsynchronized data source is received. Values of the system timestamp can be modified, and can be replaced by timestamps that correspond to the time base used by the trigger board.

Claims

1. A method that facilitates synchronizing data from different data sources having different time bases, the method comprising:

receiving a plurality of images captured by a camera;

receiving, for the plurality of images captured by the camera, a respective first plurality of timestamps having a first time base;

assigning a first plurality of system timestamps to the plurality of images, the first plurality of system timestamps being of a second time base;

receiving a plurality of data packets that are output by a sensor, the plurality of data packets having a third time base and comprising data that is indicative of position of the camera over time;

receiving a sampling frequency of the sensor;

assigning a second plurality of system timestamps to the plurality of data packets, the second plurality of system timestamps being of the second time base; and

synchronizing the plurality of images captured by the camera with the plurality of data packets based at least in part upon the first time base, the second time base, and the sampling frequency of the sensor, such that the plurality of data packets are assigned timestamps having the first time base.

2. The method of claim 1, the sensor being a Global Positioning System sensor.

3. The method of claim 1, further comprising:

transmitting a plurality of trigger signals to the camera that cause the camera to capture the plurality of images, the trigger signals having the plurality of timestamps having the first time base.

4. The method of claim 1 configured for execution on an automobile that comprises the camera and the sensor.

5. The method of claim 1, further comprising:

computing the sampling frequency of the sensor based at least in part upon the first plurality of system timestamps being of the second time base that are assigned to the data packets.

6. The method of claim 1, wherein the sensor is one of a distance measurement instrument, an inertial navigation system, or a velocity sensor.

7. The method of claim 1, further comprising:

transmitting a trigger signal that causes the camera and at least one other camera to simultaneously capture images, the simultaneously captured images assigned timestamps of the first time base having the same value.

8. The method of claim 1, wherein the plurality of images captured by the camera and the plurality of data packets output by the sensor are received by way of a serial interface of a computing device.

9. The method of claim 1, wherein the camera is a near infrared camera.

10. A system that facilitates synchronizing data from different data sources, comprising:

a trigger board that outputs trigger signals according to a first time base;

a camera in communication with the trigger board that captures a plurality of images having a respective plurality of timestamps of the first time base corresponding thereto, the camera capturing an image each time a trigger signal is received from the trigger board, and each image captured by the camera assigned a timestamp of the first time base;

a first sensor that outputs a plurality of data packets at a sampling frequency that corresponds to a second time base, the data packets output by the first sensor being indicative of location of the camera over time; and

a computing apparatus that receives the plurality of images and corresponding timestamps of the first time base and the plurality of data packets output by the first sensor, assigns the plurality of images a first plurality of system timestamps having a third time base, assigns the plurality of data packets a second plurality of system timestamps having the third time base, and assigns the plurality of data packets output by the first sensor timestamps of the first time base based at least in part upon the timestamps of the plurality of images, the first plurality of system timestamps, the second plurality of system timestamps, and the sampling frequency of the first sensor.

11. The system of claim 10, wherein the camera is a near infrared camera.

12. The system of claim 10, wherein the first sensor is one of an inertial navigation system, a Global Positioning System sensor, a velocity sensor, and an accelerometer.

13. The system of claim 10, wherein the computing apparatus receives the plurality of images and corresponding timestamps and the plurality of data packets output by the sensor by way of a serial interface.

14. The system of claim 10, further comprising a plurality of cameras oriented in varying directions, each of the plurality of cameras capturing images responsive to receipt of trigger signals from the trigger board.

15. The system of claim 10, further comprising a second sensor that outputs data packets indicative of the location of the camera at a second sampling frequency, and wherein the computing apparatus assigns timestamps of the first time base to the data packets output by the second sensor based at least in part upon the timestamps of the plurality of images and the second sampling frequency.

16. The system of claim 10 coupled to an automobile.

17. A computer-readable data storage device comprising instructions that, when executed by a processor, cause the processor to perform acts comprising:

receiving a plurality of images with a respective plurality of timestamps of a first time base;

assigning a first plurality of system timestamps to the plurality of images, the first plurality of system timestamps being of a second time base;

receiving a plurality of data packets from a sensor that outputs the plurality of data packets at a sampling frequency;

assigning a second plurality of system timestamps to the plurality of data packets, the second plurality of system timestamps being of the second time base; and

assigning a third plurality of timestamps of the first time base to the plurality of data packets based at least in part upon the first plurality of system timestamps assigned to the plurality of images and the second plurality of system timestamps to the plurality of data packets.

18. The computer-readable data storage device of claim 17 being affixed to an automobile.

19. The computer-readable data storage device of claim 17, wherein the sensor is a Global Positioning system unit.

20. The computer-readable data storage device of claim 17, wherein the plurality of timestamps of the first time base are received from an external trigger board that transmits signals that respectively cause a camera to capture the of images.