Synchronization of multiple data source to a common time base

• US 7,974,314 B2
• Filed: 01/16/2009
• Issued: 07/05/2011
• Est. Priority Date: 01/16/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Curlander, John Charles, Neuhold, Gerhard, Kroepfl, Michael, Pehserl, Joachim, Bernögger, Stefan, Ponticelli, Martin Josef, Kimchi, Gur
• Primary Examiner(s): Vu; Huy D
• Assistant Examiner(s): Lamont; Benjamin
• Application Number:
  US12/354,793
• Publication Number:
  US 20100183034A1
• Time in Patent Office: 900 Days
• Field of Search: 375/356, 375/371, 375/373, 375/376, 370/395.62, 370503-509
• US Class Current: 370/503
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks

Abstract

Systems and methods are described herein that cause data from asynchronous data sources to be provided with a timestamp that corresponds to a common time base. A trigger board can be used to control synchronized data sources, and can generate timestamps when data is collected by the synchronized data sources. Unsynchronized data sources can generate data independent of the trigger board. System timestamps are generated each time data from the synchronized data source and the unsynchronized data source is received. Values of the system timestamp can be modified, and can be replaced by timestamps that correspond to the time base used by the trigger board.

Claims

1. A method for causing data from a plurality of data sources to have a common time base, comprising the following computer executable acts:

receiving a first plurality of trigger timestamps from a trigger component, wherein the trigger component generates the first plurality of trigger timestamps using a first time base, wherein the trigger component causes a first data source to capture data, and wherein the trigger component generates a trigger timestamp each time the trigger component causes the first data source to capture data;

assigning a first plurality of system timestamps to the plurality of trigger timestamps, wherein the system timestamps are generated using a second time base that is independent of the first time base, and wherein each trigger timestamp in the plurality of trigger timestamps is assigned a system timestamp;

receiving a first plurality of data packets from a second data source, wherein the second data source is configured to capture data independent of the trigger component and wherein the second data source is one of a distance measurement instrument, an inertial navigation system, or a velocity sensor;

assigning a second plurality of system timestamps to the first plurality of data packets, wherein each data packet in the first plurality of data packets is assigned a system timestamp;

comparing the first plurality of trigger timestamps with the first plurality of system timestamps;

determining a correlation between the first time base and the second time base based at least in part upon the comparison; and

assigning a plurality of timestamps that conform to the first time base to the first plurality of data packets based at least in part upon the determined correlation between the first time base and the second time base.

2. The method of claim 1, further comprising:

receiving a second plurality of data packets from the first data source;

assigning a third plurality of system timestamps to the second plurality of data packets, wherein each data packet in the second plurality of data packets is assigned a system timestamp; and

assigning the first plurality of trigger timestamps to the second plurality of data packets, wherein each data packet in the second plurality of data packets is assigned a trigger timestamp.

3. The method of claim 2, further comprising placing an image of faç

ade of a building on a computer-implemented three-dimensional representation of the building based at least in part upon the first plurality of data packets and the second plurality of data packets.

4. The method of claim 2, further comprising:

determining an earliest timestamp in the second plurality of system timestamps;

determining an earliest timestamp in the third plurality of system timestamps; and

comparing the earliest timestamp in the second plurality of timestamps with the earliest timestamp in the third plurality of timestamps; and

removing at least one data packet from one of the first plurality of data packets or the second plurality of data packets based at least in part upon the comparison.

5. The method of claim 1, further comprising:

determining that a trigger timestamp in the first plurality of trigger timestamps fails to correspond to a data packet in the second plurality of data packets;

generating a data packet to correspond to the trigger timestamp in the first plurality of trigger timestamps; and

marking the data packet as invalid.

6. The method of claim 1, wherein the first data source is a digital camera.

7. The method of claim 6, wherein the first data source and the second data source are fixed to an automobile.

8. The method of claim 1, wherein the second data source captures data packets at a frequency that is independent of the first time base and the second time base, and further comprising:

determining the frequency that the second data source captures data packets; and

adjusting the second plurality of timestamps assigned to the first plurality of data packets based at least in part upon the determined frequency.

9. The method of claim 8, wherein at least some of the second plurality of system timestamps are identical, and wherein adjusting the second plurality of timestamps comprises using interpolation to cause at least some of the plurality of system timestamps to be adjusted to conform to the determined frequency.

10. The method of claim 9, wherein adjusting the second plurality of system timestamps comprises adjusting at least some of the plurality of timestamps backwards in time.

11. The method of claim 1, wherein the first plurality of data packets and the second plurality of data packets are received by way of a serial interface of a computing device.

12. The method of claim 1, wherein the first data source is one of a global positioning system sensor or a near infrared camera.

13. The method of claim 1, further comprising generating a three-dimensional representation of a geographic region based at least in part upon data from the first data source and the second data source.

14. A system that facilitates corresponding synchronized and unsynchronized data to a common time base, comprising:

a synchronized data source that generates a first data packet in response to receipt of a trigger signal;

a trigger component that transmits the trigger signal to the synchronized data source and generates a trigger timestamp using a first time base when the trigger signal is transmitted to the synchronized data source;

an unsynchronized data source that generates a second data packet independent of the trigger component, wherein the unsynchronized data source is one of a distance measurement instrument, an inertial navigation system, or a velocity sensor;

a system timestamp generator component that;

a) receives the trigger timestamp from the trigger component and assigns a first system timestamp to the trigger timestamp, wherein the first system timestamp corresponds to a second time base that is independent of the first time base; and

b) receives the second data packet from the unsynchronized data source and assigns a second system timestamp to the second data packet, wherein the second system timestamp corresponds to the second time base; and

a synchronizer component that;

a) compares the trigger timestamp with the first system timestamp; and

b) assigns a timestamp to the second data packet that conforms to the first time base based at least in part upon the comparison.

15. The system of claim 14, further comprising a frequency sampler component that samples data packets output by the unsynchronized data source and determines an output frequency of the unsynchronized data source.

16. The system of claim 15, further comprising a replacer component that modifies the second system timestamp based at least in part upon the determined output frequency of the unsynchronized data source.

17. The system of claim 16, further comprising a shifter component that further modifies the second system timestamp backwards in time based at least in part upon the determined output frequency of the unsynchronized data source.

18. The system of claim 17, further comprising a match component that compares system timestamps corresponding to data packets output by the synchronized data source and the unsynchronized data source and automatically removes at least one data packet from one of the synchronized data source or the unsynchronized data source based at least in part upon the comparison.

19. The system of claim 14, wherein the synchronized data source is a digital camera, and wherein the synchronized and unsynchronized sources are mounted to an automobile.

20. A computer-readable memory comprising instructions that, when executed by a processor, cause the processor to perform acts comprising:

receiving a first data packet from a first data source, wherein the first data source is a digital camera;

receiving a trigger timestamp that corresponds to the first data packet from an external trigger board, wherein the trigger timestamp corresponds to a first time base, and wherein the trigger board transmits the trigger timestamp in response to causing the first data source to capture the first data packet;

assigning a first system timestamp to the first data packet, wherein the first system timestamp corresponds to a second time base that is independent of the first time base;

correlating the first data packet and the first system timestamp with the trigger timestamp;

receiving a second data packet from a second data source, wherein the second data source transmits the second data packet independent of the trigger board, wherein the second data source is one of a velocity sensor, a distance measurement unit, or an inertial navigation system;

assigning a second system timestamp to the second data packet, wherein the second system timestamp corresponds to the second time base;

performing a comparison between the first system timestamp with the trigger timestamp to determine a correlation between the first time base and the second time base; and

changing a value of the second system timestamp to correspond to the first time base based at least in part upon the comparison.