SYNCHRONIZING SENSOR DATA ACROSS DEVICES

• US 20120163520A1
• Filed: 12/27/2010
• Published: 06/28/2012
• Est. Priority Date: 12/27/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Liu, Shao, Finocchio, Mark, Bar-Zeev, Avi, Margolis, Jeffrey, Flaks, Jason, Crocco, Robert Jr., Kipman, Alex Aben-Athar
• Granted Patent: US 9,116,220 B2
• Time in Patent Office: Days
• US Class Current: 375/356
• CPC Class Codes: A63F 2300/10 characterized by input arrangements for converting player-generated signals into game device control signals
• CPC Class Codes: G01D 21/00 Measuring or testing not otherwise provided for
• CPC Class Codes: G01S 19/03 Cooperating elements; Interaction or communication between different cooperating elements or between cooperating elements and receivers
• CPC Class Codes: G01S 5/0018 Transmission from mobile station to base station
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: H04J 3/0667 Bidirectional timestamps, e.g. NTP or PTP for compensation of clock drift and for compensation of propagation delays arrangements for monitoring round trip delays in packet switching networks H04L43/0864
• CPC Class Codes: H04W 84/18 Self-organising networks, e.g. ad-hoc networks or sensor networks

Abstract

Techniques are provided for synchronization of sensor signals between devices. One or more of the devices may collect sensor data. The device may create a sensor signal from the sensor data, which it may make available to other devices upon a publisher/subscriber model. The other devices may subscribe to sensor signals they choose. A device could be a provider or a consumer of the sensor signals. A device may have a layer of code between an operating system and software applications that processes the data for the applications. The processing may include such actions as synchronizing the data in a sensor signal to a local time clock, predicting future values for data in a sensor signal, and providing data samples for a sensor signal at a frequency that an application requests, among other actions.

Claims

1. A method comprising:

determining, by a first device, which sensor signals are available over one or more networks, each of the sensor signals is available from another device that has a sensor from which the sensor signal is derived;

subscribing, by the first device, to a first sensor signal of the sensor signals;

receiving the first sensor signal at the first device over the one or more networks, the first sensor signal including timing information and data based on sensor data; and

processing the data in the first sensor signal to make it compatible for an application running on first device, the processing performed by code other than the application.

2. The method of claim 1, wherein the processing the data includes synchronizing the data in the first sensor signal to a local time clock of the first device.

3. The method of claim 1, wherein the signals include a first type of signal for continuous signals and a second type of signal for state signals.

4. The method of claim 3, wherein the first signal is a continuous signal, and wherein the processing the data includes interpolating the continuous signal to make the data compatible with the application.

5. The method of claim 3, wherein the first signal is a continuous signal, and wherein the processing the data includes extrapolating the data in the continuous signal.

6. The method of claim 5, wherein the extrapolating the data includes determining values that are limited based on physical properties of the subject matter represented by the continuous signal.

7. The method of claim 5, wherein the extrapolating the data generates a first prediction, and further comprising:

receiving one or more additional data samples in the first signal;

extrapolating the continuous signal based at least on the additional data samples to form a second prediction; and

gradually merging the first prediction to the second prediction.

8. The method of claim 1, further comprising:

requesting to subscribe to a second signal of the signals by the second device;

receiving the second signal at the second device, the second signal including timing information and data based on sensor data; and

synchronizing in time the data in the first signal with the data in the second signal.

9. The method of claim 1, further comprising determining a common time reference between the first device and the other device that provides the first sensor signal.

10. A method comprising:

publishing that one or more sensor signals are available from a first device, the one or more sensor signals are based on sensor data;

subscribing a second device to a first signal of the one or more signal;

collecting sensor data at the first device;

creating the first signal based on the sensor data, the first signal includes timing information associated with the collected data;

providing the first signal from the first device over one or more network to the second device.

11. The method of claim 10, wherein publishing that one or more sensor signals are available includes publishing what type of the signal each of the sensor signals is, the type is one of a plurality of types that includes a continuous signal type and a state signal type.

12. The method of claim 10, further comprising:

receiving a request that the second device subscribes to a second sensor signal of the one or more sensor signals;

collecting sensor data for the second signal at the first device;

creating the second sensor signal from sensor data for the second signal, the second sensor signal includes timing information and data based on the sensor data for the second sensor signal; and

providing the second sensor signal from the first device over one or more network for the second device.

13. The method of claim 10, wherein the creating the first sensor signal includes:

determining a user requested command based on the sensor data; and

creating one or more packets that contain the command and a time stamp for the time that the command was requested by the user.

14. The method of claim 10, wherein the providing the first sensor signal from the first device over one or more network for the second device includes the first device providing the first signal directly to the second device.

15. The method of claim 10, wherein the providing the providing the first signal from the first device over one or more network for the second device includes:

the first device providing the first sensor signal to a network server; and

the network server forwarding the first sensor signal to the second device.

16. An apparatus, comprising:

a processor; and

a computer readable storage medium coupled to the processor, the computer readable storage medium having instructions stored thereon which, when executed on the processor, cause the processor to;

determine which sensor signals are available over one or more networks, the sensor signals are based on sensor data, the sensor signals include a first type of signal for continuous signals and a second type of signal for state signals;

subscribe to a first sensor signal of the sensor signals;

establish a common time reference with a device that publishes the first sensor signal;

receive the first sensor signal over the one or more networks, the first sensor signal including timing information and data based on sensor data;

synchronize the data in the first signal to a local clock using the common time reference; and

provide the synchronized data to an application.

17. The apparatus of claim 16, wherein the instructions that cause the processor to synchronize the data include instructions that cause the processor to:

access samples in the first sensor signal;

access a time stamp in the first sensor signal for each of the samples;

convert the time stamp to local time; and

store each of the samples in association with the respective local time for each sample.

18. The apparatus of claim 16, wherein the instructions that cause the processor to synchronize the data include instructions that cause the processor to:

access commands in the first sensor signal;

access a time stamp in the first sensor signal for each of the commands;

convert the time stamp to local time; and

store each of the commands in association with the respective local time for each command.

19. The apparatus of claim 16, wherein the instructions that cause the processor to provide the synchronized data to an application include instructions that cause the processor to:

receive a request from an application for state information within a specified time period;

access the stored signal data to find state information within the given time period; and

provide the state information for the given time period to the application.

20. The apparatus of claim 16, wherein the instructions that cause the processor to provide the synchronized data to an application include instructions that cause the processor to:

receive a request from an application to detect an event;

monitor the first signal for occurrence of the event; and

report to the application in response to detecting that the first sensor signal indicates that the event occurred.