SYSTEMS AND METHODS FOR SYNCHRONIZING SENSOR DATA

• US 20170359518A1
• Filed: 06/10/2016
• Published: 12/14/2017
• Est. Priority Date: 06/10/2016
• Status: Active Grant

• Current Assignee: Movea
• Original Assignee: Movea
• Inventors: de Foras, Etienne, Hall, Daniela
• Granted Patent: US 10,506,163 B2
• Time in Patent Office: Days
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/687 by shifting the lens or sensor position
• CPC Class Codes: H04N 5/04 Synchronising for television systems using pulse code modulation H04N7/56

Abstract

A method for synchronizing data provided by an image sensor and a motion sensor rigidly linked together, comprising the steps of:

• • obtaining image sensor data and motion sensor data;
  • processing at least one of the image sensor data and the motion sensor data;
  • extracting motion characteristics from filtering said processed data, wherein said filtering comprises at least applying a high pass filter; and
    synchronizing the image sensor data and the motion sensor data provided by the rigidly linked sensors using synchronization data derived from identification of substantially similar motion characteristics in the filtered image sensor data and the filtered motion sensor data.

Claims

1. A method for synchronizing data provided by an image sensor and a motion sensor rigidly linked together, comprising the steps of:

obtaining image sensor data and motion sensor data;

processing at least one of the image sensor data and the motion sensor data;

extracting motion characteristics from filtering said processed data, wherein said filtering comprises at least applying a high pass filter; and

synchronizing the image sensor data and the motion sensor data provided by the rigidly linked sensors using synchronization data derived from identification of substantially similar motion characteristics in the filtered image sensor data and filtered the motion sensor data.

2. The method of claim 1, wherein the step of synchronizing is obtained by applying a correction based on the synchronization data to at least one of the image sensor data and the motion sensor data.

3. The method of claim 2, wherein the correction consists of applying a time delay to at least one of the image sensor data and the motion sensor data.

4. The method of claim 2, wherein the correction consists of modifying a time interval between at least two successive data samples from at least one of the image sensor data and the motion sensor data.

5. The method of claim 2, wherein the correction comprises an interpolation of one of the image sensor data and the motion sensor data.

6. The method of claim 2, wherein the image sensor data and the motion sensor data comprise timestamp data, and wherein the step of synchronizing consists in modifying the timestamp data for at least one of the image sensor data and the motion sensor data.

7. The method of claim 1, wherein the synchronization data is derived using a pattern matching of the motion characteristics of the image sensor data and the motion sensor data.

8. The method of claim 1, wherein the motion sensor has a plurality of measuring axes, and the step of processing processes the motion sensor data to determine a motion vector.

9. The method of claim 8, further comprising the step of determining a norm of the motion vector.

10. The method of claim 1, wherein the image sensor produces a plurality of image frames, and the step of processing of the image sensor data comprises determining displacement data related to points of interest in the image derived from a comparison of at least two image frames.

11. The method of claim 10, wherein a motion vector is derived based on the displacement data.

13. The sensor synchronization unit of claim 11, further configured to provide the synchronized data to an image stabilization system.

15. The system of claim 11, further comprising a memory for buffering at least one of the image sensor data and the motion sensor data.

16. The system of claim 11, wherein the motion sensor comprises a gyroscope.

17. The system of claim 11, wherein the motion sensor comprises a gyroscope and an accelerometer, and the motion data is based on an orientation determined based on fusion of at least the gyroscope data and accelerometer data.

12. A sensor synchronization unit, comprising an image sensor and a motion sensor (12) rigidly linked together, configured to:

obtain image data from the image sensor;

obtain motion data from the motion sensor;

process at least one of the image sensor data and the motion sensor data;

extract motion characteristics from filtering said processed data, wherein said filtering comprises at least applying a high pass filter; and

synchronize the image sensor data and the motion sensor data provided by the rigidly linked sensors using synchronization data derived from identification of substantially similar motion characteristics in the filtered image sensor data and the filtered motion sensor data.

14. A system for synchronizing image sensor data and motion sensor data, comprising:

an image sensor;

a motion sensor rigidly linked to the image sensor; and

a processor configure to;

obtain image data from the image sensor;

obtain motion data from the motion sensor;

process at least one of the image sensor data and the motion sensor data;

filter the processed data, wherein said filtering comprises applying a high-pass filter, and;

synchronize the image sensor data and the motion sensor data provided by the rigidly linked sensors using synchronization data derived from identification of substantially similar motion characteristics in the filtered image sensor data and the filtered motion sensor data.