METHOD AND DEVICE FOR PROCESSING HIGH-FREQUENCY MOVEMENTS IN AN OPTRONIC SYSTEM

• US 20170214854A1
• Filed: 07/15/2015
• Published: 07/27/2017
• Est. Priority Date: 07/18/2014
• Status: Active Grant

• Current Assignee: Safran Electronics and Defense SAS (Safran SA)
• Original Assignee: Safran Electronics and Defense SAS (Safran SA)
• Inventors: REYMOND, Georges-Olivier
• Granted Patent: US 10,003,748 B2
• Time in Patent Office: Days
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/685 performed by mechanical compensation
• CPC Class Codes: H04N 23/687 by shifting the lens or sensor position

Abstract

A method for processing high-frequency movements in an optronic system including an image sensor functioning with a first acquisition frequency, each image obtained by the image sensor representing a scene, the method includes a step of obtaining signal values representing a movement of the scene from a high-frequency movement sensor functioning with a second acquisition frequency higher than the first acquisition frequency, a step of determining values representing a movement of an image in the course of acquisition by the image sensor from values of the signals representing the movement of the scene, and a step of transmitting the values representing the movement of the image in the course of acquisition by the image sensor to a movement compensation device so that the movement compensation device can compensate for the movement of the image in the course of acquisition by the image sensor.

Claims

1. A method for processing high-frequency movements in an optronic system comprising an image sensor functioning with a first signal-acquisition frequency, referred to as the image frequency, each image obtained by the image sensor representing a scene, wherein the method comprises:

obtaining signal values representing a movement of said scene from a high-frequency movement sensor, the high-frequency movement sensor generating signal values representing movements in said scene with a second signal-acquisition frequency, referred to as the movement frequency, higher than the image frequency;

determining values representing a movement in an image in the course of acquisition by the image sensor from values of the signals representing the movement of said scene, the determination of the values representing the movement of said scene, the determination of the values representing the movement of the image in the course of acquisition by the image sensor comprises a matrix operation between the values of the signals representing the movement of said scene and a conversion matrix;

transmitting the values representing the movement of the image in the course of acquisition by the image sensor thus determined to a movement compensation device so that said movement compensation device can implement feedback in the optronic system in order to compensate for the movement of the image in the course of acquisition by the image sensor, the feedback being implemented in the optronic system with a frequency less than or equal to the movement frequency;

an inverse matrix of the conversion matrix being determined by a determination method comprising;

obtaining a reference image from an original image acquired by the image sensor,

simulating first values of the signals representing a movement obtained by the high-frequency movement sensor when the high-frequency movement sensor is subjected to a light beam corresponding to the reference image,

applying movements of predetermined movement values to the reference image in order to obtain a set of moved images,

for each moved image in the set of moved images, simulating second values of the signals representing a movement obtained by the high-frequency movement sensor when the high-frequency movement sensor is subjected to a light beam corresponding to the moved image, and

determining the inverse matrix of the conversion matrix from first and second values of the signals representing a movement obtained by the high-frequency movement sensor and predetermined movement values.

2. The method according to claim 1, wherein the high-frequency movement sensor is a multi-element photodiode.

3. The method according to claim 1, wherein the conversion matrix is updated periodically at a frequency lower than or equal to the image frequency.

4. The method according to claim 1, wherein, for each moved reference image, the predetermined movement values comprise a horizontal movement value and a vertical movement value.

5. The method according to claim 1, wherein, for each moved reference image, the predetermined movement values comprise a movement direction value and a movement amplitude value.

6. The method according to claim 1, wherein the set of moved reference images comprises at least two images.

7. The method according to claim 1, wherein the reference image results from an application of a sub-pixel interpolation to the original image obtained by the image sensor.

11. A non-transitory information storage medium storing a computer program product containing instructions for implementing, by a device, the method according to claim 1, when said program is executed by a processor of said device.

12. A non-transitory information storage medium storing a computer program containing instructions for implementing, by a device, the method according to claim 1 when said program is executed by a processor of said device.

8. A device for processing high-frequency movements in an optronic system comprising an image sensor functioning with a first signal-acquisition frequency, referred to as the image frequency, each image obtained by the image sensor representing a scene, wherein the device being configured for:

obtaining signal values representing a movement of said scene from a high-frequency movement sensor, the high-frequency movement sensor generating signal values representing movements in said scene with a second signal-acquisition frequency, referred to as the movement frequency, higher than the image frequency,

determining values representing a movement of an image in the course of acquisition by the image sensor from values of the signals representing the movement of said scene, the means for determining the values representing the movement of the image in the course of acquisition by the image sensor comprising means for performing a matrix operation between the values of the signals representing the movement of said scene and a conversion matrix;

transmitting determined values representing the movement of the image in the course of acquisition by the image sensor to a movement compensation device so that said movement compensation device can implement feedback in the optronic system in order to compensate for the movement of the image in the course of acquisition by the image sensor, the feedback being implemented by the optronic system with a frequency less than or equal to the movement frequency;

said device being further configured for determining an inverse matrix of the conversion matrix by

obtaining a reference image from an original image acquired by the image sensor,

simulating first values of the signals representing a movement obtained by the high-frequency movement sensor when the high-frequency movement sensor is subjected to a light beam corresponding to the reference image,

applying movements of predetermined movement values to the reference image in order to obtain a set of moved images,

simulating, for each moved image of the set of moved images, second signal values representing a movement obtained by the high-frequency movement sensor when the high-frequency movement sensor is subjected to a light beam corresponding to the moved image, and

determining the inverse matrix of the conversion matrix from first and second values of the signals representing a movement obtained by the high-frequency movement sensor and predetermined movement values.

9. The device according to claim 8, wherein a correction filter is inserted in front of the image sensor and/or in front of the high-frequency sensor in order to compensate for any difference between a spectral response of the image sensor and a spectral response of the high-frequency movement sensor.

10. An optronic system comprising an image sensor functioning with a first signal-acquisition frequency, each image obtained by the image sensor representing a scene, and a movement compensation device, wherein the system comprises:

a high-frequency movement sensor generating signal values representing movements of said scene with a second signal-acquisition frequency higher than the first signal-acquisition frequency, and

a device for processing high-frequency movements according to claim 8 able to control the movement compensation device according to said signal values representing movements.