Method of restoring movements of the line of sight of an optical instrument

• US 8,068,696 B2
• Filed: 04/03/2007
• Issued: 11/29/2011
• Est. Priority Date: 04/03/2006
• Status: Active Grant

• Current Assignee: Astrium Sas (Airbus SE)
• Original Assignee: Astrium Sas (Airbus SE)
• Inventors: Polverini, Umberto, Flandin, Gregory, Crombez, Vincent
• Primary Examiner(s): Repko; Jason M
• Assistant Examiner(s): Thirugnanam; Gandhi
• Application Number:
  US11/695,896
• Publication Number:
  US 20080031528A1
• Time in Patent Office: 1,701 Days
• Field of Search: 382/284, 382/294
• US Class Current: 382/294
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area

Abstract

A method of estimation of the direction of the line of sight comprising forming at least two images representing one and the same portion of the scene (P) at two successive instants using an instrument mounted on a moving carrier, and the determination of the change of orientation of the line of sight by matching homologous points (A1, A2) in the two images of the same portion of scene by calculating the angular variations in attitude between the two instants at which the two successive images are obtained and by correcting on the basis of the deviation, between the estimate of the point (A2) homologous in the image provided by a model, and the actual position of the homologous point, and integrating and time filtering pertaining to a sequence of several successive determinations of attitude variations, leading to an estimate of the direction of the line of sight.

Claims

1. Method of restoring the movements of a line of sight (V) in terms of roll, pitch and yaw of an observation instrument during an image sequence, according to which, at least a first and a second image containing one and the same portion of a scene being formed, the first and the second images being formed over the picture-taking duration [Ti, Tf] on the basis of the observation instrument mounted on a moving carrier, the takes of the first and the second images being offset in time by a substantially constant known value T;

Tn being the average instant of acquisition of a chosen image n, n being the indice of an image, n going from i to f, i being the indice of the first image, f being the indice of the last image, T being the temporal deviation between the acquisition of two images;

said movements of the line of sight (V) of the observation instrument in the course of the image sequence being determined by at least the following steps;

a) a step of selecting several characteristic zones in a first image and matching these zones with homologous zones in the second image, by textural or radiometric analysis, these homologous zones being taken in the vicinity of instants respectively Tn and Tn+T distributed between Ti and Tf in the said images of the same portion of the scene so as to obtain a measurement of the position at the instants respectively Tn and Tn+T of the said homologous zones, or of at least a particular point of these zones, in each of the two images of the said portion,

b) a step of determining an image matching model by a geometric modeling of with an optical system;

said image matching model giving for any point of a first image shot at the instant Tn, an estimate of the position of its homologous point in a second image shot at the instant Tn+T ;

said image matching model being a function of, in particular, the orientation of the line of sight (V) of the instrument at the instant Tn and angular variations of this line of sight (V) between Tn and Tn +T;

wherein the first and the second images have been acquired in push-broom mode and wherein the method further comprises the following steps;

c) a step of calculating the angular variations of the line of sight (V) of the observation instrument between the instants Tn and Tn+T by adjusting said image matching model, the adjustment of said image matching model consisting in calculating the values of the angular variations which minimize a function of the deviations between the position of the homologous zones of the second image that are predicted by the image matching model, and the position of the homologous zones of the second image as it is matched in step (a), and

d) a step of reconstructing the movements of the line of sight (V) over the time horizon [Ti, Tf] by integrating and filtering the angular variations calculated in step (c).

2. The restoring method according to claim 1, wherein step (a) implements a matching of characteristic zones of images that is based on a radiometric or textural resemblance between at least two images acquired during the image sequence.

3. The restoring method according to claim 1, wherein at least one of the following procedures i) to iv) is used to select the characteristic zones associated with a given image sequence date Tn:

i) selection of the points with double gradient greater than a given threshold in a zone of the image corresponding to the image sequence taken in the vicinity of the instant Tn;

ii) selection of points having positions that are predetermined independently of the content of the image in the vicinity of the selected points, with possible rejection of points for which a matching quality criterion is below a given threshold;

iii) selection of predetermined points, given by a priori knowledge of particular elements present in the scene;

iv) selection of a subset of points on the basis of a set of characteristic points calculated by one of the above procedures or a similar procedure, so as to maximize the spatial distribution of the selected subset of points, and the quality of matching of the characteristic zones in the vicinity of the selected subset of points;

this selection procedure being followed by a selection of a vignette in the vicinity of the points selected according to any of the procedures i) to iv) so as to improve the quality of matching.

4. The restoring method according to claim 1, wherein the matched images are taken in spectral bands that may be identical or different.

5. The restoring method according to claim 1, wherein in step (c), the deviations involved in the function to be minimized are weighted by a positive scalar proportional to the estimated quality of the said matching of the characteristic zones between the two images.

6. The restoring method according to claim 1, wherein in the course of step (d), an integration and a filtering of the attitude variation estimates arising from step (c) are performed so as to reconstruct the movements of the line of sight of the instrument, apart from absolute component of the orientation of sighting, over the time interval [Ti, Tf] by at least one of the following procedures:

using an inverse filtering, making it possible to enhance certain attenuated frequencies while limiting the impact of noise,

by fitting an analytical model.

7. The restoring method according to claim 1, wherein in the absence of sufficient quality matching of the characteristic zones selected at the instant Tn, the orientation of the line of sight at the instant Tn is advantageously calculated by interpolating the line-of-sight orientations reconstructed by repeating steps a) to d) of claim 1 at instants preceding and following Tn.

8. The restoring method according to claim 6 wherein the a priori knowledge of the frequency content of the variations of the orientation of the line of sight is used to perform a filtering prior to step (d) that retains only the frequencies concerned.

9. The restoring method according to claim 1, wherein a number of images of one and the same scene that is strictly greater than two is used, steps a) to d) of claim 1 being applied to several pairs of images and the results obtained from the various pairs of images of the same scene are hybridized by implementing a global or iterative refinement of the current knowledge of the orientation of the line of sight during the image sequence.

10. The restoring method according to claim 1, wherein steps a) to d) of claim 1 is applied to a first pair of images advantageously from among the pairs of images that are assumed to be best correlated, so as to obtain a first reconstruction of the movements of the line of sight of the observation instrument in the course of the image sequence, then other pairs of images are iteratively processed, initializing the matching model on the basis of the line-of-sight orientation values calculated in the previous iteration, until the pairs of images are exhausted or convergence is obtained.

11. The restoring method according to claim 1, wherein the instrument makes it possible to form at least three images offset over time, at least two of the images exhibiting a strong mutual correlation, and at least one other image that is weakly correlated with the other two, and that the movements of the line of sight are corrected on at least one image that is weakly correlated with the other two by reconstructing the movements of the line of sight by repeating steps a) to d) of claim 1 to at least one pair of images exhibiting a strong mutual correlation.

12. The restoring method according to claim 1, wherein the matching model is initialized with a priori knowledge of the orientation of the line of sight in the course of the image sequence, the said a priori knowledge arising from an orientation setpoint or from a measurement derived from attitude sensors such as star sensors, gyrometers, or from a knowledge of the absolute position on the surface of the terrestrial globe of characteristic zones recognizable in the image, or a hybridization of such knowledge.

13. The restoring method according to claim 1, wherein parallax effects are taken into account in the matching model, using a priori knowledge of the relief of the scenes observed.

14. An apparatus for restoring the movements of a line of sight (V) of an observation instrument onboard a satellite, wherein:

the observation instrument comprises in its focal plane at least one strip-type detector of one to a few tens of lines, each line comprising a few thousand to ten thousand pixels,

the satellite is placed in a non-geostationary orbit,

the satellite is pointed in such a way that the at least one strip-type detector scans a zone to be observed during image sequences, and

a processor configured for performing the restoring method according to claim 1 to estimate the orientation of the line of sight (V) of the observation instrument in the course of an image sequence.

15. An apparatus for restoring the movements of a line of sight (V) of an observation instrument onboard a satellite, wherein:

the observation instrument comprises in its focal plane at least one strip-type detector,

the line of sight (V) of the instrument is controlled in such a way that the at least one detector scans the surface of a terrestrial globe, either by rotation of the satellite, or by the scanning of an orientatable mirror placed in the optical path of the instrument, and

a processor configured for performing the restoring method according to claim 1 to estimate the orientation of the line of sight (V) of the observation instrument during an image sequence.

16. The apparatus according to claim 15, wherein the orbit of the satellite is geostationary.

17. The apparatus according to claim 15, wherein

the instrument comprises in its focal plane at least two strip-type detectors, the strips being aligned in such a way that their lines and columns are respectively substantially parallel, and the two strips of each pair of strips, at least one pair of which is thus in the focal plane, being separated from one another by a determined distance in the focal plane, the detectors thus observing approximately the same portion of the scene during an image sequence, with a time offset depending directly on the distance between the detectors at the focal plane.

18. The apparatus according to claim 15, wherein each line of the at least one strip-type detector of the focal plane forms an image of the scene, and that a subset of the images thus created by the at least one strip-type detector of the focal plane can be processed according to the method for restoring the orientation of the line of sight (V) of the instrument.

19. The apparatus according to claim 17, wherein the separation of the detectors in the focal plane is optimized with the aim of reconstructing the movements of the line of sight (V) in a given frequency band.

20. The apparatus according to claim 14, wherein there is just one detector at the focal plane scanning the scene, and that the processor is configured for performing the restoring method to at least two images formed on the basis of at least two lines of the detector.

21. The apparatus according to claim 20 wherein, the choice of the at least two lines of the detector that are used to form the at least two images is optimized with the aim of reconstructing the movements of the line of sight (V) in a given frequency band.

22. The restoring method according to claim 7, characterized in that a priori knowledge of the frequency content of the variations of the orientation of the line of sight is used to perform a filtering prior to step (c) that retains only the frequencies concerned.

23. The apparatus according to claim 14, wherein

the instrument comprises in its focal plane at least two strip-type detectors, the strips being aligned in such a way that their lines and columns are respectively substantially parallel, and the two strips of each pair of strips, at least one pair of which is thus in the focal plane, being separated from one another by a determined distance in the focal plane, the detectors thus observing approximately the same portion of the scene during an image sequence, with a time offset depending directly on the distance between the detectors at the focal plane.

24. The apparatus according to claim 14, wherein each line of the at least one strip-type detector of the focal plane forms an image of the scene, and that a subset of the images thus created by the at least one strip-type detector of the focal plane can be processed according to the method for restoring the orientation of the line of sight (V) of the instrument.

25. The apparatus according to claim 23, wherein the separation of the detectors in the focal plane is optimized with the aim of reconstructing the movements of the line of sight (V) in a given frequency band.

26. The apparatus according to claim 15, wherein there is just one detector at the focal plane scanning the scene, and that the processor is configured for performing the restoring method to at least two images formed on the basis of at least two lines of the detector.

27. The apparatus according to claim 26 wherein, the choice of the at least two lines of the detector that are used to form the at least two images is optimized with the aim of reconstructing the movements of the line of sight (V) in a given frequency band.