System and method for stabilizing an image

• US 7,932,925 B2
• Filed: 11/06/2005
• Issued: 04/26/2011
• Est. Priority Date: 11/14/2004
• Status: Active Grant

• Current Assignee: Elbit Systems Limited
• Original Assignee: Elbit Systems Limited
• Inventors: Inbar, Shamir, Davidov, Gavriel
• Primary Examiner(s): HERNANDEZ, NELSON D
• Application Number:
  US11/667,737
• Publication Number:
  US 20080136923A1
• Time in Patent Office: 1,997 Days
• Field of Search: 348/208.99, 3482081-20816, 348/143, 348/154, 348/155, 348169-172
• US Class Current: 348/208.2
• CPC Class Codes: G02B 27/646 compensating for small deviations, e.g. due to vibration or shake movement of one or more optical elements for control of motion blur in cameras, projectors or printers G03B2205/0007; image stabilisation in cameras peculiar to the presence or use of an electronic image sensor H04N23/68
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6842 by controlling the scanning position, e.g. windowing
• CPC Class Codes: H04N 23/687 by shifting the lens or sensor position

Abstract

System for producing a stabilized image of a scene viewed by an image detector, the image detector detecting a plurality of images of the scene, the image detector being supported by gimbals, the gimbals including a servo system for moving the gimbals at least one degree of freedom, the system including an angular movement detection device firmly coupled with the image detector, and a processor coupled with the angular movement detection device and with the image detector, the angular movement detection device detecting at least a portion of angular movements of the image detector about at least one axis of rotation, the angular movement detection device producing an angular output respective of the portion, the processor receiving the images from the image detector, and the angular output from the angular movement detection device, the processor determining at least one shifting displacement and at least one shifting direction for shifting a current image detected by the image detector, relative to a reference image of the scene detected by the image detector, according to the angular output, the processor producing the stabilized image, by shifting the current image by the shifting displacement and in the shifting direction.

Claims

1. System for producing a stabilized image of a scene viewed by an image detector, the image detector detecting a plurality of images of the scene, the image detector being supported by gimbals, the gimbals including a servo system for moving the gimbals at at least one degree of freedom, the system comprising:

an angular movement detection device firmly coupled with said image detector, said angular movement detection device detecting at least a portion of angular movements of said image detector about at least one axis of rotation, said angular movement detection device producing an angular output respective of said at least one portion;

an angular movement filter coupled with said angular movement detection device, said angular movement filter receiving said angular output, said angular movement filter producing a filtered angular output by substantially filtering out from said angular output, spontaneously generated noise respective of said angular movement detection device, wherein said angular movement filter further receives at least one control signal, and wherein said angular movement filter produces said filtered angular output, by substantially filtering out said at least one control signal from said angular output;

a processor coupled with said angular movement filter and with said image detector, said processor receiving said images from said image detector, and said filtered angular output from said angular movement filter, said processor determining at least one shifting displacement and at least one shifting direction for shifting a current image detected by said image detector, relative to a reference image of said scene detected by said image detector, according to said filtered angular output, said processor producing said stabilized image, by shifting said current image by said at least one shifting displacement and in said at least one shifting direction;

wherein said processor comprises;

an integrator coupled with said angular movement filter, said integrator receiving said filtered angular output, said integrator producing an integrated filtered angular output, by performing an integration operation on said filtered angular output; and

an image processor coupled with said image detector and with said integrator, said image processor receiving said images from said image detector, said image processor receiving said integrated filtered angular output from said integrator, said image processor determining said at least one shifting displacement and said at least one shifting direction according to said integrated filtered angular output, said image processor shifting said current image by said at least one shifting displacement and in said at least one shifting direction;

a scene tracker coupled with said image processor, said servo system of said gimbals, and with said angular movement filter, said scene tracker tracking said scene by processing said stabilized image, said scene tracker sending a scene tracking signal to said servo system to move said image detector for tracking said scene, said scene tracker sending said scene tracking signal to said angular movement filter; and

a target tracker coupled with said image processor, said servo system, and with said angular movement filter, said target tracker tracking a moving target within said scene by processing said stabilized image, said target tracker sending a target tracking signal to said servo system to move said image detector for tracking said moving target in said scene, said target tracker sending said target tracking signal to said angular movement filter,

wherein said at least one control signal includes at least one of said scene tracking signal and said target tracking signal.

2. The system according to claim 1, wherein said at least one portion is selected from the list consisting of:

pitch;

yaw; and

roll.

3. The system according to claim 1, wherein said processor receives from said angular movement filter, a filtered reference angular output respective of said at least one portion when said image detector detects said reference image,

wherein said processor associates said filtered reference angular output with said reference image,

wherein said processor receives from said angular movement filter, a filtered current angular output respective of said at least one portion when said image detector detects said current image,

wherein said processor associates said filtered current angular output with said current image, and

wherein said processor determines said at least one shifting displacement and said at least one shifting direction, by comparing said filtered current angular output with said filtered reference angular output.

4. The system according to claim 1, wherein said reference image is an image of said scene detected by said image detector, substantially coincident with an activation event of said system.

5. The system according to claim 1, wherein said reference image is one of said images detected by said image detector, before detecting said current image.

6. The system according to claim 1, wherein said angular output is respective of a parameter selected from the list consisting of:

angular displacement;

angular velocity; and

linear acceleration.

7. The system according to claim 1, wherein said angular movement detection device is a device selected from the list consisting of:

electromechanical free gyro;

rate gyro;

rate integrating gyro;

dynamically tuned gyro;

Coriolis gyro;

interferometric fiber optic gyro;

ring laser gyro;

inertial measurement unit;

inertial navigation system; and

inertial sensor.

8. The system according to claim 1, wherein said processor determines said at least one shifting displacement according to the characteristics of an optical assembly associated with said image detector, and

wherein said at least one shifting direction is substantially opposite to at least one other respective direction associated with a respective one of said at least one portion.

9. The system according to claim 1, further comprising a directional stabilizer coupled with said servo system of said gimbals and with said angular movement filter, said directional stabilizer detecting at least one of pitch, yaw and roll of said image detector, said directional stabilizer sending a directional corrective signal to said servo system of said gimbals to move said image detector to compensate for at least one of said pitch, yaw and roll, said directional stabilizer sending said directional corrective signal to said angular movement filter, wherein said at least one control signal includes said directional corrective signal.

10. The system according to claim 1, further comprising a user interface coupled with said servo system of said gimbals and with said angular movement filter, said user interface producing a movement command signal to move said image detector, according to a user command received from a user, said user interface sending said movement command signal to said servo system to move said image detector according to said user command, said user interface sending said movement command signal to said angular movement filter,

wherein said at least one control signal includes said movement command signal.

11. The system according to claim 10, wherein said user interface is selected from the list consisting of:

joystick; and

coordinate input device.

12. The system according to claim 1, wherein said angular movement filter comprises:

a coordinate transformation module coupled with said angular movement detection device at a junction, said coordinate transformation module receiving said at least one control signal, said coordinate transformation module producing a transformed signal by transforming said at least one control signal, from a respective at least one coordinate system associated with said at least one control signal, to a coordinate system associated with said image detector, said transformed signal to be subtracted from said angular output to produce a control-free signal; and

a noise filter coupled with said angular movement detection device and with said coordinate transformation module at said junction, and further coupled with said integrator, said noise filter producing said filtered angular output, by filtering out spontaneously generated noise respective of said angular movement detection device, from said control-free signal.

13. The system according to claim 12, wherein any of said noise filter and said angular movement filter is selected from the list consisting of:

Kalman filter;

extended Kalman filter;

unscented Kalman filter;

recursive least square algorithm; and

interacting multiple model filter.

14. The system according to claim 1, wherein said spontaneously generated noise is selected from the list consisting of:

calibration factor instability factor;

temperature dependent parameters;

mounting misalignment of angular detector on the said gimbals;

angle random walk;

bias instability;

rate random walk;

rate ramp;

Random drift;

quantization noise;

correlated noise; and

sinusoidal noise.

15. The system according to claim 1, wherein said image processor comprises:

a movement processing module coupled with said integrator, said movement processing module determining said at least one shifting displacement and said at least one shifting direction, according to said integrated filtered angular output; and

an image transformation module, coupled with said image detector and with said movement processing module, said image transformation module shifting said current image by said at least one shifting displacement and in said at least one shifting direction.

16. The system according to claim 15, further comprising a range finder coupled with said image transformation module, said range finder determining the range between said image detector and said scene,

wherein said image transformation module modifies a scale of said current image according to movement of said image detector toward or away from said scene.

17. The system according to claim 15, wherein said image processor further comprises an image summing module coupled with said image transformation module, said image summing module adding together a series of images produced by said image transformation module.

18. The system according to claim 15, wherein said image processor further comprises a buffer coupled with said image transformation module, said buffer accumulating a series of images produced by said image transformation module.

19. The system according to claim 15, wherein said image processor further comprises an image enhancement module coupled with said image transformation module, said image enhancement module performing an enhancement operation on said image.

20. The system according to claim 19, wherein enhancement operation is selected from the list consisting of:

varying the contrast of said image; and

varying the brightness of said image.

21. The system according to claim 1, wherein said processor produces said stabilized image for a device selected from the list consisting of:

display;

memory; and

communication interface.

22. The system according to claim 1, wherein said image detector is selected from the list consisting of:

forward looking infrared camera;

gated camera;

television camera; and

high definition television camera.

23. The system according to claim 1, wherein said image detector is selected from the list consisting of:

charge-coupled device; and

complementary metal oxide semiconductor.

24. The system according to claim 1, further comprising:

another angular movement detection device coupled with said image detector, said other angular movement detection device detecting at least another portion of said angular movements, said other angular movement detection device producing another angular output according to said at least other portion; and

another processor coupled with said other angular movement detection device and with said servo system, said other processor determining a compensating signal according to said at other angular output, for compensating for said at least other portion, said servo system moving said image detector according to said compensating signal, by at least another displacement proportional to said at least other portion, and in at least one opposite direction.

25. The system according to claim 24, wherein said angular movement detection device detects said at least one portion at a first predetermined range of at least one angular parameter, and

wherein said other angular movement detection device detects said at least other portion at a second predetermined range of said at least one angular parameter.

26. The system according to claim 24, wherein said angular movement detection device detects said at least one portion at a first predetermined range of angular frequencies, and

wherein said other angular movement detection device detects said at least other portion at a second predetermined range of angular frequencies.

27. The system according to claim 24, wherein a single angular movement detection device comprises said angular movement detection device and said other angular movement detection device.

28. The system according to claim 1, wherein said system is coupled with a vehicle.

29. The system according to claim 28, wherein said vehicle is selected from the list consisting of:

flying vehicle;

ground vehicle; and

marine vehicle.

30. The system according to claim 1, wherein said system is coupled with an observation post.

31. The system according to claim 1, wherein said system is coupled with a portable device.

32. The system according to claim 1, wherein said at least one axis of rotation is respective of said image detector.

33. The system according to claim 1, wherein said at least one axis of rotation is respective of said gimbals.

34. The system according to claim 1, further comprising another angular movement detection device firmly coupled with said gimbals, said other angular movement detection device detecting at least another portion of said angular movements, and producing a respective other angular output,

wherein said servo system receives said respective other angular output from said other angular movement detection device, and

wherein said servo system moves said gimbals according to said respective other angular output, to stabilize said gimbals against said at least other portion.

35. Method for producing a stabilized image of a scene viewed by an image detector, the method comprising the procedures of:

producing a transformed signal by transforming at least one control signal, from a respective at least one coordinate system associated with said at least one control signal, to a coordinate system associated with said image detector;

producing a control-free signal by subtracting said transformed signal from said angular output;

producing a filtered angular output, by substantially filtering out from said control-free signal, spontaneously generated noise, respective of said angular movement detection device;

producing a reference angular output respective of at least a portion of angular movements of said image detector, said reference angular output being associated with a reference image of said scene detected by said image detector;

producing a current angular output respective of said at least one portion, said current angular output being associated with a current image of said scene detected by said image detector, wherein each of said reference angular output and said current angular output is included in an angular output produced by an angular movement detection device, detecting said at least one portion;

producing a filtered reference angular output and a filtered current angular output, by substantially filtering out from said reference angular output and said current angular output, spontaneously generated noise, respective of said angular movement detection device;

determining at least one shifting displacement and at least one shifting direction, for shifting said current image relative to said reference image, by comparing said filtered current angular output with said filtered reference angular output; and

shifting said current image by said at least one shifting displacement and in said at least one shifting direction.

36. The method according to claim 35, further comprising preliminary procedures of:

detecting said at least one portion; and

detecting a plurality of images of said scene by said image detector.

37. The method according to claim 35, wherein said at least one portion is selected from the list consisting of:

pitch;

yaw; and

roll.

38. The method according to claim 35, wherein said at least one shifting direction is substantially opposite to respective at least one other direction, associated with a respective one of said at least one portion.

39. The method according to claim 35, further comprising a procedure of integrating said filtered angular output, at least once.

40. The method according to claim 35, wherein before performing said procedure of determining, said method comprises the procedures of:

processing said stabilized image for tracking said scene;

producing a scene tracking signal; and

sending said scene tracking signal to a servo system of gimbals, said gimbals being firmly coupled with said image detector,

wherein said at least one control signal includes said scene tracking signal.

41. The method according to claim 35, wherein before performing said procedure of determining, said method comprises the procedures of:

processing said stabilized image for tracking a moving target within said scene;

producing a target tracking signal; and

sending said target tracking signal to a servo system of gimbals, said gimbals being firmly coupled with said image detector,

wherein said at least one control signal includes said target tracking signal.

42. The method according to claim 35, wherein before performing said procedure of determining, said method comprises the procedures of:

processing said stabilized image for stabilizing said image detector against at least one of pitch, yaw and roll of said image detector;

producing a directional corrective signal; and

sending said directional corrective signal to a servo system of gimbals, said gimbals being firmly coupled with said image detector,

wherein said at least one control signal includes said directional corrective signal.

43. The method according to claim 35, wherein before performing said procedure of determining, said method comprises the procedures of:

producing a movement command signal according to a user command, for moving said image detector; and

sending said movement command signal to a servo system of gimbals, said gimbals being firmly coupled with said image detector,

wherein said at least one control signal includes said movement command signal.

44. The method according to claim 35, further comprising the procedures of:

determining a range between said image detector and said scene; and

modifying magnification of said stabilized image according to the movement of said image detector toward or away from said scene.

45. The method according to claim 35, further comprising a procedure of stabilizing gimbals by a servo system coupled with said gimbals, according to at least another portion of said angular movements, before performing said procedure of determining, said gimbals being firmly coupled with said image detector.

46. The method according to claim 35, further comprising the procedures of:

repeating said procedures of determining and shifting for a set of images; and

adding together said set of shifted images to produce a superposed image.

47. The method according to claim 35, further comprising the procedures of:

repeating said procedures of determining and shifting for a set of images; and

accumulating said set of shifted images in a buffer.

48. The method according to claim 35, further comprising a procedure of performing an enhancement operation on said image.

49. Apparatus for viewing a substantially blur-free image of a scene, the apparatus comprising:

an optical assembly; and

a system for producing a stabilized image of said scene viewed by an image detector, said image detector detecting a plurality of images of said scene, said image detector being supported by gimbals, said gimbals including a servo system for moving said gimbals at at least one degree of freedom, said apparatus comprising;

an angular movement detection device firmly coupled with said image detector, said angular movement detection device detecting at least a portion of angular movements of said image detector about at least one axis of rotation, said angular movement detection device producing an angular output respective of said at least one portion;

an angular movement filter coupled with said angular movement detection device, said angular movement filter receiving said angular output, said angular movement filter producing a filtered angular output by substantially filtering out from said angular output, spontaneously generated noise respective of said angular movement detection device, wherein said angular movement filter further receives at least one control signal, and wherein said angular movement filter produces said filtered angular output, by substantially filtered out said at least one control signal from said angular output;

a processor coupled with said angular movement filter and with said image detector, said processor receiving said images from said image detector, and said angular output from said angular movement filter, said processor determining at least one shifting displacement and at least one shifting direction for shifting a current image detected by said image detector, relative to a reference image of said scene detected by said image detector, according to said filtered angular output, said processor producing said stabilized image, by shifting said current image by said at least one shifting displacement and in said at least one shifting direction;

wherein said processor comprises;

an integrator coupled with said angular movement filter, said integrator receiving said filtered angular output, said integrator producing an integrated filtered angular output, by performing an integration operation on said filtered angular output; and

an image processor coupled with said image detector and with said integrator, said image processor receiving said images from said image detector, said image processor receiving said integrated filtered angular output from said integrator, said image processor determining said at least one shifting displacement and said at least one shifting direction according to said integrated filtered angular output, said image processor shifting said current image by said at least one shifting displacement and in said at least one shifting direction;

a scene tracker coupled with said image processor, said servo system of said gimbals, and with said angular movement filter, said scene tracker tracking said scene by processing said stabilized image, said scene tracker sending a scene tracking signal to said servo system to move said image detector for tracking said scene, said scene tracker sending said scene tracking signal to said angular movement filter; and

a target tracker coupled with said image processor, said servo system, and with said angular movement filter, said target tracker tracking a moving target within said scene by processing said stabilized image, said target tracker sending a target tracking signal to said servo system to move said image detector for tracking said moving target in said scene, said target tracker sending said target tracking signal to said angular movement filter,

wherein said at least one control signal includes at least one of said scene tracking signal and said target tracking signal.

50. The apparatus according to claim 49, wherein said optical assembly is selected from the list consisting of:

microscope;

binoculars;

head-up display;

head-mounted display;

night vision system;

telescope;

periscope;

light amplifier; and

a combination of above.