System And Method For Stabilizing An Image

• US 20080136923A1
• Filed: 11/06/2005
• Published: 06/12/2008
• Est. Priority Date: 11/14/2004
• Status: Active Grant

• Current Assignee: Elbit Systems Limited
• Original Assignee: Elbit Systems Limited
• Inventors: Inbar, Shamir, Davidov, Gavriel
• Granted Patent: US 7,932,925 B2
• Time in Patent Office: Days
• US Class Current: 348/208.2
• CPC Class Codes: G02B 27/646 compensating for small deviations, e.g. due to vibration or shake movement of one or more optical elements for control of motion blur in cameras, projectors or printers G03B2205/0007; image stabilisation in cameras peculiar to the presence or use of an electronic image sensor H04N23/68
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6842 by controlling the scanning position, e.g. windowing
• CPC Class Codes: H04N 23/687 by shifting the lens or sensor position

Abstract

System for producing a stabilized image of a scene viewed by an image detector, the image detector detecting a plurality of images of the scene, the image detector being supported by gimbals, the gimbals including a servo system for moving the gimbals at least one degree of freedom, the system including an angular movement detection device firmly coupled with the image detector, and a processor coupled with the angular movement detection device and with the image detector, the angular movement detection device detecting at least a portion of angular movements of the image detector about at least one axis of rotation, the angular movement detection device producing an angular output respective of the portion, the processor receiving the images from the image detector, and the angular output from the angular movement detection device, the processor determining at least one shifting displacement and at least one shifting direction for shifting a current image detected by the image detector, relative to a reference image of the scene detected by the image detector, according to the angular output, the processor producing the stabilized image, by shifting the current image by the shifting displacement and in the shifting direction.

Claims

1. System for producing a stabilized image of a scene viewed by an image detector, the image detector detecting a plurality of images of the scene, the image detector being supported by gimbals, the gimbals including a servo system for moving the gimbals at least one degree of freedom, the system comprising:

an angular movement detection device firmly coupled with said image detector, said angular movement detection device detecting at least a portion of angular movements of said image detector about at least one axis of rotation, said angular movement detection device producing an angular output respective of said at least one portion; and

a processor coupled with said angular movement detection device and with said image detector, said processor receiving said images from said image detector, and said angular output from said angular movement detection device, said processor determining at least one shifting displacement and at least one shifting direction for shifting a current image detected by said image detector, relative to a reference image of said scene detected by said image detector, according to said angular output, said processor producing said stabilized image, by shifting said current image by said at least one shifting displacement and in said at least one shifting direction.

2. The system according to claim 1, wherein said at least one portion is selected from the list consisting of:

pitch;

yaw; and

roll.

3. The system according to claim 1, wherein said processor receives from said angular movement detection device, a reference angular output respective of said at least one portion when said image detector detects said reference image,

wherein said processor associates said reference angular output with said reference image,

wherein said processor receives from said angular movement detection device, a current angular output respective of said at least one portion when said image detector detects said current image,

wherein said processor associates said current angular output with said current image, and

wherein said processor determines said at least one shifting displacement and said at least one shifting direction, by comparing said current angular output with said reference angular output.

4. The system according to claim 1, wherein said reference image is an image of said scene detected by said image detector, substantially coincident with an activation event of said system.

5. The system according to claim 1, wherein said reference image is one of said images detected by said image detector, before detecting said current image.

6. The system according to claim 1, wherein said angular output is respective of a parameter selected from the list consisting of:

angular displacement;

angular velocity; and

linear acceleration.

7. The system according to claim 1, wherein said angular movement detection device is a device selected from the list consisting of:

electromechanical free gyro;

rate gyro;

rate integrating gyro;

dynamically tuned gyro;

Coriolis gyro;

interferometric fiber optic gyro;

ring laser gyro;

inertial measurement unit;

inertial navigation system; and

inertial sensor.

8. The system according to claim 1, wherein said processor determines said at least one shifting displacement according to the characteristics of an optical assembly associated with said image detector, and

wherein said at least one shifting direction is substantially opposite to at least one other respective direction associated with a respective one of said at least one portion.

9. The system according to claim 1, wherein said processor comprises:

an angular movement filter coupled with said angular movement detection device, said angular output having an angular velocity unit, said angular movement filter receiving said angular output, said angular movement filter producing a filtered angular output by substantially filtering out from said angular output, spontaneously generated noise respective of said angular movement detection device;

an integrator coupled with said angular movement filter, said integrator receiving said filtered angular output, said integrator producing an integrated filtered angular output having an angular displacement unit, by performing an integration operation on said filtered angular output; and

an image processor coupled with said image detector and with said integrator, said image processor receiving said images from said image detector, said image processor receiving said integrated filtered angular output from said integrator, said image processor determining said at least one shifting displacement and said at least one shifting direction according to said integrated filtered angular output, said image processor shifting said current image by said at least one shifting displacement and in said at least one shifting direction.

10. The system according to claim 9, wherein said angular movement filter further receives at least one control signal, and

wherein said angular movement filter produces said filtered angular output, by substantially filtering out said at least one control signal from said angular output.

11. The system according to claim 10, further comprising:

a scene tracker coupled with said image processor, said servo system of said gimbals, and with said angular movement filter, said scene tracker tracking said scene by processing said stabilized image, said scene tracker sending a scene tracking signal to said servo system to move said image detector for tracking said scene, said scene tracker sending said scene tracking signal to said angular movement filter; and

a target tracker coupled with said image processor, said servo system, and with said angular movement filter, said target tracker tracking a moving target within said scene by processing said stabilized image, said target tracker sending a target tracking signal to said servo system to move said image detector for tracking said moving target in said scene, said target tracker sending said target tracking signal to said angular movement filter,

wherein said at least one control signal includes at least one of said scene tracking signal and said target tracking signal.

12. The system according to claim 10, further comprising a directional stabilizer coupled with said servo system of said gimbals and with said angular movement filter, said directional stabilizer detecting at least one of pitch, yaw and roll of said image detector, said directional stabilizer sending a directional corrective signal to said servo system of said gimbals to move said image detector to compensate for at least one of said pitch, yaw and roll, said directional stabilizer sending said directional corrective signal to said angular movement filter,

wherein said at least one control signal includes said directional corrective signal.

13. The system according to claim 10, further comprising a user interface coupled with said servo system of said gimbals and with said angular movement filter, said user interface producing a movement command signal to move said image detector, according to a user command received from a user, said user interface sending said movement command signal to said servo system to move said image detector according to said user command, said user interface sending said movement command signal to said angular movement filter,

wherein said at least one control signal includes said movement command signal.

14. The system according to claim 13, wherein said user interface is selected from the list consisting of:

joystick; and

coordinate input device.

15. The system according to claim 9, wherein said angular movement filter comprises:

a coordinate transformation module coupled with said angular movement detection device at a junction, said coordinate transformation module receiving said at least one control signal, said coordinate transformation module producing a transformed signal by transforming said at least one control signal, from a respective at least one coordinate system associated with said at least one control signal, to a coordinate system associated with said image detector, said transformed signal to be subtracted from said angular output to produce a control-free signal; and

a noise filter coupled with said angular movement detection device and with said coordinate transformation module at said junction, and further coupled with said integrator, said noise filter producing said filtered angular output, by filtering out spontaneously generated noise respective of said angular movement detection device, from said control-free signal.

16. The system according to claim 15, wherein said spontaneously generated noise is selected from the list consisting of:

calibration factor instability factor;

temperature dependent parameters;

depended

mounting misalignment of angular detector on the said gimbals;

angle random walk;

bias instability;

rate random walk;

rate ramp;

Random drift;

quantization noise;

correlated noise; and

sinusoidal noise.

17. The system according to claim 15, wherein said noise filter is selected from the list consisting of:

Kalman filter;

extended Kalman filter;

unscented Kalman filter;

recursive least square algorithm; and

interacting multiple model filter.

18. The system according to claim 9, wherein said image processor comprises:

a movement processing module coupled with said integrator, said movement processing module determining said at least one shifting displacement and said at least one shifting direction, according to said integrated filtered angular output; and

an image transformation module, coupled with said image detector and with said movement processing module, said image transformation module shifting said current image by said at least one shifting displacement and in said at least one shifting direction.

19. The system according to claim 18, further comprising a range finder coupled with said image transformation module, said range finder determining the range between said image detector and said scene,

wherein said image transformation module modifies a scale of said current image according to movement of said image detector toward or away from said scene.

20. The system according to claim 18, wherein said image processor further comprises an image summing module coupled with said image transformation module, said image summing module adding together a series of images produced by said image transformation module.

21. The system according to claim 18, wherein said image processor further comprises a buffer coupled with said image transformation module, said buffer accumulating a series of images produced by said image transformation module.

22. The system according to claim 18, wherein said image processor further comprises an image enhancement module coupled with said image transformation module, said image enhancement module performing an enhancement operation on said image.

23. The system according to claim 22, wherein enhancement operation is selected from the list consisting of:

varying the contrast of said image; and

varying the brightness of said image.

24. The system according to claim 1, wherein said processor comprises:

an angular movement filter coupled with said angular movement detection device, said angular output having an angular displacement unit, said angular movement filter producing a filtered angular output by filtering out from said angular output, spontaneously generated noise respective of said angular movement detection device; and

an image processor coupled with said image detector and with said angular movement filter, said image processor receiving said images from said image detector, said image processor receiving said filtered angular output from said angular movement filter, said image processor determining said at least one shifting displacement and said at least one shifting direction, according to said filtered angular output, said image processor shifting said current image by said at least one shifting displacement and in said at least one shifting direction.

25. The system according to claim 1, wherein said processor produces said stabilized image for a device selected from the list consisting of:

display;

memory; and

communication interface.

26. The system according to claim 1, wherein said image detector is selected from the list consisting of:

forward looking infrared camera;

gated camera;

television camera; and

high definition television camera.

27. The system according to claim 1, wherein said image detector is selected from the list consisting of:

charge-coupled device; and

complementary metal oxide semiconductor.

28. The system according to claim 1, further comprising:

another angular movement detection device coupled with said image detector, said other angular movement detection device detecting at least another portion of said angular movements, said other angular movement detection device producing another angular output according to said at least other portion; and

another processor coupled with said other angular movement detection device and with said servo system, said other processor determining a compensating signal according to said at other angular output, for compensating for said at least other portion, said servo system moving said image detector according to said compensating signal, by at least another displacement proportional to said at least other portion, and in at least one opposite direction.

29. The system according to claim 28, wherein said angular movement detection device detects said at least one portion at a first predetermined range of at least one angular parameter, and

wherein said other angular movement detection device detects said at least other portion at a second predetermined range of said at least one angular parameter.

30. The system according to claim 28, wherein said angular movement detection device detects said at least one portion at a first predetermined range of angular frequencies, and

wherein said other angular movement detection device detects said at least other portion at a second predetermined range of angular frequencies.

31. The system according to claim 28, wherein a single angular movement detection device comprises said angular movement detection device and said other angular movement detection device.

32. The system according to claim 1, wherein said image detector, said gimbals, said angular movement detection device and said processor are coupled with a vehicle.

33. The system according to claim 32, wherein said vehicle is selected from the list consisting of:

flying vehicle;

ground vehicle; and

marine vehicle.

34. The system according to claim 1, wherein said image detector, said gimbals, said angular movement detection device and said processor are coupled with an observation post.

35. The system according to claim 1, wherein said image detector, said gimbals, said angular movement detection device and said processor are coupled with a portable device.

36. The system according to claim 1, wherein said at least one axis of rotation is respective of said image detector.

37. The system according to claim 1, wherein said at least one axis of rotation is respective of said gimbals.

38. The system according to claim 1, further comprising another angular movement detection device firmly coupled with said gimbals, said other angular movement detection device detecting at least another portion of said angular movements, and producing a respective other angular output,

wherein said servo system receives said respective other angular output from said other angular movement detection device, and

wherein said servo system moves said gimbals according to said respective other angular output, to stabilize said gimbals against said at least other portion.

61. System for producing a stabilized image of a scene viewed by an image detector, according to any of claims 1-38 and 57 substantially as described hereinabove.

62. System for producing a stabilized image of a scene viewed by an image detector, according to any of claims 1-38 and 57 substantially as illustrated in any of the drawings.

39. Method for producing a stabilized image of a scene viewed by an image detector, the method comprising the procedures of:

producing a reference angular output respective of at least a portion of angular movements of said image detector, said reference angular output being associated with a reference image of said scene detected by said image detector;

producing a current angular output respective of said at least one portion, said current angular output being associated with a current image of said scene detected by said image detector;

determining at least one shifting displacement and at least one shifting direction, for shifting said current image relative to said reference image, by comparing said current angular output with said reference angular output; and

shifting said current image by said at least one shifting displacement and in said at least one shifting direction.

40. The method according to claim 39, further comprising preliminary procedures of:

detecting said at least one portion; and

detecting a plurality of images of said scene by said image detector.

41. The method according to claim 39, wherein said at least one portion is selected from the list consisting of:

pitch;

yaw; and

roll.

42. The method according to claim 39, wherein said at least one shifting direction is substantially opposite to respective at least one other direction, associated with a respective one of said at least one portion.

43. The method according to claim 39, further comprising a procedure of producing a transformed signal by transforming at least one control signal, from a respective at least one coordinate system associated with said at least one control signal, to a coordinate system associated with said image detector, before performing said procedures of producing each of said reference angular output and said current angular output.

44. The method according to claim 43, wherein each of said reference angular output and said current angular output is included in an angular output produced by an angular movement detection device, detecting said at least one portion.

45. The method according to claim 44, further comprising a procedure of producing a control-free signal by subtracting said transformed signal from said angular output, before performing said procedures of producing each of said reference angular output and said current angular output.

46. The method according to claim 45, further comprising a procedure of producing a filtered angular output, by substantially filtering out from said control-free signal, spontaneously generated noise, respective of said angular movement detection device, before performing said procedures of producing each of said reference angular output and said current angular output.

47. The method according to claim 46, further comprising a procedure of integrating said filtered angular output, at least once.

48. The method according to claim 43, wherein before performing said procedure of determining, said method comprises the procedures of:

processing said stabilized image for tracking said scene;

producing a scene tracking signal; and

sending said scene tracking signal to a servo system of gimbals, said gimbals being firmly coupled with said image detector,

wherein said at least one control signal includes said scene tracking signal.

49. The method according to claim 43, wherein before performing said procedure of determining, said method comprises the procedures of:

processing said stabilized image for tracking a moving target within said scene;

producing a target tracking signal; and

sending said target tracking signal to a servo system of gimbals, said gimbals being firmly coupled with said image detector,

wherein said at least one control signal includes said target tracking signal.

50. The method according to claim 43, wherein before performing said procedure of determining, said method comprises the procedures of:

processing said stabilized image for stabilizing said image detector against at least one of pitch, yaw and roll of said image detector;

producing a directional corrective signal; and

sending said directional corrective signal to a servo system of gimbals, said gimbals being firmly coupled with said image detector,

wherein said at least one control signal includes said directional corrective signal.

51. The method according to claim 43, wherein before performing said procedure of determining, said method comprises the procedures of:

producing a movement command signal according to a user command, for moving said image detector; and

sending said movement command signal to a servo system of gimbals, said gimbals being firmly coupled with said image detector,

wherein said at least one control signal includes said movement command signal.

52. The method according to claim 39, further comprising the procedures of:

determining a range between said image detector and said scene; and

modifying magnification of said stabilized image according to the movement of said image detector toward or away from said scene.

53. The method according to claim 39, further comprising a procedure of stabilizing gimbals by a servo system coupled with said gimbals, according to at least another portion of said angular movements, before performing said procedure of determining, said gimbals being firmly coupled with said image detector.

54. The method according to claim 39, further comprising the procedures of:

repeating said procedures of determining and shifting for a set of images; and

adding together said set of shifted images to produce a superposed image.

55. The method according to claim 39, further comprising the procedures of:

repeating said procedures of determining and shifting for a set of images; and

accumulating said set of shifted images in a buffer.

56. The method according to claim 39, further comprising a procedure of performing an enhancement operation on said image.

63. Method for producing a stabilized image of a scene viewed by an image detector, according to any of claims 39-56 substantially as described hereinabove.

64. Method for producing a stabilized image of a scene viewed by an image detector, according to any of claims 39-56 substantially as illustrated in any of the drawings.

57. System for producing a stabilized image of a scene viewed by an image detector, the image detector detecting a plurality of images of the scene, the image detector being supported by gimbals, the gimbals including a servo system for moving the gimbals at least one degree of freedom, the system comprising:

a first angular movement detection device firmly coupled with said image detector, said first angular movement detection device detecting at least a portion of angular movements of said image detector about at least one axis of rotation, said first angular movement detection device producing a first angular output respective of said at least one portion;

a processor coupled with said first angular movement detection device and with said image detector, said processor receiving said images from said image detector, and said first angular output from said first angular movement detection device, said processor determining at least one first shifting displacement and at least one first shifting direction for shifting a current image detected by said image detector, relative to a reference image of said scene detected by said image detector, according to said first angular output, said processor producing said stabilized image, by shifting said current image by said at least one first shifting displacement and in said at least one first shifting direction, said processor detecting at least one displacement of said current image relative to a previous image included in said images, in at least one direction, said processor determining at least one second shifting displacement to shift said current image relative to said previous image, in at least one second shifting direction substantially opposite to said at least one direction, said processor shifting said current image by said at least one second shifting displacement, in said at least one second direction;

a second angular movement detection device coupled with said image detector, said second angular movement detection device detecting at least another portion of said angular movements, said second angular movement detection device producing a second angular output respective of said at least other portion; and

a movement processor coupled with said second angular movement detection device and with said servo system, said movement processor determining at least one compensating movement for said image detector, according to said second angular output, said servo system moving said image detector according to said at least one compensating movement.

58. Device for producing a stabilized image of a scene viewed by an image detector, the image detector detecting a plurality of images of the scene, the image detector being supported by gimbals, the gimbals including a servo system, a first angular movement detection device, and a movement processor, the servo system moving the gimbals at least one degree of freedom, the first angular movement detection device detecting at least one portion of angular movements of the image detector, the angular movement detection device producing a first angular output respective of the at least one portion, the movement processor determining at least one compensating movement for the image detector, according to the first angular output, the servo system moving the image detector according to the at least one compensating movement, the system further including a first processor receiving the images from the image detector, the first processor detecting at least one displacement of a current image of the scene detected by the image detector, relative to a previous image included in the images, in at least one direction, the first processor determining at least one first shifting displacement to shift the current image relative to the previous image, in at least one first shifting direction substantially opposite to the at least one direction, the first processor shifting the current image by the at least one first shifting displacement, in the at least one first direction, the device comprising:

a second angular movement detection device firmly coupled with said image detector, said second angular movement detection device detecting at least another portion of said angular movements, said second angular movement detection device producing a second angular output respective of said at least other portion; and

a second processor coupled with said second angular movement detection device, said image detector, and with said first processor, said second processor receiving said images from said image detector, and said second angular output from said second angular movement detection device, said second processor determining at least one second shifting displacement and at least one second shifting direction for shifting said current image, relative to a reference image of said scene detected by said image detector, according to said second angular output, said second processor producing said stabilized image, by shifting said current image by said at least one second shifting displacement and in said at least one second shifting direction.

59. Apparatus for viewing a substantially blur-free image of a scene, the apparatus comprising:

an optical assembly; and

a system for producing a stabilized image of said scene viewed by an image detector, said image detector detecting a plurality of images of said scene, said image detector being supported by gimbals, said gimbals including a servo system for moving said gimbals at least one degree of freedom, said apparatus comprising;

an angular movement detection device firmly coupled with said image detector, said angular movement detection device detecting at least a portion of angular movements of said image detector about at least one axis of rotation, said angular movement detection device producing an angular output respective of said at least one portion; and

a processor coupled with said angular movement detection device and with said image detector, said processor receiving said images from said image detector, and said angular output from said angular movement detection device, said processor determining at least one shifting displacement and at least one shifting direction for shifting a current image detected by said image detector, relative to a reference image of said scene detected by said image detector, according to said angular output, said processor producing said stabilized image, by shifting said current image by said at least one shifting displacement and in said at least one shifting direction.

60. The apparatus according to claim 59, wherein said optical assembly is selected from the list consisting of:

microscope;

binoculars;

head-up display;

head-mounted display;

night vision system;

telescope;

periscope;

light amplifier; and

a combination of above.