Automatic image stabilization device

• US 5,107,293 A
• Filed: 09/06/1989
• Issued: 04/21/1992
• Est. Priority Date: 09/09/1988
• Status: Expired due to Term

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Noguchi, Kazuhiro, Takahashi, Koji, Sekine, Masayoshi, Nakajima, Toshiyuki, Toyama, Masamichi
• Primary Examiner(s): Hix, L. T.
• Assistant Examiner(s): Gray, David M.
• Application Number:
  US07/403,455
• Time in Patent Office: 958 Days
• Field of Search: 354/430, 354/70, 354/65, 358/222, 358/227, 358/213.11, 350/500
• US Class Current: 396/55
• CPC Class Codes: G02B 27/646 compensating for small deviations, e.g. due to vibration or shake movement of one or more optical elements for control of motion blur in cameras, projectors or printers G03B2205/0007; image stabilisation in cameras peculiar to the presence or use of an electronic image sensor H04N23/68
• CPC Class Codes: G02B 7/36 using image sharpness techniques , e.g. image processing techniques for generating autofocus signals image data processing per se G06T; in cameras having a solid state image sensor H04N23/67
• CPC Class Codes: G03B 2217/005 Blur detection
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/675 comprising setting of focusing regions
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/6815 by distinguishing pan or tilt from motion
• CPC Class Codes: H04N 23/686 with a variable apex prism
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

An automatic image stabilization device which is arranged to detect image movement information from a sensed image signal produced from an image sensor and to reduce the image movement comprises a movement detecting circuit which obtains movement vector information for each of a plurality of parts of an image plane, and a weight setting circuit arranged to attach weight to each movement vector information and to vary the weighting degree according to photographing conditions. A camera using this device is arranged to correct the shaking of the camera on the basis of the result of a computing operation performed on a camera shake detected from the signal produced from the image sensor together with a camera shake physically detected by means of an accelerometer.

Claims

1. An image sensing apparatus arranged to detect information on an image movement through an image signal from image sensing means and to reduce said image movement on the basis of said movement information, comprising:

a) movement detecting means for obtaining vector information on image movements taking place at a plurality of points on an image plane;

b) weight setting means arranged to attach weight to said vector information obtained from each of said points and to vary said weight according to photographing conditions; and

c) computing means for computing and obtaining the degree of image movement on the basis of movement vector information weighted by said weight setting means.

2. An apparatus according to claim 1, further comprising correction means for offsetting said image movement according to the output of said computing means.

3. An apparatus according to claim 2, further comprising a photo-taking optical system which is arranged to form an object'"'"'s image on said image sensing means, and wherein said correction means includes optical axis varying means for varying the optical axis of said photo-taking optical system.

4. An apparatus according to claim 3, wherein said optical axis varying means is a variable angle prism.

5. An apparatus according to claim 1, wherein said movement detecting means is arranged to divide said image plane into a plurality of blocks, to compute the vector of movement taking place within each of said blocks and to detect parts showing movement on said image plane, the magnitude and direction of said movement.

6. An apparatus according to claim 2, wherein said weight setting means is arranged to change the weight to be attached to said movement vector information according to each of a plurality of photographing modes.

7. An apparatus according to claim 6, further comprising automatic focusing means arranged to detect the focused state of the object'"'"'s image formed on said image sensing means and to automatically maintain a photo taking-optical system in an in-focus state.

8. An apparatus according to claim 7, wherein said weight setting means is arranged to change the degree of weight to be attached to said blocks of said image plane according to the output of said automatic focusing means and the photographing mode of said apparatus.

9. An apparatus according to claim 8, wherein said automatic focusing means includes manual focusing means, a fixed area focusing means arranged to automatically adjust a focus of said optical system to an object'"'"'s image obtained within a focus detecting area fixedly set on said image plane, and object tracing means for adjusting the focus to an object'"'"'s image by shifting a focus detecting area set on said image plane in such a way as to trace the movement of said object'"'"'s image taking place within said image plane.

10. An apparatus according to claim 9, wherein said fixed area focusing means is arranged to fixedly set said focus detecting area in a central part of said image plane.

11. An apparatus according to claim 9, wherein said object tracing means includes setting means for setting an object tracing area within said image plane in a shiftable manner and control means for controlling the setting position of said object tracing area in a direction in which a difference in luminance between the inside and outside of said object tracing area increases, and wherein said focus detecting area is set in the position of said object tracing area.

12. An apparatus according to claim 9, wherein said weight setting means is arranged to attach weight to the movement vector of a peripheral part of said image plane to a greater degree than to that of a central part of said image plane when the manual focusing means of said automatic focusing means is selected.

13. An apparatus according to claim 9, wherein said weight setting means is arranged to set the movement vector weighting degree at zero for the inside of said focus detecting area and at an increased value for the outside of said area when said fixed area focusing means is selected.

14. An apparatus according to claim 9, wherein said weight setting means is arranged to set the movement vector weighting degree at zero for the inside of said focus detecting area in each of the shifted positions thereof and at an increased value for the outside of said area when said object tracing means is selected.

15. An apparatus according to claim 9, wherein said weight setting means is arranged to set the movement vector weighting degree at an increased value for the inside of said focus detecting area and at zero for the outside of said area when the apparatus is moved to trace a moving object with said focus detecting area fixed.

16. An automatic image stabilization device arranged to detect the shaking of images through an image signal produced from image sensing means and to correct said shaking of images, comprising:

a) movement detecting means for obtaining information on image movements taking place at a plurality of points of an image sensing plane;

b) mode discriminating means for discriminating photographing modes from each other; and

c) weight setting means arranged to attach weight to information on movement taking place in each of blocks set on said image sensing plane and to vary the degree of weight according to the output of said mode discriminating means;

d) computing means for computing the degree of image movement on the basis of movement information which is weighted by said weight setting means; and

e) correction means including a correction optical system which is arranged to offset said image movement according to the output of said computing means.

17. A device according to claim 16, further comprising a photo-taking optical system arranged to form an object'"'"'s image on said image sensing means, and wherein said correction optical means includes optical axis varying means for varying the direction of the optical axis of said photo-taking optical system.

18. A device according to claim 17, wherein said optical axis varying means is a variable angle prism.

19. A device according to claim 16, wherein said movement detecting means is arranged to compute the image movement vector of each of a plurality of blocks obtained by dividing said image sensing plane and to find parts of said image sensing plane having image movement and the magnitude and direction of said image movement.

20. A device according to claim 16, further comprising automatic focusing means arranged to detect the focused state of an object'"'"'s image formed on said image sensing means and to automatically maintain said image in an in-focus state.

21. A device according to claim 20, wherein said weight setting means is arranged to change the degree of weight to be attached to information on the movement taking place on said image sensing plane in accordance with the photographing mode of said automatic focusing means and said image sensing means.

22. A device according to claim 21, wherein said movement detecting means is arranged to have the movement information weighted to a less degree for the continuously moving image plane parts and to a greater degree for image plane parts which are not continuously moving.

23. A device according to claim 22, wherein said automatic focusing means includes manual focusing means, a fixed area focusing means arranged to automatically adjust a focus of said optical system to an object'"'"'s image obtained within a focus detecting area fixedly set on said image sensing plane, and object tracing means for adjusting the focus to an object'"'"'s image by shifting a focus detecting area set on said image sensing plane in such a way as to trace the movement of said object'"'"'s image taking place within said image sensing plane.

24. A device according to claim 22, wherein said object tracing means includes setting means for setting an object tracing area within said image sensing plane in a shiftable manner and control means for controlling the setting position of said object tracing area in a direction in which a difference in luminance between the inside and outside of said object tracing area increases, and wherein said focus detecting area is set in the position of said object tracing area.

25. A device according to claim 22, wherein said weight setting means is arranged to attach weight to the movement vector of a peripheral part of said image sensing plane to a greater degree than to that of a central part of said image sensing plane when the manual focusing means of said automatic focusing means is selected.

26. A device according to claim 22, wherein said weight setting means is arranged to set the movement vector weighting degree at zero for the inside of said focus detecting area and at an increased value for the outside of said area when said fixed area focusing means is selected.

27. A device according to claim 22, wherein said weight setting means is arranged to set the movement vector weighting degree at zero for the inside of said focus detecting area in each of the shifted positions thereof and at an increased value for the outside of said area when said object tracing means is selected.

28. A device according to claim 22, wherein said weight setting means is arranged to set the movement vector weighting degree at an increased value for the inside of said focus detecting area and at zero for the outside of said area when the device is moved to trace a moving object with said focus detecting area fixed.

29. An image sensing apparatus comprising:

a) first movement detecting means for detecting the degree of shaking of said apparatus;

movement detecting means for detecting b) second movement detecting means for detecting the degree of shaking of said apparatus on the basis of information which differs from information used by said first movement detecting means;

c) third movement detecting means for producing a shaking degree by computing the outputs of said first and second movement detecting means; and

d) correction means for correcting said shaking degree on the basis of said shaking degree produced by said third movement detecting means.

30. An apparatus according to claim 29, further comprising an image sensor arranged to produce a sensed image signal by photo-electric converting an incident image of an object to be photographed.

31. An apparatus according to claim 30, wherein said first movement detecting means is arranged to obtain from said sensed image signal the vector of image movement taking place in each of a plurality of blocks set on the image sensing plane of said image sensor and to obtain the degree of shaking of said image by computing movement vectors thus obtained.

32. An apparatus according to claim 31, wherein said second movement detecting means is an external angular velocity sensor arranged to physically detect the angular velocity of movement of the apparatus.

33. An apparatus according to claim 32, wherein said second movement detecting means is arranged to detect a panning or tilting movement of the apparatus from the vector of movement of the apparatus detected by said angular velocity sensor.

34. An apparatus according to claim 33, wherein said second movement detecting means is arranged to compare in a cycle of fields the movement vectors detected by said angular velocity sensor and to produce a first control signal by deciding that the apparatus is making a panning or tilting movement if the movement vector is found to exceed a predetermined magnitude and remains unchanged over a period of several fields.

35. An apparatus according to claim 34, wherein said correction means is arranged to make no shake correction if said shaking degree is determined by said second movement detecting means to be resulting from panning or tilting.

36. An apparatus according to claim 35, wherein said computing means is arranged to detect the number of movement vectors which are in the same direction as the movement vector of the apparatus detected by said second movement detecting means among the vectors of image movement taking place in said blocks detected by said first movement detecting means, to compute a correlation between the image movement and the apparatus movement on the basis of the number thus detected, and further to compute and obtain from said number of movement vectors a representative value to be used for shake correction.

37. An apparatus according to claim 36, wherein said computing means is arranged to detect the number of movement vectors which are in the same direction as the movement vectors of the apparatus detected by said second movement detecting means among the vectors of image movement taking place in said blocks detected by said first movement detecting means, and to produce a second control signal indicating whether or not panning or tilting must be taken into consideration for the image movement.

38. An apparatus according to claim 36, wherein said computing means is arranged to supply the average vector of said number of vectors to said correction means as a representative value of movement vectors when the number detected indicates the necessity of taking panning or tilting into consideration.

39. An apparatus according to claim 36, wherein said computing means is arranged to supply, as a representative value of movement vectors to said correction means, the average of movement vectors of blocks which show the same direction as the movement vector of a central part of said image sensing plane among the movement vectors of a plurality of blocks detected by said first detecting means when said number indicates nonnecessity of taking the panning or tilting movement into consideration.

40. An apparatus according to claim 37, wherein said correction means is arranged to be controlled as to execution or inexecution of shake correction.

41. A camera comprising:

a) image sensing means for producing a sensed image signal by photo-electric converting an image formed by a photo-taking optical system;

b) first detecting means for detecting from said sensed image signal the movement of said image within the image sensing plane of said image sensing means;

c) second detecting means for detecting the movement of the camera;

d) a correction optical system arranged to offset said camera movement; and

e) control means for controlling the correcting action and correcting degree of said correction optical system on the basis of the outputs of said first and second detecting means.

42. A camera according to claim 41, wherein said first detecting means is arranged to obtain from said sensed image signal the vector of image movement in each of a plurality of blocks set on the image sensing plane of said image sensing means and to obtain the shaking degree of the image by computing said movement vectors.

43. A camera according to claim 42, wherein said second movement detecting means is an external angular velocity sensor which is arranged to physically detect the movement of the camera, and wherein a panning or tilting movement of the camera is detected from the vector of camera movement detected by said angular velocity sensor.

44. A camera according to claim 43, wherein said second detecting means is arranged to compare in a cycle of fields the values of the movement vectors detected by said angular velocity sensor, and to produce a panning signal by deciding said movement to be a panning or tilting movement if said movement vector exceeds a predetermined value and lasts over a period of predetermined number of fields.

45. A camera according to claim 44, wherein said control means is arranged to have no shake correction performed by said correction optical system when the camera is determined to be in panning or tiling movement.

46. A camera according to claim 45, wherein said control means is arranged to detect the number of movement vectors showing the same direction as the camera movement vector detected by said second movement detecting means among the image movement vectors of blocks detected by said first movement detecting means, to produce, on the basis of the number detected, a determination signal indicating whether or not panning or tilting is to be taken into consideration, and to compute a representative value of movement vectors for determining the direction and degree of shake correction.

47. A camera according to claim 46, wherein said control means is arranged to supply the average of said number of vectors to said correction optical system as said representative value of movement vectors when said number indicates the necessity of consideration of a panning or tilting movement of the camera.

48. A camera according to claim 46, wherein said control means is arranged to supply, as said representative value to said correction optical system, the average of movement vectors which show the same direction as the image movement vector of a central part of said image sensing plane among the movement vectors detected from the plurality of blocks by said first detecting means when said number indicates the necessity of consideration of a panning or tilting movement of the camera.

49. A camera according to claim 48, wherein said control means is arranged to perform control as to execution or inexecution of shake correction on the basis of a logical product of said determination signal and said panning signal.

50. An automatic image stabilization camera comprising:

a) image sensing means arranged to produce a sensed image signal by photo-electric converting an image formed by a photo-taking optical system;

b) detection means for detecting from said sensed image signal the movement of said image on the image sensing plane of said image sensing means;

c) an angular velocity sensor arranged to detect a panning or tilting movement of the camera;

d) a correction optical system arranged to correct the shaking of the camera by varying the optical axis of said photo-taking optical system;

e) computing means arrangedto determine necessity or nonnecessity of the shake correction by said correction optical system and to determine the degree of said shake correction by computing the output of said detection means and that of said angular velocity sensor; and

f) control means for controlling said correction optical system on the basis of the output of said computing means.