Image shake detecting device

• US 5,012,270 A
• Filed: 03/06/1989
• Issued: 04/30/1991
• Est. Priority Date: 03/10/1988
• Status: Expired due to Term

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Sekine, Masayoshi, Nakajima, Toshiyuki, Toyama, Masamichi, Yoshimura, Katsuji, Kai, Takashi
• Primary Examiner(s): ADAMS JR, RUSSELL E
• Application Number:
  US07/319,658
• Time in Patent Office: 785 Days
• Field of Search: 354/430
• US Class Current: 396/54
• CPC Class Codes: G01S 3/7864 T.V. type tracking systems
• CPC Class Codes: G03B 2217/005 Blur detection
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/223 using block-matching
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/673 based on contrast or high frequency components of image signals, e.g. hill climbing method
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6815 by distinguishing pan or tilt from motion
• CPC Class Codes: H04N 23/686 with a variable apex prism
• CPC Class Codes: H04N 23/687 by shifting the lens or sensor position
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

An image shake detecting device for detecting a shake of an image on an image sensing plane on the basis of a video signal output from an image sensor includes detection circuits arranged to detect image displacement in a plurality of areas set on the image sensing plane, and a control microcomputer which makes a discrimination, on the basis of information output from these detection circuits, between a movement of a camera and a solo movement of a photographed object. The device thus accurately makes compensation for an image shake by judging the state of the image on the basis of information output from these detection circuits. The image shake detecting device further includes a computing circuit arranged to compute a quantity of an image shake on the basis of a difference in detecting timing of a feature point of the image and a sensitivity control circuit arranged to change the detection sensitivity of the shake detection circuits. Further disclosed is an object tracing device to which the invented image shake detecting device is applied.

Claims

1. An image shake detecting device for detecting a shake of an image on an image sensing plane on the basis of an image signal output from image sensing means, comprising:

a) first detection means for detecting the displacement of the image on the basis of an image signal corresponding to a first detection area set on said image sensing plane;

b) second detection means for detecting the displacement of the image on the basis of an image signal corresponding to a second detection area set on said image sensing plane;

c) switching means for selecting one of said first detection means and said second detection means; and

d) image shake detecting means for computing the shake of the image on the basis of an output of said first detection means or said second detection means selected by said switching means.

2. A device according to claim 1, further comprising:

a) compensating means for displacing an optical axis of a photo-taking optical system which forms an image from an incident light on an imaging plane of said image sensing means; and

b) driving control means for driving said compensating means on the basis of an output of said image shake detecting means.

3. A device according to claim 2, wherein said compensating means is a variable apex-angle prism inserted in said photo-taking optical system, and wherein said driving control means is arranged to vary an apex angle of said variable apex-angle prism in the direction of decreasing a quantity of the shake of the image on the basis of information on the shake of the image output from said image shake detecting means.

4. A device according to claim 3, wherein said first detection area is an area set approximately in a central part of said image sensing plane, and wherein said second detection area is an area located around said first detection area.

5. A device according to claim 4, wherein an image located within said first detection area corresponds to a main object image in said image sensing plane, and wherein said second detection area corresponds to a background of said main object image in said image sensing plane.

6. A device according to claim 5, wherein each of said first and second detection means is arranged to obtain the center of gravity of an image from a histogram formed by converting a luminance level of the image signal into a binary value and to compute a movement vector of an image on the basis of a movement quantity of said center of gravity in a plurality of images obtained at intervals of a predetermined period of time, and wherein said driving control means is arranged to control said compensating means on the basis of said movement vector.

7. A device according to claim 4, wherein said image shake detecting means has a first mode in which a shake of an image is detected on the basis of detection information obtained from said first detection area and a second mode in which a shake of an image is detected on the basis of detection information obtained from said second detection area.

8. A device according to claim 7, wherein said switching means includes means for manually selecting one of said first mode and said second mode.

9. A device according to claim 7, wherein said switching means includes means for automatically selecting one of said first mode and said second mode.

10. A device according to claim 9, wherein said image shake detecting means includes means for detecting a predetermined signal component corresponding to a sharpness in the image signal obtained from each of said first and second detection areas, and wherein said switching means is arranged to select one of said first and second detection areas which is greater in a quantity of detection of said predetermined signal component.

11. A device according to claim 10, wherein said image shake detecting means is arranged to stop performing an image shake detecting action thereof when the quantity of detection of said predetermined signal component in said first or second detection area is less than a predetermined level.

12. A device according to claim 11, wherein said predetermined signal component is a high-frequency component included in the image signal.

13. A device according to claim 12, wherein said predetermined signal component is an edge component included in said image signal.

14. An image shake detecting device for detecting a shake of an image on an image sensing plane on the basis of an image signal output from image sensing means, comprising:

a) movement detecting means for detecting the displacement of the image on the basis of an image signal corresponding to a predetermined detection area set on said image sensing plane;

b) compensating means for compensating for a movement of the image, by displacing an optical axis of a photo-taking optical system which images an incident light on an imaging plane of said image sensing means, on the basis of information on the displacement of the image output from said movement detecting means;

c) sharpness detecting means for detecting a signal component indicative of a sharpness of the image on the basis of an image signal corresponding to said detection area; and

d) control means for controlling an operation of said compensation means and/or movement detecting means on the basis of an output of said sharpness detecting means.

15. A device according to claim 14, wherein said compensating means is a variable apex-angle prism inserted in a photo-taking optical system, which is arranged to vary an apex angle thereof inn the direction of decreasing a quantity of the shake of the image in accordance with said information on the displacement of the image.

16. A device according to claim 14, wherein said movement detecting means is arranged too detect the displacement of the image in a first area set approximately in a central part of said image sensing plane and a second area located around said first area.

17. A device according to claim 16, wherein an image located within said first area corresponds to a main object image in said image sensing plane, and wherein said second area corresponds to a background of said main object.

18. A device according to claim 17, wherein said movement detecting means is arranged to obtain the center of gravity of an image from a histogram formed by converting into a binary value a luminance level of the image signal and to compute a movement vectors of an image on the basis of a movement quantity of said center of gravity in a plurality of images obtained at intervals of a predetermined period of time, and wherein said control means is arranged to control said compensating means on the basis of said movement vector.

19. A device according to claim 17, wherein said movement detecting means has a first mode in which a shake of an image is detected on the basis of detection information obtained from said first area and a second mode in which a shake of an image is detected on the basis of detection information obtained from said second area.

20. A device according to claim 19, wherein said control means includes switching means for selecting one of said first mode and said second mode.

21. A device according to claim 20, wherein said switching means includes means for manually selecting one of said first mode and said second mode.

22. A device according to claim 20, wherein said switching means includes means for automatically selecting one of said first mode and said second mode.

23. A device according to claim 22, wherein said control means is arranged to select one of said first and second areas which is greater in an output of said sharpness detecting means.

24. A device according to claim 14 or 20, wherein said control means is arranged to stop an action of said compensating means when the output of said sharpness detecting means is less than a predetermined value.

25. A device according to claim 14, wherein said sharpness detecting means is arranged to detect a sharpness on the basis of a quantity of a high-frequency component of the image signal.

26. A device according to claim 25, wherein said sharpness is an edge component of the image signal.

27. An image shake detecting device for detecting a shake of an image on an image sensing plane on the basis of an image signal output from image sensing means, comprising:

a) extracting means for extracting a feature point from within said image sensing plane;

b) detection means for detecting a difference in generation timing of said feature point between a plurality of temporally different images;

c) computing means for computing a quantity of the shake of the image on the basis of an output of said detection means; and

d) sensitivity control means for varying a sensitivity of said detection means.

28. A device according to claim 27, further comprising compensating means for compensating for the shake of the image on the basis of a result of a computing operation of said computing means.

29. A device according to claim 28, wherein said compensating means is a variable apex-angle prism for compensating an optical axis of a photo-taking optical system which forms an object image on said image sensing means.

30. A device according to claim 27, wherein said extracting means is arranged to extract a feature point from a signal pattern obtained by converting into a binary value an edge component of a luminance component of the image signal output from said image sensing means.

31. A device according to claim 27, wherein said detection means includes clock pulse generating means and is arranged to detect the difference in generation timing of said feature point between temporally different field images by counting clock pulses generated by said clock pulse generating means.

32. A device according to claim 31, wherein said computing means is arranged to compute the quantity of the shake of the image on the basis of a counted number of said clock pulses.

33. A device according to claim 32, further comprising area setting means for extracting an area having a movement on said image sensing plane to designate an image shake detection area.

34. A device according to claim 33, wherein said area setting means is arranged to divide said image sensing plane into an m×

n number of blocks, to detect a movement part of said image sensing plane having a local image movement by comparing binary signals of a luminance level in each of said blocks between a plurality of temporally different images, and to designate an area other than said movement part as said image shake detection area.

35. device according to claim 34, wherein said sensitivity control means is arranged to control a sensitivity of said detection means by varying a clock signal frequency.

36. A device according to claim 35, wherein said sensitivity control means is arranged to lower the sensitivity at said movement part and to raise the sensitivity at a part having not much movement.

37. A device according to claim 36, wherein said sensitivity control means is arranged to vary stepwise the sensitivity of said detection means in each of said m×

n number of blocks.

38. An image shake detecting device comprising:

a) scanning means for scanning at the same speed a plurality of images obtained at intervals of a predetermined period of time;

b) detection means for detecting a time difference between said plurality of images in timing at which corresponding points of feature are generated during a scanning process of said scanning means, by counting the number of clock pulses;

c) computing means for detecting a quantity of a shake of an image by computing an output of said detection means; and

d) sensitivity control means for varying a detection sensitivity of said detection means by changing a period of said clock pulses.

39. A device according to claim 38, further comprising compensating means for compensating for the quantity of the shake on the basis of a result of a computing operation of said computing means.

40. A device according to claim 39, wherein said compensating means is a variable apex-angle prism for compensating an optical axis of a photo-taking optical system which forms an object image on image sensing means.

41. A device according to claim 39, wherein said computing means is arranged to compute the quantity of the shake on the basis of a counted number of said clock pulses.

42. A device according to claim 41, further comprising area setting means for extracting an area having a movement on an image sensing plane to designate an image shake detection area.

43. A device according to claim 42, wherein said area setting means is arranged to divide said image sensing plane into an m X n number of blocks, to detect a movement part of said image sensing plane having a local movement by comparing, between a plurality of temporally different images, binary signals of a luminance level in each of said blocks, and to designate an area other than said movement part as said image shake detection area.

44. A device according to claim 43, wherein said sensitivity control means is arranged to control a detection sensitivity of said detection means by varying a clock signal frequency.

45. A device according to claim 44, wherein said sensitivity control means is arranged to lower the sensitivity at said movement part and to raise the sensitivity at a part having not much movement.

46. A device according to claim 45, wherein said sensitivity control means is arranged to vary stepwise the sensitivity of said detection means in each of said m×

n number of blocks.

47. A control device for an image sensing optical system, comprising:

a) first control means for feedback-controlling said image sensing optical system on the basis of detection information obtained by timewise dispersively detecting a state of said image sensing optical system;

b) memory means for storing control information obtained on the basis of a quantity of control performed over said image sensing optical system when said detection information is sampled; and

c) second control means for controlling said image sensing optical system on the basis of a result obtained by computing said control information stored in said memory means and said detection information.

48. A device according to claim 47, further comprising a compensating optical system for varying an optical axis of said image sensing optical system.

49. A device according to claim 48, wherein said detection information represents a quantity of a shake of said image sensing optical system, and wherein said first control means is arranged to compensate for a state of said compensating optical system to decrease the quantity of the shake in accordance with said detection information.

50. A device according to claim 49, wherein said compensating optical system is a variable apex-angle prism for varying the optical axis of said image sensing optical system.

51. A device according to claim 50, wherein said memory means is arranged to store a quantity of a control over said compensating optical system for compensation of the shake.

52. A device according to claim 51, wherein said second control means is arranged to supply a previous quantity of a control stored in said memory means to a control loop of said first control means.

53. A device according to claim 52, wherein said first control means is arranged to control said compensating optical system on the basis of a value obtained by adding current detection information to the previous quantity of the control supplied from said second control means.

54. A device according to claim 53, wherein said second control means includes means for multiplying information stored in said memory means by a predetermined coefficient.

55. A device according to claim 54, wherein a detecting period of said detection information and a period of said control are set at a value which is an integer times as much as one field period.

56. A device according to claim 54, wherein the quantity of the control stored in said memory means is a quantity of an actual operation of a controlled system.