Image blur correcting apparatus

• US 5,450,126 A
• Filed: 11/20/1992
• Issued: 09/12/1995
• Est. Priority Date: 10/17/1989
• Status: Expired due to Fees

• Current Assignee: Mitsubishi Denki Kabushiki Kaisha (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Denki Kabushiki Kaisha (Mitsubishi Electric Corporation)
• Inventors: Nishida, Yoshihiro
• Primary Examiner(s): Britton, Howard W.
• Application Number:
  US07/979,779
• Time in Patent Office: 1,026 Days
• Field of Search: 348/208, 348/699
• US Class Current: 348/207.99
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/223 using block-matching
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/687 by shifting the lens or sensor position
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

An image blur correcting apparatus includes a lens-barrel part having a lens and an imaging device, an image signal processing device for forming a video signal from an electrical signal obtained at the imaging device, a supporting body for supporting the lens-barrel part in a freely pivoting manner around a rotation axis crossing a beam axis incident to the lens-barrel part at approximately right angles, and an actuator device attached between the lens-barrel part and the supporting body for rotating the lens-barrel part. The apparatus further includes a relative angle detecting device for detecting a relative angle between the lens-barrel part and the supporting body and a device for detecting a moving vector indicating an amount of blur of the image from correlation between two frames of image information from the image signal processing means, which are continuous in view of time. Finally, the device includes an angular velocity accumulating device for calculating rotating angular velocity using a zoom multiplying factor or the like from the detected result of the moving vector.

Claims

1. An image blur correcting apparatus comprising:

a lens-barrel part including a lens and an imaging device;

image signal processing means for forming a video signal from an electrical signal obtained at said imaging device;

a supporting body for supporting said lens-barrel part in a freely pivoting manner around a rotating axis crossing a beam axis incident to said lens-barrel part at approximately right angles;

actuator means, attached between said lens-barrel part and said supporting body, for rotating said lens-barrel part;

relative angle detecting means for detecting a relative angle between said lens-barrel part and said supporting body;

means for detecting a moving vector indicating an amount of blur of the image from correlation between two fields of image information from said image signal processing means, which are continuous in view of time;

angular velocity operating means for calculating rotation angular velocity using a zoom multiplying factor from the detected moving vector; and

calculating controlling means for outputting a control signal of an actuator in accordance with outputs of said relative angle detecting means and said angular velocity detecting means;

wherein the amount of blur of the image is determined in both a pitch and yaw direction.

2. The image blur correcting apparatus of claim 1, wherein the two fields are in the same frame.

3. The image blur correcting apparatus of claim 1, wherein the two fields are in adjacent frames.

4. A video camera, including the image blur correcting apparatus of claim 1.

5. The image blur correcting apparatus of claim 1, said actuator means including,

back yoke means including a magnet, attached to said lens-barrel part, for generating a field magnetic flux,

coil yoke means including a coil yoke, attached to said supporting body, and

relative difference means for detecting the field magnetic flux and for generating electrical signals representative of a relative difference between an angle of said magnet and an angle of said coil yoke.

6. The image blur correcting apparatus of claim 5, wherein said relative difference means is a Hall element.

7. The image blur correcting apparatus of claim 5, said relative angle detecting means including,

differential amplifying means or amplifying the electrical signals generated by said relative difference means, and

biasing means for biasing the amplified electrical signals to determine the relative angle between said lens-barrel part and said supporting body.

8. The image blur correcting apparatus of claim 1, said means for detecting including,

A/D converting means for receiving and A/D converting the video signal from said image signal processing means, to produce a digital signal,

latching means for delaying all pixels of the digital signal,

representative point memory means for selecting a representative pixel from all the pixels of the digital signal, to produce a representative point signal,

absolute value means for calculating an absolute value of a difference between the representative point signal and the delayed digital signal for each of a plurality of blocks which comprise the image,

accumulated addition storing means for storing the absolute value, calculated for each of the plurality of blocks, and

comparing means for comparing the absolute values for each of the plurality of blocks and generating the moving vector indicating the amount of blur of the image.

9. The image blur correcting apparatus of claim 8, wherein said latching means delays all pixels of the digital signal by one frame.

10. The image blur correcting apparatus of claim 9, wherein the absolute value of the difference between the representative port signal and the delayed digital signal for each of the plurality of blocks is represented by:

##EQU2## Where;

Dxy=the absolute value of the difference for each of the plurality of blocks,

R_ij =data of the representation pixel

S_i+x,j+y =data of a pixel of the delayed digital signal.

11. The image blur correcting apparatus of claim 10, wherein the moving vector is obtained by averaging movement of the entire image.

12. The image blur correcting apparatus of claim 11, wherein the moving vector is obtained by adding each of the representation pixels for each of the plurality of blocks of the image in order to average the movement of the entire image.

13. The image blur correcting apparatus of claim 1, said means for detecting including,

A/D connecting means for receiving and A/D converting the video signal from said image signal processing means to produce a digital signal,

latching means for delaying all pixels of the digital signal,

representative point memory means for selecting a representative pixel from all the pixels of the digital signal to produce a representative point signal,

absolute value means for calculating an absolute value of a difference between the representative point signal and the delayed digital signal for each of a plurality of blocks which comprise the image,

dividing means for dividing each of the plurality of blocks into a plurality of areas, each of the plurality of areas having a corresponding accumulation table,

accumulation means for adding the absolute value of the difference between the representative point signal and the delayed digital signal for each of the plurality of blocks to a plurality of table value comparing means, one for each of the plurality of areas, to produce a plurality of moving vector values,

moving vector means for averaging the plurality of moving vector values,

multiplying means for multiplying the average by a predetermined value,

parallel movement/zooming operation determining means for receiving the plurality of moving vector values and detecting parallel movement or a zooming operation of the image, and

switching means for setting the moving vector equal to the average of the plurality of moving vector values calculated by said moving vector means when parallel movement is detected by said parallel movement/zooming operation detecting means and for setting the moving vector equal to the multiplied average, calculated by said multiplying means when the zooming operation is detected by said parallel movement/zooming operation determining means.

14. The image blur correcting apparatus of claim 13, wherein said parallel movement/zooming operation determining means determines the image has parallel movement when a direction and a magnitude of each of the plurality of moving vector values approximates a direction and a magnitude of the average of the plurality of moving vector values and determines the zooming operation has been performed on the image when the directions of the plurality of moving vector values are varied.

15. The image blur correcting apparatus of claim 13, wherein said latching means delays all pixels of the digital signal by one frame.

16. The image blur correcting apparatus of claim 15, wherein the absolute value of the difference between the representative point signal and the delayed digital signal for each of the plurality of blocks is represented by:

##EQU3## Where;

Dxy=the absolute value of the difference for each of the plurality of blocks,

R_ij =data of the representation pixel,

S_i+x,j+y =data of a pixel of the delayed digital signal.

17. The image blur correcting apparatus of claim 16, wherein the moving vector is obtained by averaging movement of the entire image.

18. The image blur correcting apparatus of claim 17, wherein the moving vector is obtained by adding each of the representation pixels for each of the plurality of blocks of the image in order to average the movement of the entire image.

19. The image blur correcting apparatus of claim 1, said means for detecting including,

A/D converting means for receiving and A/D converting the video signal from said image signal processing means to produce a digital signal,

latching means for delaying all pixels of the digital signal,

representative point memory means for selecting a representative pixel from all the pixels of the digital signal to produce a representative point signal,

vertical and horizontal absolute value means for calculating a vertical and horizontal absolute value, respectively, the vertical and horizontal absolute values representing a difference between a vertical and horizontal component of the representative point signal and a vertical and horizontal component of the delayed digital signal, respectively, for each of a plurality of blocks which comprise the image,

vertical and horizontal accumulated addition tables for storing the vertical and horizontal absolute values, respectively, calculated for each of the plurality of blocks,

one-dimensional vector detecting means for detecting a vertical vector and an intersecting horizontal vector from the vertical and horizontal absolute values stored in said vertical and horizontal accumulated addition tables, respectively, and

two dimensional vector detecting means for detecting the moving vector from the vertical vector and the intersecting horizontal vector.

20. The image blur correcting apparatus of claim 19, wherein the vertical and horizontal accumulation tables form an inverted cone with an apex at (c,b,0) wherein an X-axis represents a horizontal directional, a Y-axis represents a vertical direction, and a z-axis represents an accumulation table value and the moving vector includes a vertical moving vector represented by (0,b) and a horizontal moving vector represented by (c,0).

21. The image blur correcting apparatus of claim 19, wherein the vertical and horizontal accumulation tables form an inverted cone with an apex at (c,b,0) wherein an X-axis represents a horizontal direction, a Y-axis represents a vertical direction, and a Z-axis represents an accumulation table value and the vertical and horizontal accumulation tables are represented by

22. The image blur correcting apparatus of claim 19, wherein said latching means delays all pixels of the digital signal by one frame.

23. The image blur correcting apparatus of claim 22, wherein the vertical and horizontal absolute values which represent the difference between the vertical and horizontal components of the representative point signal and the vertical and horizontal components of the delayed digital signal for each of the plurality of blocks, are represented by:

space="preserve" listing-type="equation">Dx=Σ

|R.sub.ij -S.sub.i,j+y |

space="preserve" listing-type="equation">Dy=Σ

|R.sub.ij -S.sub.i+x,j |

where;

Dx=the horizontal absolute value for each of the plurality of blocks,

Dy=the vertical absolute value for each of the plurality of blocks,

R_ij =data of the representative pixel,

S_i,j+y =data of a pixel of the delayed digital signal in a y-z plane, and

S_i+x,j =data of a pixel of the delayed signal in a x-z plane.

24. The image blur correcting apparatus of claim 23, wherein the moving vector is obtained by averaging movement of the entire image.

25. The image blur correcting apparatus of claim 24, wherein the moving vector is obtained by adding each of the representation pixels for each of the plurality of blocks of the image in order to average the movement of the entire image.

26. A video camera comprising:

an image blur correcting apparatus in which moving vectors of the whole screen are detected from an amount of movement of a pixel in each block, based upon representative points set in a plurality of blocks in an image, in order to detect an amount of movement between frames of the image, said image blur correcting apparatus including;

means for calculating an absolute value of a difference between a pixel of a representative point for each block of the frame directly, before a frame whose movement vector is to be detected, and the pixels on a horizontal straight line and a vertical straight line passing through the representative point, respectively, of the frame whose movement vector is to be detected;

horizontal and vertical accumulated addition means for accumulatedly adding horizontal and vertical absolute values calculated;

means for detecting two intersecting one-dimensional moving vectors from the horizontal and vertical accumulated added values; and

means for calculating a two-dimensional moving vector from said two intersecting one-dimensional moving vectors.

27. The video camera of claim 26, wherein said one-dimensional moving vector detecting means detects moving vectors in a horizontal and vertical direction, based upon coordinates of a minimum accumulated addition value of said horizontal and vertical accumulated addition means.

28. The video camera of claim 26, wherein said one-dimensional moving vector detecting means obtains a curved surface representing the accumulated added values from the accumulated addition means, detects a point closest to an XY plane on the curved surface, and obtains moving vectors in the horizontal and vertical direction, based upon coordinates of the closest point.

29. A video camera comprising:

an image blur correcting apparatus including,

means for selecting a representative point in a first of two continuous screens, for dividing a second of the two continuous screens into a plurality of blocks, and finding an absolute value of a difference between the representative point and each of the plurality of blocks;

means for dividing each of the plurality of blocks into a plurality of areas and for calculating a table by adding the absolute value of the difference for each of the plurality of areas for each of the corresponding plurality of blocks to said table;

means for finding a moving vector in each of the plurality of areas from said table accumulated for each of the plurality of areas;

parallel movement/zooming operation determining means for determining whether each of the plurality of areas of the second of two continuous screens are moving parallel to each other or if the second of the two continuous screens is moving according to a zooming operation, depending on a moving vector determined for each of the plurality of areas; and

means for outputting a screen moving vector, having a value obtained by averaging the moving vectors of each of the plurality of areas for the second of the two continuous screens when said parallel movement/zooming operation determining means determines the second of the two continuous screens is moving in parallel and for outputting the screen moving vector having a value obtained by multiplying an average value of the second of the two continuous screens by a prescribed attenuation constant K (0≦

k<

1) when said means determines the second of the two continuous screens to be moving according to a zooming operation.

30. The video camera of claim 29, wherein the second of the two continuous screens is divided into nine blocks.

31. The video camera of claim 30, wherein said parallel movement/zooming operation determining means determines parallel movement when the moving vector of each of the plurality of areas is in the same direction.

32. The video camera of claim 30, wherein said parallel movement/zooming operation determining means determines the second of the two continuous screens to be in a zooming operation when the moving vector of the plurality of areas have different directions.

33. A video camera comprising:

an image blur correcting apparatus including,

a lens-barrel part having a lens and an imaging device;

image signal processing means for forming a video signal from an electrical signal obtained at said imaging device;

a supporting body for supporting said lens-barrel part in a freely pivoting manner around a rotating axis crossing a beam axis incident to said lens-barrel part at approximately right angles;

actuator means, attached between said lens-barrel part and said supporting body, for rotating said lens-barrel part;

relative angle detecting means for detecting a relative angle between said lens-barrel part and said supporting body;

means for detecting a moving vector indicating an amount of blur of the image from correlation between two frames, which are continuous in view of time, of image information from said image signal processing means;

angular velocity operating means for calculating rotation angular velocity using a zoom multiplying factor from the detected moving vector; and

calculating controlling means for outputting a control signal of an actuator in accordance with outputs of said relative angle detecting means and said angular velocity detecting means.

34. An image blur converting apparatus comprising:

a moving vector detecting apparatus in which moving vectors of at least a part of the screen are detected from an amount of movement of a pixel in each block, based upon representative points set in a plurality of blocks in an image, in order to detect an amount of movement between at least one of frames and fields of the image, said moving vector detecting apparatus including,

means for detecting two intersecting one-dimensional moving vectors from vertical accumulated addition tables corresponding to pixels on a vertical straight line passing through a representative point, and from horizontal accumulated addition tables corresponding to pixels on a horizontal straight line passing through the representative point, and

means for calculating a two-dimensional moving vector from said two intersecting one-dimensional moving vectors.

35. A video camera including the image blur correcting apparatus of claim 34.

36. The image blur correcting apparatus of claim 34, wherein the fields are in the same frame.

37. The image blur correcting apparatus of claim 34, wherein the fields are in adjacent frames.

38. A moving vector detector, comprising:

representative point memory means for selecting a representative pixel from all pixels of a digital signal, to produce a representative point signal,

absolute value means for calculating an absolute value of a difference between the representative point signal and delayed digital signal for each of a plurality of blocks which comprise an image,

accumulated addition storing means for storing the absolute value, calculated for each of the plurality of blocks, and

comparing means for comparing the absolute values for each of the plurality of blocks and generating a moving vector indicating an amount of blur of the image.

39. A video camera including the moving vector detector of claim 38.

40. An image blur correcting apparatus including the moving vector detector of claim 38.

41. The moving vector detector of claim 40, wherein the absolute value of the difference between the representative point signal and the delayed digital signal for each of the plurality of blocks is represented by:

##EQU4## where;

Dxy=the absolute value of the difference for each of the plurality of blocks,

R_ij =data of the representation pixel

S_i+x,j+y =data of a pixel of the delayed digital signal.

42. The moving vector detector of claim 41, wherein the moving vector is obtained by averaging movement of the entire image.

43. The moving vector detector of claim 42, wherein the moving vector is obtained by adding each of the representation pixels for each of the plurality of blocks of the image in order to average the movement of the entire image.

44. A moving vector detector, comprising:

representative point memory means for selecting a representative pixel from all pixels of a digital signal to produce a representative point signal,

vertical and horizontal absolute value means for calculating a vertical and horizontal absolute value, respectively, the vertical and horizontal absolute values representing a difference between a vertical and horizontal component of the representative point signal and a vertical and horizontal component of a delayed digital signal, respectively, for each of a plurality of blocks which comprise an image,

vertical and horizontal accumulated addition tables for storing the vertical and horizontal absolute values, respectively, calculated for each of the plurality of blocks,

one-dimensional vector detecting means for detecting a vertical vector and an intersecting horizontal vector from the vertical and horizontal absolute values stored in said vertical and horizontal accumulated addition tables, respectively, and

two dimensional vector detecting means for detecting a moving vector from the vertical vector and the intersecting horizontal vector.

45. An image blur correcting apparatus including the moving vector detector of claim 44.

46. A video camera including the moving vector detector of claim 44.

47. The moving vector detector of claim 44, wherein the vertical and horizontal accumulation tables form an inverted cone with an apex at (c,b,0) wherein an X-axis represents a horizontal directional, a Y-axis represents a vertical direction, and a z-axis represents an accumulation table value and the moving vector includes a vertical moving vector represented by (0,b) and a horizontal moving vector represented by (c,0).

48. The moving vector detector of claim 44, wherein the vertical and horizontal accumulation tables form an inverted cone with an apex at (c,b,0) wherein an X-axis represents a horizontal direction, a Y-axis represents a vertical direction, and a Z-axis represents an accumulation table value and the vertical and horizontal accumulation tables are represented by

49. The moving vector detector of claim 44, wherein the vertical and horizontal absolute values which represent the difference between the vertical and horizontal components of the representative point signal and the vertical and horizontal components of the delayed digital signal for each of the plurality of blocks, are represented by:

space="preserve" listing-type="equation">Dx=Σ

|R.sub.ij -S.sub.i,j+y |

space="preserve" listing-type="equation">Dy=Σ

|R.sub.ij -S.sub.i+x,j |

where;

Dx=the horizontal absolute value for each of the plurality of blocks,

Dy=the vertical absolute value for each of the plurality of blocks,

R_ij =data of the representative pixel,

S_i,j+y =data of a pixel of the delayed digital signal in a y-z plane, and

S_i+x,j =data of a pixel of the delayed signal in a x-z plane.

50. The moving vector detector of claim 49, wherein the moving vector is obtained by averaging movement of the entire image.

51. The moving vector detector of claim 50, wherein the moving vector is obtained by adding each of the representation pixels for each of the plurality of blocks of the image in order to average the movement of the entire image.

52. A moving vector detector, comprising:

representative point memory means for selecting a representative pixel from all pixels of a digital signal to produce a representative point signal,

absolute value means for calculating an absolute value of a difference between the representative point signal and a delayed digital signal for each of a plurality of blocks which comprise an image,

dividing means for dividing each of the plurality of blocks into a plurality of areas, each of the plurality of areas having a corresponding accumulation table,

accumulation means for adding the absolute value of the difference between the representative point signal and the delayed digital signal for each of the plurality of blocks to a plurality of table value comparing means, one for each of the plurality of areas, to produce a plurality of moving vector values,

moving vector means for averaging the plurality of moving vector values,

multiplying means for multiplying the average by a predetermined value,

parallel movement/zooming operation determining means for receiving the plurality of moving vector values and detecting parallel movement or a zooming operation of the image, and

switching means for setting the moving vector equal to the average of the plurality of moving vector values calculated by said moving vector means when parallel movement is detected by said parallel movement/zooming operation detecting means and for setting the moving vector equal to the multiplied average, calculated by said multiplying means when the zooming operation is detected by said parallel movement/zooming operation determining means.

53. An image blur correcting apparatus including the moving vector detector of claim 52.

54. A video camera including the moving vector detector of claim 52.

55. The moving vector detector of claim 52, wherein said parallel movement/zooming operation determining means determines the image has parallel movement when a direction and a magnitude of each of the plurality of moving vector values approximates a direction and a magnitude of the average of the plurality of moving vector values and determines the zooming operation has been performed on the image when the directions of the plurality of moving vector values are varied.

56. The moving vector detector of claim 52, wherein the absolute value of the difference between the representative point signal and the delayed digital signal for each of the plurality of blocks is represented by:

##EQU5## Where;

Dxy=the absolute value of the difference for each of the plurality of blocks,

R_ij =data of the representation pixel,

S_i+x,j+y =data of a pixel of the delayed digital signal.

57. The moving vector detector of claim 56, wherein the moving vector is obtained by averaging movement of the entire image.

58. The moving vector detector of claim 57, wherein the moving vector is obtained by adding each of the representation pixels for each of the plurality of blocks of the image in order to average the movement of the entire image.