Movement vector detection device

• US 5,296,925 A
• Filed: 10/27/1992
• Issued: 03/22/1994
• Est. Priority Date: 04/27/1990
• Status: Expired due to Term

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Sekine, Masayoshi, Kondo, Toshiaki
• Primary Examiner(s): Peng, John K.
• Application Number:
  US07/967,569
• Time in Patent Office: 511 Days
• Field of Search: 358/105
• US Class Current: 348/208.1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/223 using block-matching
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

The present invention relates to a movement vector detection apparatus for obtaining the movement amount of an image from the density difference between frames and the spatial inclination. More particularly, the present invention relates to a movement vector detection apparatus having a first operation circuit for operating the movement vector in accordance with an estimation equation approximating an influence of the diagonal spatial inclination component, a second operation circuit for obtaining a signal denoting the amount of the diagonal spatial inclination component of a supplied image signal and a third operation circuit for averaging the output from the first operation circuit under a weight which corresponds to the output from the second operation circuit.

Claims

1. A movement vector detection device for detecting a movement vector of an image signal output from image sensing means based on a spatial inclination in a frame detected in the image signal, said movement vector detection device comprising:

(A) first operation means for receiving the image signal output from the image sensing means and for determining the movement vector by detecting the spatial inclination in the frame detected in the image signal, said first operation means outputting a movement vector signal;

(B) second operation means for detecting a spatial inclination component in a diagonal direction of the image signal and for outputting a diagonal inclination signal;

(C) third operation means for weighted-averaging the movement vector signal output from said first operation means in accordance with the diagonal inclination signal output from said second operation means to generate a weighted-averaged movement vector; and

(D) output means for outputting the weighted-averaged movement generated by said third operation means.

2. A movement vector detection device according to claim 1, wherein said first operation means further comprises density difference detection means for detecting the density difference between a plurality of frames wherein the movement vector is obtained by operating the output from said density difference detection means and the spatial inclination.

3. A movement vector detection device according to claim 1, wherein said first operation means determines said movement vector in accordance with an approximation operation expression formed by estimating the influence of the diagonal spatial inclination of said image signal.

4. A movement vector detection device according to claim 3, wherein said third operation means averages a plurality of movement vectors determined by said first operation means by weighting them in accordance with the degree of spatial inclinations detected by said second operation means so that the influence of a particular movement vector on the average is reduced in proportion to the degree of that movement vector'"'"'s spatial inclination.

5. A movement vector detection device according to claim 4, wherein said third operation means sequentially arranges a plurality of movement vectors determined by said first operation means for respective movement vectors in accordance with the degree of spatial inclinations detected by said second operation means and sequentially changes the weight and the amount so that the movement vectors are averaged.

6. A movement vector detection device according to claim 3, wherein said third operation means weights a plurality of movement vectors determined by said first operation means in accordance with their diagonal inclination components by multiplying them by the inverse numbers of the corresponding outputs from said second operation means.

7. A movement vector detection device for detecting a movement vector of an image signal output from image sensing means by detecting the spatial inclination in a frame of the image signal comprising detected blocks, said movement vector detection device comprising:

(A) movement vector operation means for receiving the image signal output from the image sensing means and for detecting, from the image signal, the spatial inclination in the frame and the time spatial inclination between frames to determine the movement vectors in each of the detected blocks in the frame, the blocks corresponding to portions of an image sensing plane of the image sensing means and said movement vector operation means transmitting movement vectors for each of the blocks;

(B) diagonal component operation means for detecting a spatial inclination component in a diagonal direction of the image signal for each of the blocks;

(C) averaging means for averaging the movement vectors from each of the blocks transmitted from said movement vector operation means by weighting them in accordance with the degree of diagonal spatial inclinations detected by said diagonal component detection means to generate a weighted-averaged movement vector, and;

(D) output means for outputting the weighted-averaged movement vector generated by said averaging means.

8. A movement vector detection device according to claim 7, wherein said movement vector operation means determines the movement vector in accordance with an approximation operation expression which has been formed by estimating the influence of the diagonal spatial inclination of the image signal.

9. A movement vector detection device according to claim 7, wherein said third operation means averages a plurality of movement vectors determined by said first operation means by weighing them in accordance with the degree of spatial inclinations detected by said second operation means so that he influence of a particular movement vector is reduced in proportion to the degree of that movement vector'"'"'s spatial inclination.

10. A movement vector detection device for detecting an image movement amount of an image signal output from image sensing means based on spatial inclination in a frame detected in the image signal, said movement vector detection device comprising:

(A) filter means for band-limiting a spatial inclination component in a diagonal direction of the image signal and for outputting a diagonal spatial inclination component;

(B) operation means for receiving the image signal from the image sensing means and for determining the movement vector based on information indicating the spatial inclination in the frame and timer spatial inclination between frames detected from the image signal in accordance with an approximation operation expression formed by estimating the influence of the diagonal spatial inclination component output from said filter means; and

(C) output means for outputting the movement vector determined by said operation means.

11. A movement vector detection device according to claim 10, wherein said filter means comprises a two-dimensional cross filter which transmits only horizontal and vertical frequency components of the image signal and stops a diagonal component.

12. A movement vector detection device according to claim 11 further comprising a two-dimensional Fourier transformation circuit and a two-dimensional Fourier inverse transformation circuit disposed across said filter means.

13. A movement vector detection device according to claim 12, wherein said filter means transmits only horizontal and vertical spatial regions of the image signal and stops a diagonal component, said filter means comprising a plurality of delay circuits, a band pass filter and an addition circuit for adding signals while weighting the signals with a predetermined weight.

14. A movement vector detection device according to claim 10, wherein said filter means limits the bend of the diagonal spatial inclination component by performing a mask convolution operation of the image signal in a spatial region.

15. A movement vector detection device for detecting a movement amount of an image signal output from image sensing means based on a spatial inclination in a frame of the image signal, said movement vector detection device comprising:

(A) first operation means for receiving the image signal from the image sensing means and for determining a movement vector for each of predetermined blocks in the frame, the blocks corresponding to portions on an image sensing plane of the image sensing means, by using a spatial inclination component in said frame;

(B) second operation means for detecting a diagonal spatial inclination component of the image signal for each of the blocks;

(C) classifying means for classifying the blocks into patterns based on a result of the operation performed by said second operation means;

(D) averaging means for averaging the movement vectors for the blocks by weighting the movement vectors in accordance with a classification to which the corresponding block belongs to generate an averaged movement vector; and

(E) output means for outputting the averaged movement vector generated by said average means.

16. A movement vector detection device according to claim 15, wherein said first operation means further comprises density difference detection means for detecting the density difference between a plurality of frames, wherein said movement vector is obtained by operating the output from said density difference detection means and the spatial inclination.

17. A movement vector detection device according to claim 16, wherein said first operation means determines said movement vector in accordance with an approximation operation expression formed by estimating the influence of the diagonal spatial inclination of said image signal.

18. A movement vector detection device according to claim 17, wherein said classifying means classifies the blocks in accordance with a degree of the diagonal spatial inclination detected by said second operation means, said averaging means averages the blocks classified by said classifying means by weighting them in accordance with the degree of the diagonal spatial inclinations in such a manner that the weight is in inverse proportion to the diagonal spatial inclination and the influence of any particular movement vector on the average is reduced in proportion to the degree of that movement vector'"'"'s spatial inclination component.

19. A movement vector detection device according to claim 15, wherein said classifying means performs said classification in such a manner that blocks in which the diagonal spatial inclination is larger than a predetermined threshold are omitted from the classifying operation and the diagonal spatial inclinations of only blocks diagonal spatial inclinations are in which the smaller than the threshold, are classified.

20. A movement vector detection device according to claim 19, wherein said classifying means comprises means for comparing an x-directional spatial inclination and a y-directional spatial inclination to weight the x-directional and y-directional evaluations.

21. An image deviation compensating device comprising:

(A) image pickup means;

(B) first operation means for operating a movement vector by detecting a spatial inclination in a frame from an image signal transmitted from said image pickup means and by using said spatial inclination;

(C) second operation means for detecting a spatial inclination component in a diagonal direction of said image signal;

(D) third operation means for averaging the output from said first operation means by weighting that output in accordance with the output from said second operation means; and

(E) movement compensation means for compensating the movement of an image in accordance with the output from said third operations means.

22. An image deviation compensating device according to claim 21, wherein said compensation means comprises means for compensating the movement of the image by optically changing the optical axis of an incidental beam upon said image pickup means.

23. An image deviation compensating device according to claim 22, wherein said movement compensating means comprises a variable-apex prism.

24. An image deviation compensating device according to claim 21 further comprising storage means for storing image signals transmitted from said image pickup means for one frame, said compensation means comprising means for changing an address of reading a stored image from said storage means in accordance with said movement vector transmitted from said third operation means, the address being changed in a direction in which the movement of said image is compensated.

25. An image deviation compensating device comprising:

(A) image pickup means;

(B) filter means for band-limiting of a spatial inclination component of an image signal transmitted from said image pickup means in a diagonal direction of an image;

(C) operation means for operating a movement vector from the image signal in accordance with an approximation operation expression formed by estimating the influence of a diagonal spatial inclination component from the output from said filter means; and

(D) movement compensation means for compensating the movement of an image in accordance with the movement vector transmitted from said operation means.

26. An image deviation compensating device according to claim 25, wherein said compensation means comprises means for compensating the movement of the image by optically changing the optical axis of an incidental beam upon said image pickup means.

27. An image deviation compensating device according to claim 26 further comprising storage means for storing image signals transmitted from said image pickup means for one frame, said compensation means comprising means for changing an address of reading a stored image from said storage means in accordance with the movement vector transmitted from said operation means, the address being changed in a direction in which the movement of the image is compensated.

28. An image deviation compensating device comprising:

(A) image pickup means for providing an image signal corresponding to an image;

(B) first operation means for obtaining the spatial inclination component in a frame from the image signal provided from said image pickup means, the frame comprising predetermined blocks, and for operating a movement vector in each of the predetermined blocks in the frame in accordance with the spatial inclination component;

(C) second operation means for detecting the spatial inclination component in a diagonal direction of the image from the image signal;

(D) classifying means for classifying each of the blocks into predetermined patterns in accordance with the result of the detection by said second operation means;

(E) averaging means for averaging the movement vector of each of the blocks calculated by said first operation means in accordance with a classification made by said classifying means while weighting the movement vectors by a predetermined weight; and

(F) movement compensating means for compensating the movement of the image in accordance with the movement vector averaged by said averaging means.

29. An image deviation compensating device according to claim 28, wherein said compensation means comprises means for compensating the movement of the image by optically changing the optical axis of an incidental beam upon said image pickup means.

30. An image deviation compensating device according to claim 28 further comprising storage means for storing image signals transmitted from said image pickup means for one frame, said compensation means comprising means for changing an address of reading a stored image from said storage means in accordance with the movement vector transmitted from said operation means, the address being changed in a direction in which the movement of the image is compensated.