Moving vector extractor

• US 5,430,479 A
• Filed: 10/16/1991
• Issued: 07/04/1995
• Est. Priority Date: 10/16/1990
• Status: Expired due to Term

• Current Assignee: Sharp Corporation (Hon Hai Precision Industry Co., Ltd.), YOZAN, Inc.
• Original Assignee: Ezel, Inc., Sharp Corporation (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Yamane, Yasukuni, Takahama, Kengo
• Primary Examiner(s): Groody, James J.
• Assistant Examiner(s): Burgess, Glenton B.
• Application Number:
  US07/776,069
• Time in Patent Office: 1,357 Days
• Field of Search: 358/105, 358/133, 358/136, 358/141, 358/209, 358/225, 358/227, 358/229, 382/1, 348/400, 348/669, 348/416, 348/208
• US Class Current: 348/207.99
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/223 using block-matching
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/682 Vibration or motion blur correction
• CPC Class Codes: H04N 23/687 by shifting the lens or sensor position

Abstract

A moving vector extractor for detecting a shaking of an image through a video camera in order to improve accuracy of the detection. Even when the image is dark or of low density gradient as a whole, a density gradient value around representative point is calculated. Basic data of moving vector of the difference value between successive frames is obtained by calculating an absolute value or a squared value. The basic data of the moving vector is weighted according to a density gradient value so as to improve the precision of moving vector extraction.

Claims

1. A moving vector extractor comprising:

a timer generating a control signal having a predetermined timing;

an image processor for receiving an input image signal which represents a physical image and for generating, responsive to said control signal, a plurality of brightness signals which represent said physical image at a plurality of successive image frames;

a representative point memory for extracting and storing representative points from said brightness signals according to said control signal generated by said timer, said representative points having corresponding brightness values;

subtracting means for calculating a difference between said brightness values of said stored representative points which are in proximate image frames;

absolute value calculating means for calculating an absolute value of said difference calculated by said subtracting means;

density gradient calculating means for calculating a density gradient in a neighborhood of said representative points;

weighting means for weighting said absolute value of said difference according to said calculated density gradient;

accumulating means for accumulating said absolute value of said weighted difference;

an accumulation memory for storing two-dimensionally an accumulated result obtained by said accumulating means corresponding to said input image signal;

a minimum value detecting means for detecting a minimum value from accumulated results stored in said accumulating means; and

moving vector extracting means for extracting a moving vector based on said detected minimum value.

2. The moving vector extractor as claimed in claim 1, wherein said indicator value is an absolute value of said difference valve.

3. The moving vector extractor as claimed in claim 1, wherein said indicator valve is a squared value of said difference valve.

4. The moving vector extractor as claimed in claim 1, wherein said predetermined area is defined as a block area having a predetermined size, said predetermined size being obtained by dividing said frames into a number of subparts.

5. The moving vector extractor as claimed in claim 1, wherein said weighting means multiplies said indicator value by a weight of "0", when said density gradient value is less than a density gradient threshold value, and by a weight of "1", when said density gradient value is equal to or more than said density gradient threshold value.

6. The moving vector extractor as claimed in claim 1, wherein said density gradient value is obtained from an addition of a horizontal density gradient value and a vertical density gradient value surrounding each of said representative points.

7. The moving vector extractor as claimed in claim 6, wherein said horizontal and vertical density gradient values are calculated according to a Sobel operator.

8. The moving vector extractor as claimed in claim 6, wherein said horizontal and vertical density gradient values are calculated according to a Prewitt operator.

9. The moving vector extractor as claimed in claim 6, wherein said horizontal and vertical density gradient values are calculated according to a Robers operator.

10. The moving vector extractor as claimed in claim 1, wherein said density means comprises a multiplier for multiplying a density of a pixel by a multiplier which is changeable according to a control signal.

11. A video camera comprising;

lens means for receiving an input image;

moving vector extractor means for extracting a moving vector from said input image comprising;

(a) difference means for calculating a difference value corresponding to brightnesses of representative points in successive frames of said input image, each of said representative points being in a predetermined area of each of said successive frames, said successive frames being a predetermined time period away from each other,

(b) density means for calculating a density gradient value around each of said representative points,

(c) indicating means for calculating an indicator value, said indicator value indicating said difference value as a positive value,

(d) weighting means for weighting said indicator value according to said density gradient value, thereby generating a weighted indicator value, and

(e) accumulating means for accumulating a summation of said weighted indicator value for each of said representative points and determining a local minimum in said summation, thereby extracting said moving vector from a positional relationship between said local minimum and said representative points;

actuator means for moving said lens means in a horizontal and a vertical direction; and

control means for controlling said movement of said actuator means in response to said moving vector extractor means.

12. A video camera as claimed in claim 11, further comprising;

zoom means for changing a magnification of said lens means; and

gain control means for changing a gain in said zoom means, said gain being controlled in reverse proportion to said magnification.

13. A moving vector extractor comprising:

an image processor for electronically generating an image output signal representative of a physical image;

image difference accumulation means for receiving said image output signal and for electronically generating signals, based on said image output signal, which are representative of time-varying differences of predetermined points in said physical image, said generated signals being responsive to local density gradients which correspond to said predetermined points; and

vector determination means for identifying a moving vector according to successive signals from said image difference accumulation means.

14. The moving vector extractor of claim 13, said image difference accumulation means comprising:

means for calculating a difference between corresponding points in said image at successive points in time; and

means for adjusting said difference according to said local density gradient of said corresponding points.

15. The moving vector extractor of claim 14, said means for calculating comprising:

a memory for storing signals representative of said image at a given point in time; and

subtracting means for calculating a difference between a brightness value of a point in said representative point memory and a corresponding point of said image.

16. The moving vector extractor of claim 15, further comprising a timer for causing said memory to store signals representative of said image.

17. The moving vector extractor of claim 15, said subtracting means comprising absolute value calculating means for calculating an absolute value of said difference between a brightness value of a point in said representative point memory and said corresponding point of said image.

18. The moving vector extractor of claim 15, said subtracting means comprising squared value calculating means for calculating a square of said difference between a brightness value of a point in said representative point memory and said corresponding point of said image.

19. The moving vector extractor of claim 14, said means for adjusting comprising:

density gradient calculating means for calculating said local density gradient of said point;

weighting means for weighting an output of said difference calculating means according to a value of said density gradient calculated by said density gradient calculating means.

20. The moving vector extractor as claimed in claim 19, wherein said weighting means multiplies said indicator value by a weight of "0" when an output of said density gradient calculating means is less than a density gradient threshold value, and by a weight of "1", when said output of said density gradient calculating means is equal to or greater than said density gradient threshold value.

21. The moving vector extractor as claimed in claim 19, wherein said density gradient value is obtained from an addition of a horizontal density gradient value and a vertical density gradient value surrounding said point.

22. The moving vector extractor as claimed in claim 21, wherein said horizontal and vertical density gradient values are calculated according to a Sobel operator.

23. The moving vector extractor as claimed in claim 21, wherein said horizontal and vertical density gradient values are calculated according to a Prewitt operator.

24. The moving vector extractor as claimed in claim 21, wherein said horizontal and vertical density gradient values are calculated according to a Robers operator.

25. The moving vector extractor of claim 13, said vector determination means comprising:

accumulation means for accumulating successive signals from said image difference accumulation means; and

means for detecting a minimum value among signals in said accumulation means.

26. The moving vector extractor of claim 25, said accumulation means comprising:

an accumulation memory for storing a two-dimensional representation of signals in said image difference accumulation means.

27. A moving vector extractor comprising:

a timer generating a control signal at predetermined intervals;

an image processor for providing, responsive to said control signal, an image output signal representative of a physical image as an image frame, said output signal comprising a plurality of multi-valued portions, each multi-valued portion being representative of a point in said physical image;

difference means for calculating a difference value corresponding to the brightness of representative points in successive frames of an input image, each of said representative points being in a predetermined area of said image and being represented by one of said plurality of multi-valued portions of said image output signal;

density means for calculating a density gradient value around each of said representative points;

indicator means for calculating an indicator value, said indicator value being based on a positive representation of said difference value;

weighting means for weighting said indicator value according to said density gradient value, thereby generating a weighted indicator value;

accumulating means for accumulating a summation of weighted indicator values including said weighted indicator value for each representative point, for determining a local minimum in said summation and for providing an accumulator output signal indicative of said local minimum; and

means for generating a signal representing a moving vector from a positional relationship between said local minimum and said representative points.

28. The moving vector extractor as claimed in claim 27, wherein said predetermined area is defined as a block area having a predetermined size, said predetermined size being obtained by dividing said frames into a number of subparts.

29. The moving vector extractor as claimed in claim 27, wherein said weighting means multiplies said indicator value by a weight of "0", when said density gradient value is less than a density gradient threshold value and by a weight of "1", when said density gradient value is equal to or more than said density gradient threshold value.

30. The moving vector extractor as claimed in claim 27, wherein said density gradient value is obtained from an addition of a horizontal density gradient value and a vertical density gradient value surrounding each of said representative points.

31. The moving vector extractor as claimed in claim 30, wherein said horizontal and vertical density gradient values are calculated according to a Sobel operator.

32. The moving vector extractor as claimed in claim 30, wherein said horizontal and vertical density gradient values are calculated according to a Prewitt operator.

33. The moving vector extractor as claimed in claim 30, wherein said horizontal and vertical density gradient values are calculated according to a Robers operator.

34. The moving vector extractor as claimed in claim 27, wherein said density means comprises a multiplier for multiplying a density of a pixel by a multiplier which is changeable according to a control signal.