Optical flow detection system

• US 5,627,905 A
• Filed: 12/12/1994
• Issued: 05/06/1997
• Est. Priority Date: 12/12/1994
• Status: Expired due to Term

• Current Assignee: Loral Defense Systems
• Original Assignee: Lockheed Martin Tactical Defense Systems (Martin Marietta Corporation)
• Inventors: Sebok, Dale R., Sebok, Thomas J.
• Primary Examiner(s): Johns, Andrew
• Assistant Examiner(s): Bella, Matthew C.
• Application Number:
  US08/353,589
• Time in Patent Office: 876 Days
• Field of Search: 382/103, 382/107, 364/516, 364/174, 348/169, 348/416, 348/699, 348/155, 348/208
• US Class Current: 382/107
• CPC Class Codes: G01S 3/7865 using correlation of the live video image with a stored image
• CPC Class Codes: G06T 7/269 using gradient-based methods
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6815 by distinguishing pan or tilt from motion
• CPC Class Codes: H04N 23/682 Vibration or motion blur correction
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20

Abstract

An optical flow detection system is employed in conjunction with a target cuer system, which includes a sensor carried on a platform, for the purpose of either removing undesirable sensor motion or for the purpose of removing undesirable background motion so that targets may be more easily identified. The optical flow detection system includes an optical flow estimator which receives a plurality of pixels and generates therefrom an instantaneous pixel velocity value and a predicted pixel velocity value. The instantaneous pixel velocity values and predicted pixel velocity values are then manipulated to remove undesirable features included in the image sequence generated by the sensor. Methods for operating the optical flow detection system are also provided.

Claims

1. An optical flow detection system for receiving a plurality of pixels so as to derive an instantaneous pixel velocity value and a predicted pixel velocity value for each pixel, comprising:

sensor means for supplying an imaging sequence which includes potential targets represented by at least one pixel;

an image queue for storing said imaging sequence;

gradient means for receiving and comparing said imaging sequence to said stored imaging sequence so as to generate a pixel intensity gradient value for each of said pixels;

calculation means for receiving said pixel intensity gradient values and a predicted pixel velocity value so as to generate an instantaneous pixel velocity value for each pixel; and

smoothing means for receiving said instantaneous pixel velocity values which are employed in a least squares fitting to generate coefficients which are used in a set of quadratic equations to generate said predicted pixel velocity values.

2. The optical flow detection system according to claim 1, wherein said smoothing means further comprises:

extraction means for receiving said instantaneous pixel velocity values and deariving therefrom a plurality of pixel velocity statistics;

fitting means for receiving said plurality of pixel velocity statistics and performing a least squares fitting of said pixel velocity statistics to said instantaneous pixel velocity value so as to generate a plurality of pixel velocity coefficients; and

expansion means for employing said pixel velocity coefficients in a set of quadratic equations so as to generate said predicted pixel velocity value for each pixel.

3. The optical flow detection system according to claim 2, wherein said set of quadratic equations is modeled upon said sensor means'"'"' expected motion.

4. The optical flow detection system according to claim 3, further comprising:

filtering means for receiving said instantaneous and predicted pixel velocity values so as to generate a combined filtered pixel velocity value;

correction means for receiving said combined filtered pixel velocity value and said imaging sequence so as to generate a stabilized imaging sequence; and

display means for providing a visual display of said stabilized imaging sequence wherein the optical flow detection system removes selected undesirable motion of said sensor means.

5. The optical flow detection system according to claim 4, wherein said filtering means further comprises:

a motion rejection filter for receiving said predicted pixel velocity values and removing those pixel velocity values attributable to motion of said sensor means so as to generate a filtered predicted pixel velocity;

accumulator means for receiving said filtered predicted pixel velocity, said instantaneous pixel velocity, and a delayed accumulated pixel velocity value so as to generate said combined filtered pixel velocity value, wherein said combined filtered pixel value is received by a time constant, said time constant generating said delayed accumulated pixel velocity value.

6. The optical flow detection system according to claim 5, wherein said accumulator means receives a plurality of selectively employed reset signals to remove undesirable background motion contained within said combined filtered pixel velocity value.

7. The motion detection system according to claim 2, wherein said set of quadratic equations is modeled upon background pixel motion within said imaging sequence, wherein background pixel motion includes expected curvilinear motion.

8. The optical flow detection system according to claim 7, further comprising:

a second image queue for storing said imaging sequence;

correction means for receiving said instantaneous and said predicted pixel velocity values and said imaging sequence so as to generate a corrected image sequence signal;

a subtractor for subtracting said stored imaging sequence from said corrected image sequence signal so as to generate a moving pixel signal; and

an object motion detection and tracking circuit for receiving said moving pixel signal so as to generate a tracking signal which clusters and confirms said moving pixel signal as potential targets wherein the optical flow detection system identifies pixels moving with respect to a background scene contained within said imaging sequence.

9. A method for estimating the optical flow of a plurality of pixels by employing a set of quadratic equations to generate a plurality of instantaneous pixel velocity values and a plurality of predicted pixel velocity values for each pixel, comprising the steps of:

providing a sensor to supply a real-time imaging sequence represented by a plurality of pixels;

storing said imaging sequence in an image queue;

comparing said stored imaging sequence to said imaging sequence so as to generate an intensity gradient value for each of said pixels;

calculating an instantaneous pixel velocity value for each pixel derived from said pixel intensity gradient values and a predicted pixel velocity value by employing a set of quadratic equations;

extracting a plurality of pixel velocity statistics from said instantaneous pixel velocity values;

using said pixel velocity statistics by a least squares fitting of said pixel velocity statistics to said instantaneous pixel velocity values so as to generate a plurality of pixel velocity coefficients; and

providing said pixel velocity coefficients in said set of quadratic equations to generate said predicted pixel velocity values for each pixel.

10. The method according to claim 9, further including the step of:

modelling said set of quadratic equations upon said sensor means'"'"' expected motion so as to remove selected undesirable sensor motion.

11. A method according to claim 10, further including the steps of:

filtering said instantaneous pixel velocity values and said predicted pixel velocity values so as to generate a combined filtered pixel velocity value for each pixel;

correcting said combined filtered pixel velocity values by combining therewith said imaging sequence so as to generate a stabilized imaging sequence represented by a plurality of stabilized pixels; and

displaying visually said plurality of stabilized pixels.

12. The method, as set forth in claim 11, wherein said step of filtering includes the steps of:

providing a motion rejection filter for receiving said predicted pixel velocity value for each pixel;

removing those pixel velocity values attributable to motion of said sensor means so as to provide a filtered predicted pixel velocity value for each pixel; and

accumulating said filtered predicted pixel velocity value, said instantaneous pixel velocity value and a delayed accumulated pixel velocity value so as to provide a combined filtered pixel velocity value.

13. The method as set forth in claim 12, further including the step of directing said combined filtered pixel velocity value through a time constant so as to provide said delayed accumulated pixel velocity.

14. The method as set forth in claim 13, wherein said step of accumulating includes the step of receiving a plurality of selectively employed reset signals to remove undesirable background motion contained within said combined filtered pixel velocity value.

15. The method according to claim 9, further including the step of:

modelling said set of quadratic equations upon background pixel motion within said imaging sequencer wherein background pixel motion includes expected curvilinear motion.

16. The method according to claim 15, further including the steps of:

storing said imaging sequence in a second image queue;

correcting said instantaneous and said predicted pixel velocity values and said imaging sequence so as to generate a corrected image sequence signal;

subtracting said stored imaging sequence from said corrected image sequence signal so as to generate a moving pixel signal; and

detecting and tracking said moving pixel signal so as to generate a tracking signal which clusters and confirms said moving pixel signal as potential targets.

17. A method for estimating the optical flow of a plurality of pixels contained within a video-based imaging sequence, comprising the steps of:

providing a sensor carried on a platform to supply a real-time imaging sequence represented by a plurality of pixels and storing said imaging sequence in an image queue;

generating directional intensity values for each of said pixels by comparing said stored imaging sequence to said imaging sequence;

employing a set of quadratic equations to calculate an instantaneous pixel velocity value for each pixel by using said directional intensity values and a plurality of predicted pixel velocity values;

extracting a plurality of pixel velocity statistics from said instantaneous pixel velocity values;

generating a plurality of pixel velocity coefficients by a least squares fit to previous iterations of said instantaneous pixel velocity values; and

generating said plurality of predicted pixel velocity values for each pixel by employing said plurality of pixel velocity coefficients in said set of quadratic equations.

18. The method according to claim 17, further including the step of:

modelling said set of quadratic equations upon said sensor'"'"'s expected motion wherein estimating the optical flow of said plurality of pixels contained within a video-based imaging sequence allows the removal of selected undesirable sensor motion.

19. The method according to claim 18, further including the step of:

modelling said set of quadratic equations upon background pixel motion within said imaging sequence wherein the optical flow detection system identifies pixels moving with respect to a background scene contained within said imaging sequence.