Tracking apparatus and methods using image processing noise reduction

• US 7,860,344 B1
• Filed: 05/06/2005
• Issued: 12/28/2010
• Est. Priority Date: 05/06/2005
• Status: Expired due to Fees

• Current Assignee: Stochastech Corporation
• Original Assignee: Stochastech Corporation
• Inventors: Wang, Yun, Fitzpatrick, Ben G.
• Primary Examiner(s): Le; Brian Q
• Assistant Examiner(s): Tsai; Tsung-Yin
• Application Number:
  US11/123,549
• Time in Patent Office: 2,062 Days
• Field of Search: 382/291
• US Class Current: 382/291
• CPC Class Codes: G06F 18/24155 Bayesian classification
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/20182 Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering
• CPC Class Codes: G06T 2207/20216 Image averaging
• CPC Class Codes: G06T 2207/30212 Military
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 7/77 using statistical methods
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects

Abstract

Improved apparatus and methodology for image processing and object tracking that, inter alia, reduces noise. In one embodiment, the methodology is applied to moving targets such as missiles in flight, and comprises processing sequences of images that have been corrupted by one or more noise sources (e.g., sensor noise, medium noise, and/or target reflection noise). In this embodiment, a multi-dimensional image is acquired for a first time step t; the acquired image is normalized and sampled, and then segmented into target and background pixel sets. Intensity statistics of the pixel sets are determined, and a prior probability image from a previous time step smoothed. The smoothed prior image is then shifted to produce an updated prior image, and a posterior probability image calculated using the updated prior probability. Finally, the position of the target is extracted using the posterior probability image. A tracking system and controller utilizing this methodology are also disclosed.

Claims

1. A method of estimating the position of a target with a tracking system using a sequence of images, comprising:

acquiring a multi-dimensional image from said tracking system for a time step t;

normalizing said image to produce a normalized image, said normalized image having an intensity within a given range;

super-sampling said normalized image with at least one sample rate to produce a super sampled image;

classifying said normalized image into target and background pixel sets;

determining at least one statistic of said pixel sets, said act of determining at least one statistic comprising;

updating for a plurality of prior time steps a temporal mean and standard deviation from said normalized image;

calculating from said temporal mean and standard deviation;

(i) a target mean and standard deviation of all pixels in said target set, and (ii) a background mean and standard deviation of all pixels in said background set;

scaling the target and background mean standard deviation by a factor; and

setting minimum and maximum bounds on said means and standard deviations;

smoothing a prior probability from a previous time step;

shifting the prior probability to produce an updated prior probability;

calculating a posterior probability using at least said updated prior probability; and

extracting a position of said target for at least one time step using at least said posterior probability.

2. The method of claim 1, wherein said act of classifying comprises:

calculating an average image from said normalized image;

selecting said target pixel set from said average image, said target pixel set comprising a collection of pixels having an intensity larger than a predetermined value; and

selecting said background set having at least first and second regions associated therewith, said selection of said background set being based at least in part on one or more pixels in said target pixel set.

3. The method of claim 1, wherein said act of smoothing further comprises estimating a smoothing parameter by at least:

estimating at least one open loop target position fluctuation by taking a difference between (i) a measured control position from a time step prior to t, and (ii) an error signal from a time step prior to t;

calculating for all time steps up to t at least one target position difference;

updating for a plurality of prior time steps a standard deviation from said target position difference; and

scaling said standard deviation by a factor.

4. The method of claim 1, wherein said act of extracting a position comprises:

smoothing the posterior probability using a smoothing function to produce a smoothed posterior probability;

combining said smoothed posterior with said super-sampled image, to produce an intensity-weighted posterior probability;

generating an estimate of an initial component of the target position in a first dimension based at least in part on said intensity weighted posterior probability; and

generating an estimate of an initial component of the target position in a second dimension based at least in part on said smoothed posterior probability.

5. A method of estimating the position of a target with a tracking system using a sequence of images, comprising:

acquiring a multi-dimensional image from said tracking system for a time step t;

normalizing said image to produce a normalized image, said normalized image having an intensity within a given range;

super-sampling said normalized image with at least one sample rate to produce a super sampled image;

classifying said normalized image into target and background pixel sets;

determining at least one statistic of said pixel sets;

smoothing a prior probability from a previous time step, said act of smoothing comprising;

estimating a smoothing parameter by at least;

estimating at least one open loop target position fluctuation by taking a difference between (i) a measured control position from a time step prior to t, and (ii) an error signal from a time step prior to t;

calculating for time steps up to t at least one target position difference;

updating for a plurality of prior time steps a standard deviation from said target position difference; and

scaling said standard deviation by a factor;

shifting the prior probability to produce an updated prior probability;

calculating a posterior probability using at least said updated prior probability; and

extracting a position of said target for at least one time step using at least said posterior probability.

6. The method of claim 5, further comprising acquiring a control position at said time step t;

wherein said act of shifting is based at least in part on said control position.

7. The method of claim 5 wherein said act of classifying comprises:

calculating an average image from said normalized image;

selecting said target pixel set from said average image, said target pixel set comprising a collection of pixels having an intensity larger than a predetermined value; and

selecting said background set having at least first and second regions associated therewith, said selection of said background set being based at least in part on one or more pixels in said target pixel set.

8. The method of claim 5, wherein said act of extracting a position comprises:

smoothing the posterior probability using a smoothing function to produce a smoothed posterior probability;

combining said smoothed posterior with said super-sampled image, to produce an intensity-weighted posterior probability;

generating an estimate of an initial component of the target position in a first dimension based at least in part on said intensity weighted posterior probability; and

generating an estimate of an initial component of the target position in a second dimension based at least in part on said smoothed posterior probability.

9. A method of estimating the position of a target with a tracking system using a sequence of images, comprising:

acquiring a multi-dimensional image from said tracking system for a time step t;

normalizing said image to produce a normalized image, said normalized image having an intensity within a given range;

super-sampling said normalized image with at least one sample rate to produce a super sampled image;

classifying said normalized image into target and background pixel sets;

determining at least one statistic of said pixel sets;

smoothing a prior probability from a previous time step;

shifting the prior probability to produce an updated prior probability;

calculating a posterior probability using at least said updated prior probability; and

extracting a position of said target for at least one time step using at least said posterior probability, said act of extracting a position comprising;

smoothing the posterior probability using a smoothing function to produce a smoothed posterior probability;

combining said smoothed posterior with said super-sampled image, to produce an intensity-weighted posterior probability;

generating an estimate of an initial component of the target position in a first dimension based at least in part on said intensity weighted posterior probability; and

generating an estimate of an initial component of the target position in a second dimension based at least in part on said smoothed posterior probability.

10. The method of claim 9, wherein said act of classifying comprises:

calculating an average image from said normalized image;

selecting said target pixel set from said average image, said target pixel set comprising a collection of pixels having an intensity larger than a predetermined value; and

selecting said background set having at least first and second regions associated therewith, said selection of said background set being based at least in part on one or more pixels in said target pixel set.

11. The method of claim 9, wherein said act of determining at least one statistic comprises:

updating for a plurality of prior time steps a temporal mean and standard deviation from said normalized image;

calculating from said temporal mean and standard deviation;

(i) a target mean and standard deviation of all pixels in said target set, and (ii) a background mean and standard deviation of all pixels in said background set;

scaling the target and background mean standard deviation by a factor; and

setting minimum and maximum bounds on said means and standard deviations.

12. The method of claim 9, wherein said act of smoothing further comprises estimating a smoothing parameter by at least:

estimating at least one open loop target position fluctuation by taking a difference between (i) a measured control position from a time step prior to t, and (ii) an error signal from a time step prior to t;

calculating for all time steps up to t at least one target position difference;

updating for a plurality of prior time steps a standard deviation from said target position difference; and

scaling said standard deviation by a factor.

13. A method of estimating the position of a target with a tracking system using a sequence of images, comprising:

acquiring a multi-dimensional image from said tracking system for a time step t;

normalizing said image to produce a normalized image, said normalized image having an intensity within a given range;

super-sampling said normalized image with at least one sample rate to produce a super sampled image;

classifying said normalized image into target and background pixel sets;

determining at least one statistic of said pixel sets;

smoothing a prior probability from a previous time step;

computing a convolution kernel;

shifting the prior probability to produce an updated prior probability using said convolution kernel;

calculating a posterior probability using at least said updated prior probability; and

extracting a position of said target for at least one time step using at least said posterior probability.

14. The method of claim 13, wherein said act of smoothing further comprises estimating a smoothing parameter by at least:

estimating at least one open loop target position fluctuation by taking a difference between (i) a measured control position from a time step prior to t, and (ii) an error signal from a time step prior to t;

calculating for all time steps up to t at least one target position difference;

updating for a plurality of prior time steps a standard deviation from said target position difference; and

scaling said standard deviation by a factor.

15. The method of claim 13, wherein said act of extracting a position comprises:

smoothing the posterior probability using a smoothing function to produce a smoothed posterior probability;

combining said smoothed posterior with said super-sampled image, to produce an intensity-weighted posterior probability;

generating an estimate of an initial component of the target position in a first dimension based at least in part on said intensity weighted posterior probability; and

generating an estimate of an initial component of the target position in a second dimension based at least in part on said smoothed posterior probability.

16. A method of estimating the position of a target with a tracking system using a sequence of images, comprising:

acquiring a multi-dimensional image from said tracking system for a time step t;

normalizing said image to produce a normalized image, said normalized image having an intensity within a given range;

super-sampling said normalized image with at least one sample rate to produce a super sampled image;

classifying said normalized image into target and background pixel sets;

determining at least one statistic of said pixel sets, said act of determining at least one statistic further comprising;

updating for a plurality of prior time steps a temporal mean and standard deviation from said normalized image;

calculating from said temporal mean and standard deviation;

(i) a target mean and standard deviation of all pixels in said target set, and (ii) a background mean and standard deviation of all pixels in said background set;

scaling the target and background mean standard deviation by a factor; and

setting minimum and maximum bounds on said means and standard deviations;

smoothing a prior probability from a previous time step;

shifting the prior probability to produce an updated prior probability;

calculating a posterior probability using at least said updated prior probability; and

extracting a position of said target for at least one time step using at least said posterior probability.

17. The method of claim 16, further comprising acquiring a control position at said time step t;

wherein said act of shifting is based at least in part on said control position.

18. The method of claim 16, wherein said act of classifying comprises:

calculating an average image from said normalized image;

selecting said target pixel set from said average image, said target pixel set comprising a collection of pixels having an intensity larger than a predetermined value; and

selecting said background set having at least first and second regions associated therewith, said selection of said background set being based at least in part on one or more pixels in said target pixel set.

19. The method of claim 16, wherein said act of smoothing further comprises estimating a smoothing parameter by at least:

estimating at least one open loop target position fluctuation by taking a difference between (i) a measured control position from a time step prior to t, and (ii) an error signal from a time step prior to t;

calculating for all time steps up to t at least one target position difference;

updating for a plurality of prior time steps a standard deviation from said target position difference; and

scaling said standard deviation by a factor.

20. The method of claim 16, wherein said act of extracting a position comprises:

smoothing the posterior probability using a smoothing function to produce a smoothed posterior probability;

combining said smoothed posterior with said super-sampled image, to produce an intensity-weighted posterior probability;

generating an estimate of an initial component of the target position in a first dimension based at least in part on said intensity weighted posterior probability; and

generating an estimate of an initial component of the target position in a second dimension based at least in part on said smoothed posterior probability.

21. A method of estimating the position of a target with a tracking system using a sequence of images, comprising:

acquiring a multi-dimensional image from said tracking system for a time step t;

normalizing said image to produce a normalized image, said normalized image having an intensity within a given range;

super-sampling said normalized image with at least one sample rate to produce a super sampled image;

classifying said normalized image into target and background pixel sets;

determining at least one statistic of said pixel sets;

smoothing a prior probability from a previous time step, said act of smoothing further comprising estimating a smoothing parameter by at least;

estimating at least one open loop target position fluctuation by taking a difference between (i) a measured control position from a time step prior to t, and (ii) an error signal from a time step prior to t;

calculating for all time steps up to t at least one target position difference;

updating for a plurality of prior time steps a standard deviation from said target position difference; and

scaling said standard deviation by a factor;

shifting the prior probability to produce an updated prior probability;

calculating a posterior probability using at least said updated prior probability; and

extracting a position of said target for at least one time step using at least said posterior probability.

22. The method of claim 21, further comprising acquiring a control position at said time step t;

wherein said act of shifting is based at least in part on said control position.

23. The method of claim 21, wherein said act of classifying comprises:

calculating an average image from said normalized image;

selecting said target pixel set from said average image, said target pixel set comprising a collection of pixels having an intensity larger than a predetermined value; and

selecting said background set having at least first and second regions associated therewith, said selection of said background set being based at least in part on one or more pixels in said target pixel set.

24. The method of claim 21, wherein said act of determining at least one statistic comprises:

updating for a plurality of prior time steps a temporal mean and standard deviation from said normalized image;

calculating from said temporal mean and standard deviation;

(i) a target mean and standard deviation of all pixels in said target set, and (ii) a background mean and standard deviation of all pixels in said background set;

scaling the target and background mean standard deviation by a factor; and

setting minimum and maximum bounds on said means and standard deviations.

25. The method of claim 21, wherein said act of extracting a position comprises:

smoothing the posterior probability using a smoothing function to produce a smoothed posterior probability;

combining said smoothed posterior with said super-sampled image, to produce an intensity-weighted posterior probability;

generating an estimate of an initial component of the target position in a first dimension based at least in part on said intensity weighted posterior probability; and

generating an estimate of an initial component of the target position in a second dimension based at least in part on said smoothed posterior probability.

26. A method of estimating the position of a target with a tracking system using a sequence of images, comprising:

acquiring a multi-dimensional image from said tracking system for a time step t;

normalizing said image to produce a normalized image, said normalized image having an intensity within a given range;

super-sampling said normalized image with at least one sample rate to produce a super sampled image;

classifying said normalized image into target and background pixel sets;

determining at least one statistic of said pixel sets;

smoothing a prior probability from a previous time step;

shifting the prior probability to produce an updated prior probability;

calculating a posterior probability using at least said updated prior probability; and

extracting a position of said target for at least one time step using at least said posterior probability;

wherein said act of extracting a position comprises;

smoothing the posterior probability using a smoothing function to produce a smoothed posterior probability;

combining said smoothed posterior with said super-sampled image, to produce an intensity-weighted posterior probability;

generating an estimate of an initial component of the target position in a first dimension based at least in part on said intensity weighted posterior probability; and

generating an estimate of an initial component of the target position in a second dimension based at least in part on said smoothed posterior probability.

27. The method of claim 26, further comprising acquiring a control position at said time step t;

wherein said act of shifting is based at least in part on said control position.

28. The method of claim 26, wherein said act of classifying comprises:

calculating an average image from said normalized image;

selecting said target pixel set from said average image, said target pixel set comprising a collection of pixels having an intensity larger than a predetermined value; and

selecting said background set having at least first and second regions associated therewith, said selection of said background set being based at least in part on one or more pixels in said target pixel set.

29. The method of claim 26, wherein said act of determining at least one statistic comprises:

updating for a plurality of prior time steps a temporal mean and standard deviation from said normalized image;

calculating from said temporal mean and standard deviation;

(i) a target mean and standard deviation of all pixels in said target set, and (ii) a background mean and standard deviation of all pixels in said background set;

scaling the target and background mean standard deviation by a factor; and

setting minimum and maximum bounds on said means and standard deviations.

30. The method of claim 26, wherein said act of smoothing further comprises estimating a smoothing parameter by at least:

estimating at least one open loop target position fluctuation by taking a difference between (i) a measured control position from a time step prior to t, and (ii) an error signal from a time step prior to t;

calculating for all time steps up to t at least one target position difference;

updating for a plurality of prior time steps a standard deviation from said target position difference; and

scaling said standard deviation by a factor.