Moving object scanning apparatus and method

• US 7,675,655 B2
• Filed: 02/27/2004
• Issued: 03/09/2010
• Est. Priority Date: 03/07/2003
• Status: Active Grant

• Current Assignee: Qinetiq Limited (QinetiQ Group PLC)
• Original Assignee: Qinetiq Limited (QinetiQ Group PLC)
• Inventors: Sinclair, Gordon Neil, Marshall, Gillian Fiona, Barrett, David John, Hughes, Janet, Kent, Philip John, Coward, Peter Russell, Savage, John Timothy, Hawkesley, Piers Thomas Winchcombe, Gillham, John Peter, Appleby, Roger
• Primary Examiner(s): Zimmerman; Mark K
• Assistant Examiner(s): Zheng; Jacky X
• Application Number:
  US10/547,652
• Publication Number:
  US 20060146377A1
• Time in Patent Office: 2,202 Days
• Field of Search: 358/486, 382/104, 382/103, 340/435, 342/22, 342/52, 348/149
• US Class Current: 358/486
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images

Abstract

Apparatus for scanning a moving object includes a visible waveband sensor 12 oriented to collect a series of images of the object as it passes through a field of view 16. An image processor 14 uses the series of images to form a composite image. The image processor 14 stores image pixel data for a current image and a predecessor image in the series. It uses information in the current image and its predecessor to analyse images and derive likelihood measures indicating probabilities that current image pixels correspond to parts of the object. The image processor 14 estimates motion between the current image and its predecessor from likelihood weighted pixels. It generates the composite image from frames positioned according to respective estimates of object image motion. Image motion may alternatively be detected by a speed sensor such as a Doppler radar 200 sensing object motion directly and providing image timing signals.

Claims

1. An apparatus for scanning a moving object, the apparatus having a sensor oriented to collect a series of images of the object as it passes through a sensor field of view and image processing means arranged to input the series of images and to form a composite image therefrom, the image processing means including:

a) motion estimating means for generating an estimate of object image motion between images in the series of images, including image analysing means for deriving, for pixels within a current image from the series of images, likelihood measures indicating likelihoods of respective pixels corresponding to parts of the object; and

b) display processing means for generating a composite image to which the series of images contributes at positions derived from estimated object image motion;

wherein

i) the series of images includes a predecessor image obtained prior to the current image, and the motion estimating means is arranged to generate an estimate of object image motion between the current image and the predecessor image from image pixels weighted in accordance with the likelihood measure, and

ii) the likelihood measure is derived from intensity of the current image pixel, intensity of a second pixel located in the predecessor image equivalently to the current image pixel, and intensity of a third pixel which is in the predecessor image and is estimated to be imaging an object region which is the same as that imaged by the current image pixel, the third pixel having a location derived by compensation for object motion, and estimates of standard deviations of noise in the object image intensity and background image intensity σ

_b in accordance with equations as follows;

$\mathrm{likelihood}<br><br><br><br>\mathrm{measure}=w_f<br><br>\left(x,y\right)=\frac{p_f<br><br>\left(x,y,\mathrm{\Delta <br><br>}<br><br><br><br>x_i\right)}{p_f<br><br>\left(x,y,\mathrm{\Delta <br><br>}<br><br><br><br>x_i\right)+p_b<br><br>\left(x,y\right)},<br><br>\mathrm{where}<br><br>\text{;<br><br>}$ $p_f<br><br>\left(x,y,\mathrm{\Delta <br><br>}<br><br><br><br>x_i\right)=\frac{1}{\sqrt[{\mathrm{\sigma <br><br>}}_b]{2<br><br><br><br>\mathrm{\pi <br><br>}}}<br><br>\exp <br><br>\left(-\frac{{\left(I_p^t<br><br>\left(x,y\right)-I_p^{t-1}<br><br>\left(x+\mathrm{\Delta <br><br>}<br><br><br><br>x_i,y\right)\right)}^2}{2<br><br><br><br>{\mathrm{\sigma <br><br>}}_f^2}\right),<br><br>\mathrm{and}$ $p_b<br><br>\left(x,y\right)=\frac{1}{\sqrt[{\mathrm{\sigma <br><br>}}_b]{2<br><br><br><br>\mathrm{\pi <br><br>}}}<br><br>\exp <br><br><br><br>\left(-\frac{{\left(I_p^t<br><br>\left(x,y\right)-I_p^{t-1}<br><br>\left(x,y\right)\right)}^2}{2<br><br><br><br>{\mathrm{\sigma <br><br>}}_b^2}\right).$

2. The apparatus according to claim 1, wherein the moving object is a vehicle controlled by a driver, and wherein the image processing means include image analysing means for deriving, for each of a number of pixels within a current image in the series of images, a respective likelihood measure indicating likelihood of pixel corresponding to a part of the vehicle, the image analysing means being arranged to use information contained within the current image and a predecessor thereof.

3. An apparatus for scanning a moving object, the apparatus having a sensor oriented to collect a series of images of the object as it passes through a sensor field of view and image processing means arranged to input the series of images and to form a composite image therefrom, the image processing means including:

a) motion estimating means for generating an estimate of object image motion between images in the series of images, including image analysing means for deriving, for pixels within a current image from the series of images, likelihood measures indicating likelihoods of respective pixels corresponding to parts of the object; and

b) display processing means for generating a composite image to which the series of images contributes at positions derived from estimated object image motion;

wherein

i) the series of images includes a predecessor image obtained prior to the current image, and the motion estimating means is arranged to generate an estimate of object image motion between the current image and the predecessor image from image pixels weighted in accordance with the likelihood measure, and

ii) the likelihood measure is derived from intensity of the current image pixel, intensity of a second pixel equivalently located in its predecessor and intensity of a third pixel which is in the predecessor image and which is estimated to be imaging an object region which is the same as that imaged by the current image pixel, the third pixel having a location derived by compensation for object motion, and the likelihood measure being given by;

$w_f<br><br>\left(x,y\right)=\frac{{\left(I_p^t<br><br>\left(x,y\right)-I_p^{t-1}<br><br>\left(x,y\right)\right)}^n}{{\left(I_p^t<br><br>\left(x,y\right)-I_p^{t-1}<br><br>\left(x,y\right)\right)}^n+{\left(I_p^t<br><br>\left(x,y\right)-I_p^{t-1}<br><br>\left(x+\mathrm{\Delta <br><br>}<br><br><br><br>x_i,y\right)\right)}^n+c<br><br>}$

where w_f(x,y) is the likelihood measure, I_p^t(x,y), I_p<sup>t−

1</sup>(x,y) and I_p<sup>t−

1</sup>(x+Δ

x_i,y) are the intensities of the first, second and third pixels respectively, n is a positive number and c is a non-negative constant.

4. The apparatus according to claim 3, wherein the image analysing means is arranged to use a first estimate of object motion to calculate a second such estimate, and thereafter to calculate iteratively the likelihood measure based on an estimate of object motion generated by the motion estimating means in a previous iteration, and the motion estimating means is arranged to generate iteratively an estimate of object motion for a current iteration based on likelihood measures updated using respective previous motion estimates.

5. The apparatus according to claim 3, wherein the sensor is a framing sensor with a substantially rectangular field of view and the display processing means is arranged, for each of the series of images, to take image data from a plurality of strips within the sensor field of view and to generate a corresponding plurality of composite images, each composite image having a contribution from a respective strip in each image.

6. The apparatus according to claim 3, wherein the sensor is a first sensor and the apparatus includes at least one other sensor oriented to obtain a respective series of images of the object as the object passes through the at least one other sensor'"'"'s field of view, the at least one additional sensor is at least one of a visible band imager, a millimeter wave imager and a thermal imager and the display processing means is further arranged to generate a respective composite image from each series of images in accordance with the estimate of image motion derived from the first sensory s series of images.

7. The apparatus according to claim 3, wherein the motion estimating means is an object speed sensor.

8. The apparatus according to claim 7, wherein the object speed sensor is a Doppler radar providing an output signal with frequency proportional to object speed.

9. The apparatus according to claim 8, arranged to provide a common clock signal to time the Doppler radar output signal and collection of images in the series of images in terms of counts of clock signal pulses and to provide respective frame signals to time images and time/position data derived from the Doppler radar output signal to time object movement, to perform an interpolation to relate image timing to object position and form the composite image from images located in accordance with object position.

10. A method of constructing a composite image from a series of images taken at a sensor as an object moves through a field of view of the sensor, the method comprising the steps of:

a) inputting a current image for processing;

b) deriving an estimate of object image motion and associating a respective estimate of object image motion with each image when designated as a current image;

c) associating the current image with a respective position in a data accumulator derived from summation of estimates of object image motion for the current image and like estimates obtained for earlier captured images in the series;

d) designating successive images in the series of images as the current image and iterating steps a) to c) for each current image so designated; and

e) constructing a composite image to which the series of image frames contributes from image central portions located according to respective positions in said data accumulator;

wherein step c) also includes the step of incrementing a count accumulator at pixel positions corresponding to those to which data is added to the data accumulator as appropriate to indicate the number of such additions for each pixel, and step e) comprises the step of dividing data accumulator pixel values with respective correspondingly positioned count accumulator pixel values.

11. The method according to claim 10, wherein each image central portion is at least as wide as an associated estimate of object image motion.

12. The method according to claim 10 including weighting both the count accumulator increment and pixel values in a central portion of the current image by left and right weighting factors to provide weighted increments and weighted pixel values respectively, the weighted increments are added to first and second integral pixel positions in the count accumulator and the weighted pixel values added to like-located pixel positions in the data accumulator, the integral pixel positions being those adjacent an estimatedly true pixel position derived from summation of object image motion for the current image and predecessors, and the left and right weighting factors being determined having regard to differences between the true pixel position and its adjacent integral pixel positions.

13. The method according to claim 10 including weighting both the count accumulator increment and pixel values in a central portion of the current image by weighting factors to provide products which are added to corresponding integral pixel positions within respective accumulators, the integral pixel positions being those adjacent an estimatedly true pixel position derived from summation of object image motion for the current image and earlier captured images and the series of weighting factors being determined in accordance with a smooth interpolation function.

14. The method according to claim 10, wherein step e) comprises the further steps of:

i) applying a spatial smoothing filter to data in the data accumulator and count accumulator; and

ii) forming the composite image either by dividing data accumulator pixels value by respective correspondingly positioned count accumulator pixel values or by dividing a filtered data accumulator pixel values by respective correspondingly positioned filtered count accumulator pixel values, depending on whether or not respective count accumulator pixel values are non-zero in each case.

15. The method according to claim 10, wherein step e) comprises the further steps of:

1) setting an initial neighbourhood range of one pixel for each pixel in the count accumulator;

2) implementing a count accumulator summation of pixel values within its respective neighbourhood range for each pixel in the count accumulator;

3) implementing a check regarding whether or not the count accumulator summation is significantly non-zero;

4) if the check has an outcome which is untrue, indicating that the count accumulator summation is not significantly non-zero, iterating steps

     2) and

     3) with increase in neighbourhood range until the check is true;

5) for a data accumulator pixel equivalently located to the count accumulator pixel referred to in step

     2), implementing a data accumulator summation over pixel values within a neighbourhood range equal to that for which the check indicated a significantly non-zero summation;

6) obtaining a division result by dividing the data accumulator summation by the significantly non-zero count accumulator summation; and

7) adding the division result to a pixel in a composite image located correspondingly to the count accumulator pixel referred to in step

     2).

16. The method of claim 10, wherein a computer-readable medium embodying program code instructions for execution by computer processing apparatus is used to perform steps b) e) wherein the instructions further relate to processing image data obtained using a sensor having a field of view through which an object moves.

17. The method according to claim 16, wherein the sensor is a first sensor and the method incorporates use of a plurality of sensors of which the first sensor is one, and the sensors include a second sensor obtaining data to generate the estimate of object image motion, and the method includes rescaling or interpolating this estimate before associating it with a respective image.

18. The method according to claim 16 including the step of measuring object speed using a speed sensor and deriving from this measurement a respective estimate of object image motion associated with each of the series of images.

19. The method according to claim 16, wherein the sensor is a framing sensor with a substantially rectangular field of view and step b) includes the additional step of writing at least one co-located non-central portion of data contained within each of the series of images to respective further image data accumulating means and step c) includes forming a plurality of composite images from data contained in respective image data accumulator arrays.