CALCULATING TIME TO GO AND SIZE OF AN OBJECT BASED ON SCALE CORRELATION BETWEEN IMAGES FROM AN ELECTRO OPTICAL SENSOR

• US 20120099764A1
• Filed: 03/18/2009
• Published: 04/26/2012
• Est. Priority Date: 03/18/2009
• Status: Active Grant

• Current Assignee: Saab AB
• Original Assignee: Saab AB
• Inventors: Jonsson, Jimmy
• Granted Patent: US 8,774,457 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 3/4069 by subpixel displacements
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G08G 5/0021 located in the aircraft
• CPC Class Codes: G08G 5/045 Navigation or guidance aids, e.g. determination of anti-collision manoeuvers radar or analogous systems specially adapted for anti-collision between aircraft G01S13/933

Abstract

A method and a system for calculating a time to go value between a vehicle and an intruding object. A first image of the intruding object at a first point of time retrieved. A second image of the intruding object at a second point of time is retrieved. The first image and the second image are filtered so that the first image and the second image become independent of absolute signal energy and so that edges become enhanced. An X fractional pixel position and a Y fractional pixel position are set to zero. The X fractional pixel position denotes a horizontal displacement at sub pixel level and the Y fractional pixel position denotes a vertical displacement at sub pixel level. A scale factor is selected. The second image is scaled with the scale factor and resampled to the X fractional pixel position and the Y fractional pixel position, which results in a resampled scaled image. Correlation values, are calculated between the first image and the resampled scaled image for different horizontal displacements at pixel level and different vertical displacements at pixel level for the resampled scaled image. A maximum correlation value at a subpixel level is found based on the correlation values. The X fractional pixel position and the Y fractional pixel position are also updated. j is set to j=j+1 and scaling of the second image, calculation of correlation values, finding the maximum correlation value and setting of j to j=j+1 are repeated a predetermined number of times. i is set to i=i+1 and selecting the scale factor, scaling of the second image, calculation of correlation values, finding the maximum correlation value, setting of j to j=j+1, and setting of i to i=i+1 are repeated a predetermined number of times. A largest maximum correlation value is found among the maximum correlation values and the scale factor associated with the largest maximum correlation value. The time to go is calculated based on the scale factor.

Claims

1. A method for calculating a Time To Go, TTG, value between a vehicle and an intruding object, said method comprising:

retrieving a first image of said intruding object at a first point of time, T₀, and a second image of said intruding object at a second point of time, T₁;

filtering said first image and said second image so that said first image and said second image become independent of absolute signal energy and so that edges become enhanced;

setting an X fractional pixel position, X_FRAC, to zero and an Y fractional pixel position, Y_FRAC, to zero, where X_FRAC denotes a horizontal displacement at sub pixel level and Y_FRAC a vertical displacement at sub pixel level;

selecting a scale factor, S_i;

scaling said second image with said scale factor, S_i, and resampling said scaled image to position X_FRAC and Y_FRAC;

resulting in a resampled scaled image, RS_iI;

calculating correlation values, C_{XPIX, YPIX, i}, between said first image and said resampled scaled image, RS_iI, for different horizontal displacements at pixel level, X_PIX, and different vertical displacements at pixel level, Y_PIX, for said resampled scaled image RS_iI;

finding a maximum correlation value at subpixel level, C_i, based on said correlation values, C_{XPIX, YPIX i}, and updating X_FRAC and Y_FRAC;

setting j=j+1 and scaling of the second image, calculation of correlation values, finding the maximum correlation value and setting of j to j=j+1 are repeated a first predetermined number of times;

setting i=i+1 and selecting the scale factor, scaling of the second image, calculation of correlation values, finding the maximum correlation value, setting of i to j=j+1, and setting of i to i=i+1 are repeated a second predetermined number of times;

finding a largest maximum correlation value, C_MAX, among said maximum correlation values, C_i, and the scale factor S_{i, MAX} associated with the largest maximum correlation value C_MAX; and

calculating the Time To Go, TTG, based on said scale factor S_{i, MAX}.

2. The method according to claim 1, wherein the Time To Go, TTG is calculated as inversely proportional to the scale factor S_{i, MAX}.

3. The method according to claim 1, wherein the Time To Go, TTG is calculated using the formula:

TTG=S_{i, MAX}*(T₁−

T₀)/(1−

S_{i, MAX}).

4. The method according to claim 1, further comprising, prior to retrieving the first image and the second image,

detecting said object in a first sequence of images from a camera creating a second sequence of images in which images are associated with information about detected objects, and wherein said retrieving the first image and the second image comprises retrieving said first and said second images from the second sequence of images.

5. The method according to claim 1, further comprising, prior to retrieving the first image and the second image,

detecting said object in a first sequence of images from a camera creating a second sequence of images in which each images are associated with information about detected objects, and after said step of detecting, tracking said object in said second sequence of images, creating a third sequence of images in which said object is centralized in the images, and wherein said retrieving the first image and the second image comprises retrieving said first and said second images from the third sequence of images.

6. The method according to claim 4, wherein said method further comprises, prior to said calculating the Time To Go, TTG,

estimating at least one initial size, σ

_Xin, σ

_Yin, of said object in an image n in said second sequence of images between T₀ and T₁.

7. The method according to claim 6, wherein said method further comprises

estimating at least one initial amplitude, A_in, after said estimating an initial size, of a Gauss function by calculating a difference between a mean of a background and a mean of said object in said image n in the third sequence of images between T₀ and T₁.

8. The method according to claim 7, wherein said method further comprises

calculating at least one size, σ

_Xn, σ

_Yn, after said step of estimating an initial amplitude, by determining a Gaussian function, G_n, so that an error between said Gaussian function, G_n, and said object in said image n in the third sequence of images between T₀ and T₁ is minimized, where said initial amplitude A_in and said initial size, σ

_Xin, σ

_Yin are used as start parameters for said Gaussian function.

9. The method according to claim 8, wherein said method of calculating a size further comprises filtering several sizes, σ

_Xn, σ

_Yn, from several images and thereby achieving a filtered size, σ

_XF, σ

_YF.

10. The method according to claim 8, wherein said method further comprises

updating said size σ

_Xn+1, G_Yn+1 for consecutive images n+1 in said third sequence based on said size σ

_Xn, σ

_Yn in said image n by using formulas;

σ

_Xn+1=σ

_Xn(TTG_n+1+1/f)/TTG_n+1

σ

_Yn+1=σ

_Yn(TTG_n+1+1/f)/TTG_n+1

where f is an image frequency between consecutive images.

11. The method according to claim 10, wherein in said updating the size said filtered size, σ

XF, σ

YF, are used as start value.

12. A computer program product for use in a vehicle for calculating a Time To Go, TTG, between said vehicle and an intruding object, the computer program product comprising

a non-transitory computer readable medium, having thereon:

computer readable code which, when run in a processor of the vehicle causes the processing means to perform;

retrieving a first image of said intruding object at a first point of time, T₀, and a second image of said intruding object at a second point of time, T₁;

filtering said first image and said second image so that said first image and said second image become independent of absolute signal energy and so that edges become enhanced;

setting an X fractional pixel position, X_FRAC, to zero and an Y fractional pixel position, Y_FRAC, to zero, where X_FRAC denotes a horizontal displacement at sub pixel level and Y_FRAC a vertical displacement at sub pixel level;

selecting a scale factor, S_i;

scaling said second image with said scale factor, S_i, and resampling said scaled image to position X_FRAC and Y_FRAC;

resulting in a resampled scaled image, RS_iI,;

calculating correlation values, C_{XPIX, YPIX, i}, between said first image and said resampled scaled image, RS_iI, for different horizontal displacements at pixel level, X_PIX, and different vertical displacements at pixel level, Y_PIX, for said resampled scaled image RS_iI;

finding a maximum correlation value at subpixel level, C_i, based on said correlation values, C_{XPIX, YPIX i}, and updating X_FRAC and Y_FRAC;

setting j=j+1 and scaling of the second image, calculation of correlation values, finding the maximum correlation value and setting of j to j=j+1 are repeated a first predetermined number of times;

setting i=i+1 and selecting the scale factor, scaling of the second images calculation of correlation values, finding the maximum correlation value, setting of j to j=j+1, and setting of i to i=i+1 are repeated a second predetermined number of times;

finding a largest maximum correlation value, C_MAX, among said maximum correlation values, C_i, and the scale factor S_{i, MAX} associated with the largest maximum correlation value C_MAX; and

calculating the Time To Go, TTG, based on said scale factor S_{i, MAX}.

13. The computer program product according to claim 12, wherein the Time To Go, TTG is calculated as inversely proportional to the scale factor S_i,MAX.

14. The computer program product according to claim 12, wherein the Time To Go, TTG is calculated using the formula:

TTG=S_{i, MAX}*(T₁−

T₀)/(1−

S_{i, MAX}).

15. The computer program product according to claim 12, wherein said computer readable code when run in said processor further causes the processor to perform;

prior to said

retrieving the first image and the second image, detecting said object in a first sequence of images from a camera creating a second sequence of images in which images are associated with information about detected objects, wherein said retrieving further comprises retrieving said first and said second images from the second sequence of images.

16. The computer program product according to claim 12, wherein said computer readable code when run in said processor further causes the processor to perform

detecting said object in a first sequence of images from a camera creating a second sequence of images in which images are associated with information about detected objects;

and after said detecting, and

tracking said object in said second sequence of images, creating a third sequence of images in which said object is centralized in the images,

wherein said retrieving further comprises retrieving said first and said second images from the third sequence of images.

17. The computer program product according to claim 15, wherein said computer readable code when run in said processor further causes the processor to perform, prior to said

calculating the Time To Go, TTG;

estimating at least one initial size, σ

_Xin, σ

_Yin, of said object in an image n in said second sequence of images between T₀ and T₁.

18. The computer program product according to claim 16, wherein said computer readable code when run in said processor further causes the processor to perform, after said estimating an initial size;

estimating at least one initial amplitude, A_in, of a Gauss function by calculating a difference between a mean of a background and a mean of said object in said image n in the third sequence of images between T₀ and T₁.

19. The computer program product according to claim 18, wherein said computer readable code when run in said processor further causes the processor to perform, after said estimating an initial amplitude;

calculating at least one size, σ

_Xn, σ

_Yn, by determining a Gaussian function, G_n, so that an error between said Gaussian function, G_n, and said object in said image n in the third sequence of images between T₀ and T₁ is minimized, wherein said initial amplitude A_in and said initial size, σ

_Xin, σ

_Yin are used as start parameters for said Gaussian function.

20. The computer program product according to claim 19, wherein said computer readable code when run in said processor further causes the processor to perform in said calculating at least one size filtering of several sizes, σ

_Xn, σ

_Yn, from several images and thereby achieving a filtered size, σ

_XF, σ

_YF.

21. The computer program product according to claim 18, wherein said computer readable code when run in said processor further causes the processor to perform;

updating said size σ

_Xn+1, σ

_Yn+1 for consecutive images n+1 in said third sequence based on said size σ

_Xn, σ

_Yn in said image n by using formulas;

σ

_Xn+1=σ

X_n(TTG_n+1+1/f)/TTG_n+1

σ

_Yn+1=σ

_Yn(TTG_n+1+1/f)/TTG_n+1

where f is the image frequency.

22. The computer program product according to claim 21, wherein said computer readable code when run in said processor further causes the processor to use said filtered size, σ

_XF, σ

_YF as start values in said updating the size.

23. A system for calculating a Time To Go, TTG, value between a vehicle and an intruding object, said system comprises comprising:

a memory module comprising a computer program product according to claim 12; and

a processor configured for running said computer program product.