Calculating time to go and size of an object based on scale correlation between images from an electro optical sensor

• US 9,208,690 B2
• Filed: 05/28/2014
• Issued: 12/08/2015
• Est. Priority Date: 03/18/2009
• Status: Active Grant

• Current Assignee: Saab AB
• Original Assignee: Saab AB
• Inventors: Jonsson, Jimmy
• Primary Examiner(s): Akhavannik, Hadi
• Application Number:
  US14/289,201
• Publication Number:
  US 20140278048A1
• Time in Patent Office: 559 Days
• Field of Search: 382/100, 382/103, 382106-107, 382/121, 382/165, 382/181, 382/211, 348/152, 348154-155, 348/169, 348170-172, 348/208.1, 348/208.2, 34820814-20816, 348352-353, 348/465, 348/558, 715/250, 715/862
• US Class Current: 1/1
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G08G 5/0021 located in the aircraft
• CPC Class Codes: G08G 5/045 Navigation or guidance aids, e.g. determination of anti-collision manoeuvers radar or analogous systems specially adapted for anti-collision between aircraft G01S13/933

Abstract

A method for calculating a Time To Go value between a vehicle and an intruding object. First and second images of the object are retrieved at first and second points of time. A scaling factor is selected and the second image scaled with the scale factor. The scaled image is resampled to a set position. Correlation values are calculated between the first image and the resampled scaled image for different horizontal displacements and/or different vertical displacements for the resampled scaled image. A maximum correlation value is found based on the correlation values. Selecting the scaling factor, resampling, calculating the correlation values, and finding the maximum correlation values are repeated a second predetermined number of times. A largest maximum correlation value is found among the maximum correlation values and the scale factor associated with the largest maximum correlation value. The Time To Go is calculated based on the associated scale factor.

Claims

1. A method for calculating a Time To Go, TTG, value between a vehicle and an intruding object, said method comprising:

retrieving a first image of said intruding object at a first point of time, T₀, and a second image of said intruding object at a second point of time, T₁;

selecting a scaling factor S, and scaling said second image with said scale factor, S_i;

resampling said scaled image to a set position X_FRAC, Y_FRAC resulting in a resampled scaled image, RS_iI;

calculating correlation values, C_{XPIX, YPIX, i}, between said first image and said resampled scaled image, RS_iI, for different horizontal displacements X_PIX and/or different vertical displacements Y_PIX for said resampled scaled image RS_iI;

finding a maximum correlation value C_i based on said correlation values, C_{XPIX, YPIX i};

repeating the selecting the scaling factor, wherein i=i+1, the resampling, calculating the correlation values, and finding the maximum correlation values a second predetermined number of times;

finding a largest maximum correlation value, C_MAX, among said maximum correlation values, C_i, and the scale factor S_{i, MAX} associated with the largest maximum correlation value C_MAX; and

calculating the Time To Go, TTG, based on said scale factor S_{i, MAX} associated with the largest maximum correlation value C_MAX.

2. The method for calculating a Time To Go, TTG, value according to claim 1, further comprising:

a first iteration performed for each selected scaling factor S, for a predetermined number of different set positions X_FRAC;

Y_FRAC; and

repeating the resampling, the correlation value calculating, and finding the maximum correlation a first predetermined number of times corresponding to the predetermined number of different set positions.

3. The method according to claim 1, wherein each set position X_FRAC, Y_FRAC denotes at least one of a horizontal displacement at sub pixel level or a vertical displacement at sub pixel level.

4. The method according to claim 3, further comprising:

initially setting the X fractional pixel position, X_FRAC, to zero and the Y fractional pixel position, Y_FRAC, to zero.

5. The method according to claim 1, further comprising:

filtering said first image and said second image so that said first image and said second image become independent of absolute signal energy and/or so that edges become enhanced.

6. The method according to claim 1, wherein the Time To Go, TTG is calculated as inversely proportional to the scale factor S_{i, MAX}.

7. The method according to claim 1, wherein the Time To Go, TTG is calculated using the formula:

TTG=S_{i, MAX}*(T₁−

T₀)/(1−

S_{i, MAX}).

8. The method according claim 1, further comprising:

prior to said retrieving, detecting said object in a first sequence of images from a camera creating a second sequence of images in which images are associated with information about detected objects, wherein said retrieving comprises retrieving said first and said second images from the second sequence of images.

9. The method according to claim 1, further comprising:

prior to said retrieving, detecting said object in a first sequence of images from a camera creating a second sequence of images in which each images are associated with information about detected objects; and

after said detecting, tracking said object in said second sequence of images, creating a third sequence of images in which said object is centralized in the images, wherein said retrieving comprises retrieving said first image and said second image from the third sequence of images.

10. The method according to claim 8, further comprising:

prior to said calculating the Time To Go, TTG, estimating at least one initial size, σ

_Xin, σ

_Yin, of said object in an image n in said second sequence of images between T₀ and T₁.

11. The method according to claim 10, further comprising:

estimating at least one initial amplitude, A_in, after estimating an initial size, of a Gauss function by calculating a difference between a mean of a background and a mean of said object in said image n in the third sequence of images between T₀ and T₁.

12. The method according to claim 11, further comprising:

calculating at least one size, σ

_Xn, σ

_Yn, after said estimating an initial amplitude, by determining a Gaussian function, G_n, so that an error between said Gaussian function, G_n, and said object in said image n in the third sequence of images between T₀ and T₁ is minimized, wherein said initial amplitude A_in and said initial size, σ

_Xin, σ

_Yin are used as start parameters for said Gaussian function.

13. The method according to claim 12, wherein calculating a size further comprises filtering several sizes, σ

_Xn, σ

_Yn, from several images and thereby achieving a filtered size, σ

_XF, σ

_YF.

14. The method according to claim 12, further comprising:

updating said size σ

_Xn+1, σ

_Yn+1 for consecutive images n+1 in said third sequence based on said size σ

_Xn, σ

_Yn in said image n by using formulas;

=σ

_Xn+1=σ

_Xn(TTG_n+1+1/f)/TTG_n+1

σ

_Yn+1=σ

_Yn(TTG_n+1+1/f)/TTG_n+1,

where f is an image frequency between consecutive images.

15. The method according to claim 14, wherein updating the size said filtered size comprises utilizing, σ

XF, σ

YF, as start value.

16. A computer program product for use in a vehicle for calculating a Time To Go, TTG, between said vehicle and an intruding object, the computer program product comprising:

a non-transitory computer readable medium; and

computer readable code recorded on the non-transitory computer readable medium, which, when run in a processor of the vehicle causes the processor to perform a method comprising retrieving a first image of said intruding object at a first point of time, T₀, and a second image of said intruding object at a second point of time, T₁,

selecting a scaling factor S, and scaling said second image with said scale factor, S_i,

resampling said scaled image to a set position X_FRAC Y_FRAC resulting in a resampled scaled image, RS_iI;

calculating correlation values, C_{XPIX, YPIX, i}, between said first image and said resampled scaled image, RS_iI, for different horizontal displacements X_PIX and/or different vertical displacements X_PIX for said resampled scaled image RS_iI,

finding a maximum correlation value C_i based on said correlation values, C_{XPIX, YPIX i},

repeating selecting the scaling factor, wherein i=i+1, the resampling, calculating the correlation values, and finding the maximum correlation a second predetermined number of times,

finding a largest maximum correlation value, C_MAX, among said maximum correlation values, C_i, and the scale factor S_{i, MAX} associated with the largest maximum correlation value C_MAX, and

calculating the Time To Go, TTG, based on said scale factor S_{i, MAX} associated with the largest maximum correlation value C_MAX.

17. The computer program product according to claim 16, wherein the computer readable code further causes the processor to perform a first iteration for each selected scaling factor S_i for a predetermined number of different set positions X_FRAC, Y_FRAC, and repeat the resampling, the correlation value calculating, and finding the maximum correlation a first predetermined number of times corresponding to the predetermined number of different set positions.

18. The computer program product according to claim 16, wherein each set position X_FRAC, Y_FRAC denotes a horizontal displacement at sub pixel level and/or a vertical displacement at sub pixel level.

19. The computer program product according to claim 18, wherein the computer readable code further causes the processor to initially set the X fractional pixel position, X_FRAC, to zero and the Y fractional pixel position, Y_FRAC, to zero.

20. The computer program product according to claim 16, wherein the computer readable code further causes the processor to filter said first image and said second image so that said first image and said second image become independent of absolute signal energy and/or so that edges become enhanced.

21. The computer program product according to claim 16, wherein the Time To Go, TTG is calculated as inversely proportional to the scale factor S_i,MAX.

22. The computer program product according to claim 16, wherein the Time To Go, TTG is calculated using the formula:

TTG=S_{i, MAX}*(T₁−

T₀)/(1−

S_{i, MAX}).

23. The computer program product according to claim 16, wherein said computer readable code further causes the processor to perform prior to said retrieving

detecting said object in a first sequence of images from a camera creating a second sequence of images in which images are associated with information about detected objects, wherein said retrieving further comprises retrieving said first and said second images from the second sequence of images.

24. A system for calculating a Time To Go, TTG, value between a vehicle and an intruding object, said system comprising:

a memory module comprising a computer program product comprising

computer readable code recorded on the memory module, which, when run in a processor of the vehicle causes the processor to perform a method comprising

retrieving a first image of said intruding object at a first point of time, T₀, and a second image of said intruding object at a second point of time, T₁,

selecting a scaling factor S, and scaling said second image with said scale factor, S_i,

resampling said scaled image to a set position X_FRAC, Y_FRAC resulting in a resampled scaled image, RS_iI;

calculating correlation values, C_{XPIX, YPIX, i}, between said first image and said resampled scaled image, RS_iI, for different horizontal displacements X_PIX and/or different vertical displacements Y_PIX for said resampled scaled image RS_iI,

finding a maximum correlation value C_i based on said correlation values, C_{XPIX, YPIX i},

repeating selecting the scaling factor, wherein i=i+1, the resampling, calculating the correlation values, and finding the maximum correlation a second predetermined number of times,

finding a largest maximum correlation value, C_MAX, among said maximum correlation values, C_i, and the scale factor S_{i, MAX} associated with the largest maximum correlation value C_MAX, and

calculating the Time To Go, TTG, based on said scale factor S_{i, MAX} associated with the largest maximum correlation value C_MAX; and

a processor configured to run said computer program product.