Vision-based highway overhead structure detection system

• US 7,327,855 B1
• Filed: 02/09/2006
• Issued: 02/05/2008
• Est. Priority Date: 06/20/2001
• Status: Expired due to Fees

• Current Assignee: HRL Laboratories LLC (The Boeing Co.)
• Original Assignee: HRL Laboratories LLC (The Boeing Co.)
• Inventors: Chen, Yang
• Primary Examiner(s): Bella; Matthew C.
• Assistant Examiner(s): Lu; Tom Y
• Application Number:
  US11/350,570
• Time in Patent Office: 726 Days
• Field of Search: 382/103, 382/104, 382/107, 701/96
• US Class Current: 382/104
• CPC Class Codes: B60T 2201/08 Lane monitoring; Lane Keeping Systems
• CPC Class Codes: B60T 2201/089 using optical detection
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors
• CPC Class Codes: G08G 1/164 Centralised systems, e.g. external to vehicles

Abstract

An image sequence is inputted from a camera and vertical motion is estimated. A windowed horizontal edge-projection is extracted from the inputted image sequence and corresponding horizontal edges are projected. The horizontal edge-projection and the vertical motion estimation are combined in a horizontal segmentation and tracking element, and are forwarded to an object parameter estimation element, where the object'"'"'s distance and height are estimated. This data is combined in a fusion with a radar detection element. By correctly matching the overhead objects sensed by the radar detection element and the video camera, the proximity and relative speed can be ascertained. Once overhead objects have been identified they can be isolated and not considered for collision avoidance purposes.

Claims

1. An identification method comprising acts of:

receiving an input image sequence;

isolating stationary objects from the input image sequence, wherein the act of isolating further includes acts of;

generating a horizontal edge projection from the input image sequence;

estimating a vertical motion from the input image sequence;

estimating an object parameter of the stationary objects from the horizontal edge projection and the vertical motion;

receiving a radar signal;

classifying objects within the radar image as moving or non-moving objects; and

comparing the non-moving objects from the radar image with the object parameter of the stationary objects, wherein non-moving objects outside of a given object parameter are classified as a non-threat, and non-moving objects within a given object parameter are classified as a threat.

2. The method of claim 1, wherein the act of estimating an object parameter further includes an act of estimating a distance to and a height of the stationary object.

3. The method of claim 2, wherein the act of isolating stationary objects further comprises an act of isolating overhead stationary objects.

4. The method of claim 2, the act of estimating a distance to and a height of the stationary object further comprises acts of:

establishing a horizontal edge projection track from the horizontal edge projection;

calculating a first coordinate of the horizontal edge projection track y_i utilizing a first equation

$y_i=\frac{\mathrm{fH}}{Z_i},$

where i=0, . . . , N−

1, H is the height of the stationary object above a predetermined point, Z_i is the distance of the stationary object to the predetermined point at frame i, and N is the number of points in the horizontal edge projection track;

calculating y_i using a second equation

y_i=Y₀−

r_i, where r_i is a location of the horizontal edge projection track for frame i, and Y₀ is a reference horizon;

calculating Z_i utilizing a third equation Z_i=D−

d_i, where D is a distance between the stationary object and the predetermined point when i=0 and d_i is the distance the predetermined point has traveled from i=0 to i=i;

substituting the second equation and the third equation into the first equation resulting in a fourth equation

(Y₀−

r_i)D−

fH=(Y₀−

r_i)d_i, where i=0, . . . , N−

1

rewriting the fourth equation as a matrix A, where the matrix A (N by

     2) and a column vector B (size N) are made up of the coefficients in the fourth equation,

$A<br><br>\left(\begin{array}{c}D\\ H\end{array}\right)=B$ $A={\left[\left(a_0\right),\left(a_1\right),\mathrm{\dots <br><br>}<br><br>,\left(a_{n-1}\right)\right]}^T<br><br>a_i={\left[Y_0-r_i-f\right]}^T={\left[y_i,f\right]}^T,i=0,\mathrm{\dots <br><br>}<br><br>,N-1$ $B={\left[b_0,b_1,\mathrm{\dots <br><br>}<br><br>,b_{-1}\right]}^T<br><br>b_{i<br><br>}=\left(Y_0-r_i\right)<br><br>d_i=y_i<br><br>d_i,i=0,\mathrm{\dots <br><br>}<br><br>,N-1$

solve matrix A for D and H using a least-squares method.

5. The method of claim 4, wherein the act of isolating stationary objects further comprises an act of isolating overhead stationary objects.

6. An identification system comprising:

a host vehicle, wherein the host vehicle comprises;

an image capture device, operatively interconnected with;

an image signal processor, operatively interconnected with;

a matching processor, operatively interconnected with;

a radar transceiver,

wherein the image signal processor further comprises;

a horizontal edge projection generator for generating a horizontal edge projection from an input image;

a vertical motion estimator for estimating a vertical motion from the input image;

an object parameter estimator for estimating an object parameter from the horizontal edge projection and the vertical motion,

wherein when the image capture device receives an input image sequence, the image signal processor isolates stationary objects from the input image sequence and provides an object parameter, the radar transceiver classifies objects within the radar image as moving or non-moving objects, and the matching processor compares the non-moving objects with the object parameter, wherein non-moving objects outside of a given parameter are classified as a non threat, and non-moving objects within a given object parameter are classified as a threat.

7. The system of claim 6, wherein the object parameter is distance and height of the stationary object.

8. The system of claim 6, wherein stationary objects are overhead stationary objects.

9. The system of claim 6, wherein the matching processor has computer further instructions for performing the operations of:

establishing a horizontal edge projection track from the horizontal edge projection;

calculating a first coordinate of the horizontal edge projection track y_i utilizing a first equation

$y_i=\frac{\mathrm{fH}}{Z_i},$

where i=0, . . . , N−

1, H is the height of the stationary object above a predetermined point, Z_i is the distance of the stationary object to the predetermined point at frame i, and N is the number of points in the horizontal edge projection track;

calculating y_i using a second equation

y_i=Y₀−

r_i, where r_i is a location of the horizontal edge projection track for frame i, and Y₀ is a reference horizon;

calculating Z_i utilizing a third equation

Z_i=D−

d_i, where D is a distance between the stationary object and the predetermined point when i=0 and d_i is the distance the predetermined point has traveled from i=0 to i=i;

substituting the second equation and the third equation into the first equation resulting in a fourth equation

(Y₀−

r_i)D−

fH=(Y₀−

r_i)d_i, where i=0, . . . , N−

1

rewriting the fourth equation as a matrix A, where the matrix A (N by

     2) and a column vector B (size N) are made up of the coefficients in the fourth equation,

$A<br><br>\left(\begin{array}{c}D\\ H\end{array}\right)=B$ $A={\left[\left(a_0\right),\left(a_1\right),\mathrm{\dots <br><br>}<br><br>,\left(a_{n-1}\right)\right]}^T<br><br>a_i={\left[Y_0-r_i-f\right]}^T={\left[y_i,f\right]}^T,i=0,\mathrm{\dots <br><br>}<br><br>,N-1$ $B={\left[b_0,b_1,\mathrm{\dots <br><br>}<br><br>,b_{-1}\right]}^T<br><br>b_{i<br><br>}=\left(Y_0-r_i\right)<br><br>d_i=y_i<br><br>d_i,i=0,\mathrm{\dots <br><br>}<br><br>,N-1$

solve matrix A for D and H using a least-squares method.

10. The method of claim 9, wherein the stationary objects are overhead stationary objects.