Vision based navigation and guidance system

• US 7,881,497 B2
• Filed: 03/08/2007
• Issued: 02/01/2011
• Est. Priority Date: 03/08/2007
• Status: Expired due to Fees

• Current Assignee: Honeywell International Inc.
• Original Assignee: Honeywell International Inc.
• Inventors: Krishnaswamy, Kailash, Ganguli, Subhabrata
• Primary Examiner(s): Lu; Tom Y
• Application Number:
  US11/683,805
• Publication Number:
  US 20080219508A1
• Time in Patent Office: 1,426 Days
• Field of Search: 382/104, 382/173, 382/181, 382/266
• US Class Current: 382/104
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0257 using a radar radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931
• CPC Class Codes: G05D 1/027 comprising intertial navigation means, e.g. azimuth detector inertial navigation G01C21/16; inertial navigation combined with non-inertial navigation instruments G01C21/165
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G05D 1/0278 using satellite positioning signals, e.g. GPS
• CPC Class Codes: G06V 10/443 by matching or filtering
• CPC Class Codes: G06V 20/38 Outdoor scenes
• CPC Class Codes: G06V 20/39 Urban scenes

Abstract

A method of controlling an autonomous vehicle with a vision-based navigation and guidance system. The method includes capturing image frames at a predetermined frame rate. Detecting image points that define an edge in each image frame. Using image points in each image frame to locate a determined edge. Dynamically predicting the location of image points that define an edge in a subsequent image frame. Using the prediction of the location of image points that define an edge to narrow the search area for the edge in the subsequent image frame. Determining offset distances between determined edges in sequential image frames and controlling the vehicle based at least in part on determined offset distances between determined edges in sequential image frames.

Claims

1. A method of controlling an autonomous vehicle, the method comprising:

capturing image frames at a predetermined frame rate;

detecting image points that define an edge in each image frame, wherein a curvature of the edge changes between at least two image frames;

fusing image points in each image frame to locate a determined edge;

dynamically predicting the location of image points that define an edge in a subsequent image frame;

using the prediction of the location of image points that define an edge to narrow the search area for the edge in the subsequent image frame;

determining offset distances between determined edges in sequential image frames; and

controlling the vehicle based at least in part on determined offset distances between determined edges in sequential image frames.

2. The method of claim 1, further comprising:

blurring edges that are not generally in the direction of travel of the vehicle in each image frame.

3. The method of claim 1, further comprising:

removing shadows in each image frame.

4. The method of claim 1, wherein detecting image points that define an edge further comprises:

using a Canny edge detection algorithm.

5. The method of claim 1, wherein fusing image points further comprises:

using a Kalman filter on the image points that define an edge.

6. The method of claim 1, further comprising:

segmenting each image frame to reduce the area that needs to be searched for the image points that define an edge.

7. A method of determining an edge in image frames, the method comprising:

capturing an image frame;

directionally filtering the image frame to blur out edges not in a desired direction;

detecting image points that define an edge in each image frame, wherein a curvature of the edge changes between at least two image frames;

fusing image points to locate a determined edge; and

outputting the determined edge for use.

8. The method of claim 7, further comprising;

removing shadows in the image frame.

9. The method of claim 7, wherein detecting image points that define an edge further comprises:

using a Canny edge detection algorithm.

10. The method of claim 7, wherein fusing image points further comprises:

using a Kalman filter on the image points that define an edge.

11. The method of claim 7, further comprising:

dynamically predicting the location of image points that define a subsequent edge in a subsequent image frame based on the outputted determined edge; and

using the prediction of the location of image points that define the subsequent edge to narrow the search area for the subsequent edge in the subsequent image frame.

12. The method of claim 11, further comprising:

determining the subsequent edge in the subsequent image frame;

determining an offset distance between the determined edge in the image frame and the subsequent edge in the subsequent image frame; and

outputting the offset distance to a controller for use in navigation of a vehicle.

13. A vision-based navigation and guidance system for an autonomous vehicle, the system comprising:

a camera configured to capture image frames at a predetermined frequency;

an edge determination device configured to process spatial instantaneous data and process temporal data across image frames wherein a curvature of an edge in a first image frame differs from a curvature of an edge in a second image frame, the edge determination device further configured to output a Δ

X signal that is indicative of a distance from a determined edge based on the processed special and temporal data; and

a controller in communication with the edge determination device, the controller configured to control the vehicle based at least in part on the Δ

X signal.

14. The system of claim 13, wherein the edge determination device further comprises:

an image segmentation module that segments captured images to reduce the area in an image frame that has to be processed.

15. The system of claim 13, wherein the edge determination device further comprises:

a directional pre-filtering module that blurs out edges extending in select directions.

16. The system of claim 13, wherein the edge determination device further comprises:

a Canny edge detection module that detects image pixels in an image frame that form an edge.

17. The system of claim 13, wherein the edge determination device further comprises:

a shadow edge removal module that removes shadows from a captured image.

18. The system of claim 13, wherein the edge determination device further comprises:

a dynamic prediction window that predicts the location of an edge in a subsequent image frame.

19. The system of claim 13, wherein the edge determination device further comprises:

a measurement fusion module configured to find the determined edge based on detected edge points in an image frame.

20. The system of claim 13, further comprising:

a navigation device configured to provide navigation information to the controller that includes at least one of inertial movement unit information, Doppler radar information, global positioning system information, stereo vision information, radar information and mapping information.