System and method for estimating lateral position

• US 5,675,489 A
• Filed: 07/06/1995
• Issued: 10/07/1997
• Est. Priority Date: 07/06/1995
• Status: Expired due to Fees

• Current Assignee: AssistWare Technology, Inc. (Cognex Corporation)
• Original Assignee: Carnegie Mellon University
• Inventors: Pomerleau, Dean A.
• Primary Examiner(s): Teska, Kevin J.
• Assistant Examiner(s): WALDER, STEPHEN JOSEPH
• Application Number:
  US08/498,593
• Time in Patent Office: 824 Days
• Field of Search: 364/424.033, 364/424.027, 364/424.028, 364/424.029, 364/424.031, 364/424.032, 364/426.041, 364/426.044, 364/426.046, 364/461, 364/559, 364/578, 382/107, 382/113, 382/123, 382/154, 382/153, 382/190, 382/193, 382/194, 382/203, 382/316
• US Class Current: 701/28
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

A system for estimating lateral offset of a vehicle from a centered position on a road is comprised of a video camera and a digitizer to produce digitized input signals representative of the road ahead of the vehicle. A circuit is provided for sampling the digitized input signal to create a perspective-free image comprised of rows and columns of pixels. The curvature of the road is determined by ascertaining an amount by which each row of pixels of the perspective-free image must be horizontally shifted to cause features to appear in the perspective-free image as a straight line. Each column of pixels of the perspective-free image is added to produce a single row, referred to as a current scanline profile, representative of the vehicle'"'"'s position on the road. The current scanline profile is compared with a template profile representative of the vehicle'"'"'s position when properly centered on a straight road to determine an amount by which the current scanline profile must be shifted to arrive at an optimal correlation with the template profile. The amount that the current scanline profile must be shifted is representative of the lateral offset of the vehicle from the center of the road. Circuitry is responsive to the lateral offset and the curvature of the road to produce an appropriate output signal.

Claims

1. A system for estimating lateral offset of a vehicle from a known position, comprising:

input means producing digitized input signals representative of the road ahead of the vehicle;

means for sampling said digitized input signals to create a perspective-free image, said perspective-free image taking the form of a matrix comprised of rows and columns;

means for determining the curvature of the road by determining an amount each row of said perspective-free image must be shifted to cause features to appear in said perspective-free image as straight lines, said means for determining including means for adding each column of said shifted perspective-free image to generate a current scanline profile representative of the vehicle'"'"'s position on the road;

means for storing at least one template profile representative of a known vehicle position; and

means for comparing said current scanline profile with said template profile to determine an amount by which said current scanline profile must be shifted to arrive at the best correlation with said template profile, said amount being representative of the lateral offset of the vehicle from the known vehicle position.

2. The system of claim 1 wherein said means for determining the curvature of the road includes:

means for generating a range of radii of curvature corresponding to hypothetical curvatures;

means for shifting said perspective-free image according to each of said radii, said means for adding each column of said shifted perspective-free images producing a row of values for each shifted perspective-free image; and

means for selecting one of said rows of values for use as the current scanline profile.

3. The system of claim 2 wherein said means for selecting selects the shifted perspective-free image having the greatest number of vertically aligned features.

4. The system of claim 3 wherein said means for selecting selects the shifted perspective-free image having the greatest number of vertically aligned features by calculating the degree of abruptness of horizontal intensity value changes for each row of values and selecting the one of said rows that has the most abrupt variations.

5. The system of claim 4 wherein said means for selecting calculates the degree of abruptness by summing the differences in values between neighboring values in each of said rows of values to create a score for each row of values.

6. The system of claim 5 wherein said means for selecting selects the row of values having the highest score.

7. The system of claim 2 wherein said means for generating a range of radii of curvature generates additional radii of curvature if said selected current scanline profile results from a radius of curvature that is on either extreme of said range.

8. The system of claim 2 wherein said means for generating a range of radii of curvature generates said range based on a radius of curvature corresponding to a previous current scanline profile.

9. The system of claim 8 wherein said means for generating a range of radii of curvature generates a plurality of radii greater than and a plurality of radii less than the radius of curvature corresponding to the previous current scanline profile.

10. The system of claim 2 wherein said means for generating a range of radii of curvature generates positive radii representative of a right-hand curve, negative radii representative of a left-hand curve, and an infinite radius representative of no curvature.

11. The system of claim 2 wherein said means for generating a range of radii of curvature is responsive to the speed of the vehicle.

12. The system of claim 2 wherein said means for shifting shifts by one of integer pixel values and fractional pixel values.

13. The system of claim 12 wherein said means for shifting shifts by fractional pixel values by linearly interpolating between integer pixel values using weighted averages of the values of said integer pixel values.

14. The system of claim 1 wherein said means for comparing said current scanline profile with said template profile includes:

means for creating a plurality of shifted copies of said current scanline profile;

means for calculating a correlation between each of said shifted copies and said template profile; and

means for selecting the one of said shifted copies with the highest correlation, the amount of shift associated therewith being representative of the lateral offset of the vehicle.

15. The system of claim 14 wherein said means for creating a plurality of shifted copies creates a range of copies wherein certain of said copies are shifted to the right and certain of said copies are shifted to the left of the current scanline profile.

16. The system of claim 15 wherein said means for creating a plurality of shifted copies creates additional shifted copies when said selected shifted copy is on either extreme of said range.

17. The system of claim 1 wherein said means for sampling identifies a window of interest within the image represented by said digitized input signals, said window of interest representing a perspective view of the road ahead.

18. The system of claim 17 wherein said means for sampling identifies a trapezoidally-shaped window of interest, the parallel sides of said trapezoidally-shaped window of interest being parallel to the top and bottom of said image with the top edge being shorter than the bottom edge.

19. The system of claim 18 wherein said means for sampling is responsive to the speed of said vehicle.

20. The system of claim 17 wherein said means for sampling segments said window of interest into said matrix of rows and columns, said matrix representing a perspective-free image translation of said window of interest, and wherein each row of said matrix represents a region of said window of interest of constant height and variable width.

21. The system of claim 1 additionally comprising means for evolving said template profile, said means for evolving including:

means for combining said current scanline profile and said template profile according to a predetermined function to form an updated template profile; and

means for replacing said template profile with said updated template profile.

22. The system of claim 21 wherein said predetermined function includes a combination of the weighted averages of said current scanline profile and said template profile.

23. The system of claim 21 wherein said means for comparing includes means for calculating a correlation between said current scanline profile and said template profile, said system additionally comprising means for comparing said correlation to an evolve threshold, and wherein said means for combining is responsive to said means for comparing.

24. The system of claim 23 wherein said means for replacing said template profile includes:

means for comparing said correlation to a reset threshold;

means for determining a second correlation between said current scanline profile and said updated template profile in response to said means for comparing;

means for comparing said second correlation to an update threshold; and

means for replacing said template profile with said updated template profile in response to said means for comparing to said update threshold.

25. The system of claim 1 additionally comprising means for creating a rapidly adapting scanline profile, said means for creating including:

means for selecting a predetermined number of rows from the top of said perspective-free image;

means for shifting said rows in response to said means for determining the curvature of the road;

means for adding each column of said shifted rows to generate a top scanline profile;

means for combining an instance of said rapidly adapting scanline profile and said top scanline profile according to a predetermined function to form a new instance of said rapidly adapting scanline profile; and

means for replacing said template profile with said rapidly adapting scanline profile.

26. The system of claim 1 wherein said means for storing stores a library of alternate template profiles, each of said alternate template profiles being representative of a known vehicle position, said system additionally comprising means for changing the template profile.

27. The system of claim 26 wherein said means for comparing includes means for calculating a correlation between said current scanline profile and said template profile, and wherein said means for changing said template profile includes:

means for comparing said correlation to a reset threshold;

means for determining additional correlations between said current scanline profile and said alternate template profiles in response to said means for comparing;

means for selecting the highest additional correlation;

means for comparing said highest additional correlation to an update threshold; and

means for replacing said template profile with said alternate template profile corresponding to said highest additional correlation in response to said means for comparing to said update threshold.

28. The system of claim 1 additionally comprising output means for producing vehicle control signals in response to said means for determining the curvature of the road and the lateral offset.

29. The system of claim 1 additionally comprising output means for producing warning signals in response to said means for determining the curvature of the road and the lateral offset.

30. A system for estimating lateral offset of a vehicle from a known position, comprising:

input means producing digitized input signals representative of the road ahead of the vehicle;

means for sampling said digitized input signals to create a perspective-free image, said perspective-free image taking the form of a matrix comprised of rows and columns;

means for determining the curvature of the road by determining an amount each row of said perspective-free image must be shifted to cause features to appear in said perspective-free image as straight lines, said means for determining including means for adding each column of said perspective-free image to generate a current scanline profile representative of the vehicle'"'"'s position on the road;

means for storing a plurality of template profiles representative of known vehicle positions;

means for selecting one of said stored template profiles; and

means for comparing said current scanline profile with said selected template profile to determine an amount by which said current scanline profile must be shifted to arrive at the best correlation with said selected template profile, said amount being representative of the lateral offset of the vehicle and the known vehicle position.

31. A method for estimating lateral offset of a vehicle from a known position, said method comprising the steps of:

producing digitized input signals representative of the road ahead of the vehicle;

sampling said digitized input signals to create a perspective-free image, said perspective-free image taking the form of a matrix comprised of rows and columns;

determining the curvature of the road by determining an amount each row of said perspective-free image must be shifted to cause features to appear in said perspective-free image as straight lines, said step of determining including the step of adding each column of said shifted perspective-free image to generate a current scanline profile representative of the vehicle'"'"'s position on the road;

storing a template profile representative of a known vehicle position; and

comparing said current scanline profile with said template profile to determine an amount by which said current scanline profile must be shifted to arrive at the best correlation with said template profile, said amount being representative of the lateral offset of the vehicle from the known vehicle position.