Method and system for guiding a vehicle with vision-based adjustment

• US 7,876,927 B2
• Filed: 05/31/2007
• Issued: 01/25/2011
• Est. Priority Date: 01/04/2005
• Status: Active Grant

• Current Assignee: Deere & Company
• Original Assignee: Deere & Company
• Inventors: Han, Shufeng, Pickett, Terence Daniel, Reid, John Franklin
• Primary Examiner(s): Tabatabai; Abolfazl
• Application Number:
  US11/755,965
• Publication Number:
  US 20080065286A1
• Time in Patent Office: 1,335 Days
• Field of Search: 382/104, 340/944, 340/985, 340/988, 340/995.25, 340/998, 701/23, 701/28, 701/200, 701/211, 701/213, 701/301
• US Class Current: 382/104
• CPC Class Codes: A01B 69/007 Steering or guiding of agricultural vehicles, e.g. steering of the tractor to keep the plough in the furrow
• CPC Class Codes: G01S 19/485 whereby the further system is an optical system or imaging system
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0278 using satellite positioning signals, e.g. GPS
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

A preliminary guidance data is determined for the vehicle during an evaluation time window. A vision module collects vision data from a vision module during the evaluation time window. Vision guidance data is determined from the collected vision data. A vision quality estimator estimates vision quality data for at least one of the vision data and the vision guidance data during the evaluation time window. The vision quality data is based on a cross correlations and an r-squared value. An adjuster adjusts the preliminary guidance data to a revised guidance data based on the vision guidance data such that the revised guidance data is registered with or generally coextensive with the vision guidance data, if the vision quality data exceeds a minimum threshold.

Claims

1. A method for guiding a vehicle, the method comprising:

determining preliminary guidance data for the vehicle based a location-determining receiver associated with the vehicle during an evaluation time window;

collecting vision data from a vision module associated with the vehicle during the evaluation time window;

determining vision guidance data from the collected vision data;

estimating vision quality data for at least one of the vision data and the vision guidance data during the evaluation time window, the vision quality data based on a cross correlations and an r-squared value, the cross correlation being between an observed intensity profile of the vision data in a section and a reference intensity profile to determine row center positions and an r-squared value being between a regression path and the row center positions; and

adjusting the preliminary guidance data to a revised guidance data based on the vision guidance data such that the revised guidance data is registered with or generally coextensive with the vision guidance data, if the vision quality data exceeds a minimum threshold.

2. The method according to claim 1 wherein the cross correlation comprises a normalized cross correlation based on a mean cross correlation for a group of sections of the vision data, each section associated with a corresponding row center position.

3. The method according to claim 2 wherein the estimating of the vision quality data comprises:

estimating the row center positions associated with corresponding sections or subdivisions of the vision data; and

determining the regression path coinciding with the row center positions.

4. The method according to claim 2 wherein the estimating of the vision quality data comprises:

organizing a collected image or the vision data into crop pixels and non-crop pixels;

determining an intensity profile based on pixel intensity values of the crop pixels;

establishing a reference intensity profile associated with a group of crop rows or a single crop row;

subdividing the collected image into multiple sections or overlapping windows;

determining a cross correlation between the intensity profile and a reference intensity profile for each one of the sections.

5. The method according to claim 4 further comprising:

estimating a row center position associated with each one of the sections; and

arranging estimated row center positions into a regression path defining a representation of a position and orientation of a crop row.

6. The method according to claim 5 further comprising:

determining an r-squared value between the regression path and the row center positions; and

determining vision quality data based on the determined cross correlations and the r-squared value for application to a vehicle controller or adjuster for steering of a vehicle.

7. The method according to claim 1 wherein the preliminary guidance data and the revised guidance data, each comprise at least one of heading data, off-track data, and curvature data.

8. A method for estimating a vision quality data for guidance of a vehicle, the method comprising:

organizing a collected image of image data into crop pixels and non-crop pixels;

determining an intensity profile based on pixel intensity values of the crop pixels;

establishing a reference intensity profile associated with a group of crop rows or a single crop row;

subdividing the collected image into multiple sections or overlapping windows;

determining a cross correlation between the intensity profile and a reference intensity profile for each one of the sections;

estimating a row center position associated with each one of the sections;

arranging estimated row center positions into a regression path defining a representation of a position and orientation of a crop row;

determining an r-squared value between the regression path and the row center positions; and

determining vision quality data based on the determined cross correlations and the r-squared value for application to a vehicle controller or adjuster for steering of a vehicle.

9. The method according to claim 8 wherein the regression path comprises a path selected from the group consisting of a generally linear path, generally curved path, and a contour path.

10. The method according to claim 8 wherein the regression path comprises a generally linear path and wherein the r-squared value indicates a degree of linearity of the row center positions with respect to the regression path.

11. The method according to claim 8 wherein the reference intensity profile comprises a sinusoidal reference intensity profile.

12. The method according to claim 11 wherein peak cross correlations correspond to the row center positions of corresponding crop row sections.

13. The method according to claim 8 wherein the crop row comprises a windrow.

14. The method according to claim 8 wherein the reference intensity profile comprises a Fermi function or a step function.

15. The method according to claim 14 wherein the width of Fermi function is proportional to a width of a windrow.

16. The method according to claim 8 wherein the vision quality data is regarded as high or adequate for guidance of the vehicle if the cross correlation meets or exceeds a first minimum threshold and if the r-squared value meets or exceeds a second minimum threshold.

17. The method according to claim 8 wherein the vision quality data is regarded as low or deficient for guidance of the vehicle if the cross correlation is less than the first minimum correlation threshold or if the r-squared value is less than the second minimum threshold.

18. The method according to claim 8 wherein the cross correlation comprises a general image cross correlation determined in accordance with the following equation:

19. The method according to claim 8 wherein the cross correlation comprises a normalized cross correlation (r_cross—

_norm) determined in accordance with the following equation;

20. The method according to claim 8 further comprising:

collecting the image data via a monocular vision imaging unit.

21. The method according to claim 8 wherein the sections comprise overlapping windows or subdivisions of the image data.

22. A system for guiding a vehicle, the system comprising:

a location-determining receiver for collecting preliminary guidance data for the vehicle during an evaluation time window;

a vision module for collecting vision data during the evaluation time window and for determining vision guidance data from the collected vision data;

a vision quality estimator for estimating vision quality data for at least one of the vision data and the vision guidance data during the evaluation time window, the vision quality data based on a cross correlations and an r-squared value, the cross correlation being between an observed intensity profile of the image data in a section and a reference intensity profile to determine row center positions and an r-squared value being between the regression path and the row center positions; and

an adjuster for adjusting the preliminary guidance data to a revised guidance data based on the vision guidance data such that the revised guidance data is registered with or generally coextensive with the vision guidance data, if the vision quality data exceeds a minimum threshold.

23. The system according to claim 22 further comprising a cross correlation module for determining a normalized cross correlation as the cross correlation, the normalized cross correlation based on a mean cross correlation for a group of sections of the image data, each section associated with a corresponding row center position.

24. The system according to claim 23 wherein the vision quality estimator further comprises:

a row center estimator for estimating the row center positions associated with corresponding sections or subdivisions of the image data; and

an arranger for determining the regression path coinciding with the row center positions.

25. The system according to claim 23 wherein the vision quality estimator comprises:

an organizer for organizing a collected image into crop pixels and non-crop pixels;

an intensity profile module for determining an intensity profile based on pixel intensity values of the crop pixels;

a data storage device for storing a reference intensity profile associated with a group of crop rows or a single crop row;

a subdivider for subdividing the collected image into multiple sections or overlapping windows;

a cross-correlation module for determining a cross correlation between the intensity profile and a reference intensity profile for each one of the sections.

26. The system according to claim 25 further comprising:

a row center estimator for estimating a row center position associated with each one of the sections; and

an arranger for arranging estimated row center positions into a regression path defining a representation of a position and orientation of a crop row.

27. The system according to claim 26 further comprising:

a variation evaluator for determining an r-squared value between the regression path and the row center positions; and

an estimator for determining vision quality data based on the determined cross correlations and the r-squared value for application to a vehicle controller or adjuster for steering of a vehicle.

28. The system according to claim 22 wherein the preliminary guidance data and the revised guidance data, each comprise at least one of heading data, off-track data, and curvature data.