Vision-aided system and method for guiding a vehicle

• US 7,610,123 B2
• Filed: 04/15/2005
• Issued: 10/27/2009
• Est. Priority Date: 01/04/2005
• Status: Active Grant

• Current Assignee: Deere & Company
• Original Assignee: Deere & Company
• Inventors: Rovira-Mas, Francisco, Reid, John Franklin, Han, Shufeng
• Primary Examiner(s): Hellner; Mark
• Assistant Examiner(s): Algahaim; Helal A
• Application Number:
  US11/106,782
• Publication Number:
  US 20060149472A1
• Time in Patent Office: 1,656 Days
• Field of Search: 701/23, 701/28, 701/50, 701/214
• US Class Current: 701/14
• CPC Class Codes: G01C 21/28 with correlation of data from several navigational instruments
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0278 using satellite positioning signals, e.g. GPS

Abstract

A method and system for guiding a vehicle comprises a location determining receiver for collecting location data for the vehicle. A vision module collects vision data for the vehicle. A location quality estimator estimates the location quality data for the location data during an evaluation time window. A vision module estimates vision quality data for the vision data during the evaluation time window. A supervisor module selects at least one of a location data weight and a vision data weight based on the quality data.

Claims

1. A method for guiding a vehicle, the method comprising:

collecting location data for the vehicle based on a location-determining receiver of a location module associated with the vehicle;

collecting vision data for the vehicle based on a vision module associated with the vehicle;

estimating a quality of the collected location data during an evaluation time window to form location quality data;

estimating a quality of the collected vision data during the evaluation time window to form vision quality data;

selecting at least one of a location data weight and a vision data weight to form a selection, wherein the location data weight is based on the location quality data and the vision data weight is based on the vision quality data; and

guiding the vehicle in accordance with the selection, wherein the selecting further comprises;

identifying a quality-mixing ratio based on respective input values associated with the vision quality data and the location quality data;

determining output values associated with the location data weight and the vision data weight based on the identified quality-mixing ratio; and

applying the vision data weight and the location data weight to determine relative contributions of the collected vision data and the collected location data to a vehicular guidance system for guiding the vehicle for a time interval.

2. The method according to claim 1 wherein estimating a quality of the collected location data comprises detecting an off track error, a heading error, and a curvature associated with the location-determining receiver.

3. The method according to claim 1 wherein estimating a quality of the collected vision data comprises detecting an off track error, a heading error, and a curvature associated with the vision module.

4. The method according to claim 1 wherein the vision data weight and the location data weight are consistent with the following equation:

α

_vision+α

_gps=1.

5. The method according to claim 1 wherein:

identifying the quality-mixing ratio based on respective input values associated with the vision quality data and the location quality data, comprises identifying the quality-mixing ratio based on respective input values associated with the vision quality data, the location quality data, and curvature data;

wherein

determining output values associated with the location data weight and the vision data weight based on the identified quality-mixing ratio, comprises determining output values associated with the location data weight, the vision data weight, and a curvature data weight based on the identified quality-mixing ratio; and

wherein

applying the vision data weight and the location data weight to determine relative contributions of the collected vision data and the collected location data to a vehicular guidance system for guiding the vehicle for a time interval, comprises applying the vision data weight, the location data weight, and the curvature data weight to determine relative contributions of the collected vision data and the collected location data to the vehicular guidance system for guiding the vehicle for the time interval.

6. The method according to claim 1 wherein the applying the vision data weight and the location data weight to determine relative contributions of the collected vision data and the collected location data to a vehicular guidance system for guiding the vehicle for a time interval, comprises:

generating an error control signal for steering the vehicle in accordance with the following equations;

y=α

_vision×

y_vision+α

_gps×

y_gps, where y is the error control signal, α

_vision is the vision data weight, y_vision is a vision error signal, α

_gps is the location data weight, and y_gps is a location error signal.

7. The method according to claim 6 wherein the y, y_vision, y_gps, α

_gps, and α

_vision are multidimensional vectors in accordance with the following expressions;

$y=\left[\begin{array}{c}{E}_{\mathrm{off}}\\ E_{\mathrm{head}}\end{array}\right],$

E_off is an aggregate off-track error from an aggregation of error data from the vision module and the location module, E_head is an aggregate heading error from the aggregation of the error data from the vision module and the location module;

$a_{\mathrm{vision}}=\left[\begin{array}{c}{\mathrm{\alpha <br><br>}}_{\mathrm{off}<br><br><br><br>\mathrm{vision}}\\ {\mathrm{\alpha <br><br>}}_{\mathrm{head\_vision}}\end{array}\right],$

where α

_vision is a vision data weight matrix, α

<sub>off—

vision</sub> is the vision data weight for off track error data, and α

<sub>head—

vision</sub> is the vision data weight for heading error data;

$a_{\mathrm{gps}}=\left[\begin{array}{c}{\mathrm{\alpha <br><br>}}_{\mathrm{off\_gps}}\\ {\mathrm{\alpha <br><br>}}_{\mathrm{head\_gps}}\end{array}\right],$

where α

_gps is a location data weight matrix, α

<sub>off—

gps</sub> is the location data weight for off track error data, and α

<sub>head—

gps</sub> is the location data weight for heading error data;

$y_{\mathrm{gps}}=\left[\begin{array}{c}{E}_{\mathrm{off\_gps}}\\ E_{\mathrm{head\_gps}}\end{array}\right],$

where E<sub>off—

gps</sub> is an off-track error estimated by the location module, and E<sub>head—

gps</sub> is a heading error estimated by the location module; and

$y_{\mathrm{vision}}=\left[\begin{array}{c}{E}_{\mathrm{off\_vision}}\\ E_{\mathrm{head\_vision}}\end{array}\right],$

where E<sub>off—

vision</sub> is an off track error estimated by the vision module and E<sub>head—

vision</sub> is a heading error estimated by the vision module.

8. The method according to claim 5 wherein the applying the vision data weight, the location data weight and the curvature data weight to determine relative contributions of the collected vision data and the collected location data to the vehicular guidance system for guiding the vehicle for the time interval, comprises:

generating an error control signal for steering the vehicle in accordance with the following equation;

y=α

_vision×

y_vision+α

_gps×

y_gps, where y is the error control signal, α

_vision is the vision data weight, y_vision is a vision error signal, α

_gps is the location data weight, and y_gps is a location error signal.

9. The method according to claim 8 wherein the y, y_vision, y_gps, α

_vision, and α

_gps, are multidimensional vectors in accordance with the following expressions;

$y=\left[\begin{array}{c}{E}_{\mathrm{off}}\\ E_{\mathrm{head}}\\ \mathrm{\rho <br><br>}\end{array}\right],$

E_off is the aggregate off-track error from the aggregation of error data from the vision module and the location module, E_head is the aggregate heading error from an aggregation of the error data from the vision module and the location module and ρ

is radius of curvature;

$a_{\mathrm{vision}}=\left[\begin{array}{c}{\mathrm{\alpha <br><br>}}_{\mathrm{off\_vision}}\\ {\mathrm{\alpha <br><br>}}_{\mathrm{head\_vision}}\\ {\mathrm{\alpha <br><br>}}_{\mathrm{curv\_vision}}\end{array}\right],$

where α

_vision is the aggregate vision data weight matrix, α

<sub>off—

vision</sub> is the vision data weight for off-track error data, α

<sub>head—

vision</sub> is the vision data weight for heading error data, and α

<sub>curv—

vision</sub> is vision data weight for curvature error data;

$y_{\mathrm{vision}}=\left[\begin{array}{c}{E}_{\mathrm{off\_vision}}\\ E_{\mathrm{head\_vision}}\\ 0\end{array}\right],$

where E<sub>off—

vision</sub> is the off track error estimated by the vision module and E<sub>head—

vision</sub> is the heading error estimated by the vision module;

$a_{\mathrm{gps}}=\left[\begin{array}{c}{\mathrm{\alpha <br><br>}}_{\mathrm{off\_gps}}\\ {\mathrm{\alpha <br><br>}}_{\mathrm{head\_gps}}\\ {\mathrm{\alpha <br><br>}}_{\mathrm{curv\_gps}}\end{array}\right],$

where α

_gps is the aggregate location data weight matrix, α

<sub>off—

gps</sub> is the location data weight for off-track error data, α

<sub>head—

gps</sub> is the location data weight for heading error data, and α

<sub>curv—

gps</sub> is the location data weight for curvature error data;

$y_{\mathrm{gps}}=\left[\begin{array}{c}{E}_{\mathrm{off\_gps}}\\ E_{\mathrm{head\_gps}}\\ {\mathrm{\rho <br><br>}}_{\mathrm{gps}}\end{array}\right],$

where E<sub>off—

gps</sub> is the off-track error estimated by the location module, E<sub>head—

gps</sub> is the heading error estimated by the location module, and ρ

_gps is radius of curvature associated with the location module.

10. A system for guiding a vehicle, the system comprising:

a location-determining receiver of location module associated with the vehicle for collecting location data for the vehicle;

a vision module associated with the vehicle for collecting vision data for the vehicle;

a location quality estimator for estimating a quality of the collected location data during an evaluation time window to form location quality data;

a vision quality estimator for estimating a quality of the collected vision data during the evaluation time window to form vision quality data; and

a selector for selecting at least one of a location data weight and a vision data weight to form a selection, wherein the location data weight is based on the location quality data and the vision data weight is based on the vision quality data; and

a vehicular guidance system for guiding the vehicle in accordance with the selection, wherein the selector selects the at least one of the location data weight and the vision data weight to form the selection by;

identifying a quality-mixing ratio based on respective input values associated with the vision quality data and the location quality data;

determining output values associated with the location data weight, and the vision data weight based on the identified quality-mixing ratio; and

applying the vision data weight and the location data weight to determine relative contributions of the collected vision data and the collected location data to the vehicular guidance system for a time interval.

11. The system according to claim 10 wherein the location quality estimator estimates the quality of the collected location data using an off-track error, a heading error, and a curvature associated with the location-determining receiver.

12. The system according to claim 10 wherein the vision quality estimator estimates the quality of the collected vision data using an off-track error, a heading error, and a curvature associated with the vision module.

13. The system according to claim 10 wherein the vision data weight and the location data weight are consistent with the following equation:

α

_vision+α

_gps=1.

14. The system according to claim 10 wherein the selector selects at least one of the location data weight and the vision data weight to form the selection by:

identifying the quality-mixing ratio based on respective input values associated with the vision quality data, the location quality data, and curvature data;

determining the output values associated with the location data weight, the vision data weight, and curvature data weight based on the identified quality-mixing ratio; and

applying the vision data weight, the location data weight, and the curvature data weight to determine relative contributions of the collected vision data and the collected location data to the vehicular guidance system for the time interval.

15. The system according to claim 10 wherein the applying the vision data weight and the location data weight to determine relative contributions of the collected vision data and the collected location data to the vehicular guidance system for the time interval, comprises:

generating an error control signal for steering the vehicle in accordance with the following equation;

y=α

_vision×

y_vision+α

_gps×

y_gps, where y is the error control signal, α

_vision is vision data weight, y_vision is a vision error signal, α

_gps is the location data weight, and y_gps is a location error signal.

16. The system according to claim 15 wherein the y, y_vision, y_gps, α

_gps, and α

_vision are multidimensional vectors in accordance with the following expressions;

$y=\left[\begin{array}{c}{E}_{\mathrm{off}}\\ E_{\mathrm{head}}\end{array}\right],$

E_off is the aggregate off-track error from the aggregation of error data from the vision module and the location module, E_head is the aggregate heading error from the aggregation of the error data from the vision module and the location module;

${\mathrm{\alpha <br><br>}}_{\mathrm{vision}}=\left[\begin{array}{c}{\mathrm{\alpha <br><br>}}_{\mathrm{off\_vision}}\\ {\mathrm{\alpha <br><br>}}_{\mathrm{head\_vision}}\end{array}\right],$

where α

_vision is the vision data weight matrix, α

<sub>off—

vision</sub> is the vision data weight for off track error data, and α

<sub>head—

vision</sub> is the vision data weight for heading error data;

${\mathrm{\alpha <br><br>}}_{\mathrm{gps}}=\left[\begin{array}{c}{\mathrm{\alpha <br><br>}}_{\mathrm{off}<br><br><br><br>\mathrm{gps}}\\ {\mathrm{\alpha <br><br>}}_{\mathrm{head}<br><br><br><br>\mathrm{gps}}\end{array}\right],$

where α

_gps is the location data weight matrix, α

<sub>off—

gps</sub> is the location data weight for off track error data, and α

<sub>head—

gps</sub> is the location data weight for heading error data;

$y_{\mathrm{gps}}=\left[\begin{array}{c}{E}_{\mathrm{off\_gps}}\\ E_{\mathrm{head}<br><br><br><br>\mathrm{gps}}\end{array}\right],$

where E<sub>off—

gps</sub> is the off-track error estimated by the location module and E<sub>head—

gps</sub> is the heading error estimated by the location module; and

$y_{\mathrm{vision}}=\left[\begin{array}{c}{E}_{\mathrm{off\_vision}}\\ E_{\mathrm{head\_vision}}\end{array}\right],$

where E<sub>off—

vision</sub> is the off track error estimated by the vision module and E<sub>head—

vision</sub> is a heading error estimated by the vision module.

17. The system according to claim 14 wherein the applying the vision data weight, the location data weight, and the curvature data weight to determine relative contributions of the collected vision data and the collected location data to the vehicular guidance system for the time interval, comprises:

generating an error control signal for steering the vehicle in accordance with the following equation;

y=α

_vision×

y_vision+α

_gps×

y_gps, where y is the error control signal, α

_vision is the vision data weight, y_vision is a vision error signal, α

_gps is the location data weight, and y_gps is a location error signal.

18. The system according to claim 17 wherein the y, y_vision, y_gps, α

_vision, and α

_gps are multidimensional vectors in accordance with the following expressions;

$y=\left[\begin{array}{c}{E}_{\mathrm{off}}\\ E_{\mathrm{head}}\\ \mathrm{\rho <br><br>}\end{array}\right],$

E_off is the aggregate off-track error from the aggregation of error data from the vision module and the location module, E_head is the aggregate heading error from an aggregation of the error data from the vision module and the location module and ρ

is radius of curvature based on the vision module and the location module;

${\mathrm{\alpha <br><br>}}_{\mathrm{vision}}=\left[\begin{array}{c}{\mathrm{\alpha <br><br>}}_{\mathrm{off\_vision}}\\ {\mathrm{\alpha <br><br>}}_{\mathrm{head\_vision}}\\ {\mathrm{\alpha <br><br>}}_{\mathrm{curv\_vision}}\end{array}\right],$

where α

_vision is the aggregate vision data weight matrix, α

<sub>off—

vision</sub> is the vision data weight for off-track error data, α

<sub>head—

vision</sub> is the vision data weight for heading error data, and α

<sub>curv—

vision</sub> is vision data weight for curvature error data;

$y_{\mathrm{vision}}=\left[\begin{array}{c}{E}_{\mathrm{off\_vision}}\\ E_{\mathrm{head\_vision}}\\ {\mathrm{\rho <br><br>}}_{\mathrm{vision}}\end{array}\right],$

where E<sub>off—

vision</sub>, is the off track error estimated by the vision module and E<sub>head—

vision</sub> is the heading error estimated by the vision module, and ρ

_v is a radius of curvature associated with the vision module;

${\mathrm{\alpha <br><br>}}_{\mathrm{gps}}=\left[\begin{array}{c}{\mathrm{\alpha <br><br>}}_{\mathrm{off\_gps}}\\ {\mathrm{\alpha <br><br>}}_{\mathrm{head\_gps}}\\ {\mathrm{\alpha <br><br>}}_{\mathrm{curv\_gps}}\end{array}\right],$

where α

_gps is the aggregate location data weight matrix, α

<sub>off—

gps</sub> is the location data weight for off-track error data, α

<sub>head—

gps</sub> is the location data weight for heading error data, and α

<sub>curv—

gps</sub> is the location data weight for curvature error data;

$y_{\mathrm{gps}}=\left[\begin{array}{c}{E}_{\mathrm{off\_gps}}\\ E_{\mathrm{head\_gps}}\\ {\mathrm{\rho <br><br>}}_{\mathrm{gps}}\end{array}\right],$

where E<sub>off—

gps</sub> is the off-track error estimated by the location module, E<sub>head—

gps</sub> is the heading error estimated by the location module, and ρ

_gps is radius of curvature associated with the location module.