Vision-aided system and method for guiding a vehicle

• US 20060149472A1
• Filed: 04/15/2005
• Published: 07/06/2006
• Est. Priority Date: 01/04/2005
• Status: Active Grant

• Current Assignee: Deere & Company
• Original Assignee: Deere & Company
• Inventors: Reid, John Franklin, Han, Shufeng, Rovira-Mas, Francisco
• Granted Patent: US 7,610,123 B2
• Time in Patent Office: Days
• US Class Current: 701/214
• CPC Class Codes: G01C 21/28 with correlation of data from several navigational instruments
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0278 using satellite positioning signals, e.g. GPS

Abstract

A method and system for guiding a vehicle comprises a location determining receiver for collecting location data for the vehicle. A vision module collects vision data for the vehicle. A location quality estimator estimates the location quality data for the location data during an evaluation time window. A vision module estimates vision quality data for the vision data during the evaluation time window. A supervisor module selects at least one of a location data weight and a vision data weight based on the quality data.

Claims

1. A method for guiding a vehicle, the method comprising:

collecting location data for the vehicle based on a location-determining receiver associated with the vehicle;

collecting vision data for the vehicle based on a vision module associated with the vehicle;

estimating location quality data for the location data during an evaluation time window;

estimating vision quality data for the vision data during the evaluation time window; and

selecting at least one of a location data weight and a vision data weight based on the quality data.

2. The method according to claim 1 wherein the estimating of the location quality data comprises detecting an off-track error, a heading error, and a curvature associated with the location-determining receiver.

3. The method according to claim 1 wherein the estimating of the vision quality data comprises detecting an off-track error, a heading error, and a curvature associated with the vision module.

4. The method according to claim 1 wherein the vision data weight and the location data weight are consistent with the following equation:

α

_vision+α

_gps=1.

5. The method according to claim 1 wherein the selecting further comprises:

identifying a quality-mixing ratio relationship based on the respective input values associated with the vision quality data and the location quality data;

determining output values associated with the location data weight, and the vision data weight based on the identified quality-mixing ratio; and

applying the vision data weight and the location data weight to determine the relative contributions of the vision data and the location data to the vehicular guidance system for a time interval.

6. The method according to claim 1 wherein the selecting further comprises:

identifying a quality-mixing ratio relationship based on the respective input values associated with the vision quality data, the location quality data, and curvature data;

determining output values associated with the location data weight, the vision data weight, and curvature data weight based on the identified quality-mixing ratio; and

applying the vision data weight, the location data weight, and the curvature data weight to determine the relative contributions of the vision data and the location data to the vehicular guidance system for a time interval.

7. The method according to claim 1 wherein the selecting further comprises:

identifying a quality-mixing ratio relationship based on the respective input values associated with the vision quality data and the location quality data;

determining output values associated with the location data weight, and the vision data weight based on the identified quality-mixing ratio; and

generating an error control signal for steering the vehicle in accordance with the following equations;

y=α

_vision×

y_vision+α

_gps×

y_gps, where y is the error control signal, α

_vision is vision data weight, y_vision is the vision error signal, α

_gps is the location data weight and y_gps is the location error signal.

8. The method according to claim 7 wherein the y, y_vision, y_gps, α

_gps, and α

_vision are multidimensional vectors in accordance with the following expressions;

$y=\left[\begin{array}{c}{E}_{\mathrm{off}}\\ E_{\mathrm{head}}\end{array}\right],$

E_off is the aggregate off-track error from the aggregation of error date from the vision module and the location module, E_head is the aggregate heading error from the aggregation of the error data from the vision module and the location module;

$a_{\mathrm{vision}}=\left[\begin{array}{c}{\mathrm{\alpha <br><br>}}_{\mathrm{off}<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{vision}}\\ {\mathrm{\alpha <br><br>}}_{\mathrm{head\_vision}}\end{array}\right],$

where α

_vision is the vision data weight matrix, α

<sub>off—

vision</sub> is the vision data weight for off track error data, and α

<sub>head—

vision</sub> is the vision data weight for heading error data;

$a_{\mathrm{gps}}=\left[\begin{array}{c}{\mathrm{\alpha <br><br>}}_{\mathrm{off\_gps}}\\ {\mathrm{\alpha <br><br>}}_{\mathrm{head\_gps}}\end{array}\right],$

where α

_gps is the location data weight matrix, α

<sub>off—

gps</sub> is the location data weight for off track error data, and α

<sub>head—

gps</sub> is the location data weight for heading error data;

$y_{\mathrm{gps}}=\left[\begin{array}{c}{E}_{\mathrm{off\_gps}}\\ E_{\mathrm{head\_gps}}\end{array}\right],$

where E<sub>off—

gps</sub> is the off-track error estimated by the location module (e.g., location-determining receiver), and E<sub>head—

gps</sub> is the heading error estimated by the location module; and

$y_{\mathrm{vision}}=\left[\begin{array}{c}{E}_{\mathrm{off\_vision}}\\ E_{\mathrm{head\_vision}}\end{array}\right],$

where E<sub>off—

vision</sub> is the off track error estimated by the vision module and E<sub>head—

vision</sub> is the heading error estimated by the vision module.

9. The method according to claim 1 wherein the selecting further comprises:

identifying a quality-mixing ratio relationship based on the respective input values associated with the vision quality data, the location quality data, and curvature data;

determining output values associated with the location data weight, the vision data weight, and curvature data weight based on the identified quality-mixing ratio; and

generating an error control signal for steering the vehicle in accordance with the following equation;

y=α

_vision×

y_vision+α

_gps×

y_gps, where y is the error control signal, α

_vision is vision data weight, y_vision is the vision error signal, α

_gps is the location data weight, and y_gps is the location error signal.

10. The method according to claim 9 wherein the y, y_vision, y_gps, α

_vision, and α

_gps, are multidimensional vectors in accordance with the following expressions;

$y=\left[\begin{array}{c}{E}_{\mathrm{off}}\\ E_{\mathrm{head}}\\ \mathrm{\rho <br><br>}\end{array}\right],$

E_off is the aggregate off-track error from the aggregation of error data from the vision module and the location module, E_head is the aggregate heading error from the aggregation of the error data from the vision module and the location module and ρ

is the radius of curvature;

$a_{\mathrm{vision}}=\left[\begin{array}{c}{\mathrm{\alpha <br><br>}}_{\mathrm{off\_vision}}\\ {\mathrm{\alpha <br><br>}}_{\mathrm{head\_vision}}\\ {\mathrm{\alpha <br><br>}}_{\mathrm{curv\_vision}}\end{array}\right],$

where α

_vision is the aggregate vision data weight matrix, α

<sub>off—

vision</sub> is the vision data weight for off-track error data, α

<sub>head—

vision</sub> is the vision data weight for heading error data, and α

<sub>curv—

vision</sub> is vision data weight for curvature error data;

$y_{\mathrm{vision}}=\left[\begin{array}{c}{E}_{\mathrm{off\_vision}}\\ E_{\mathrm{head\_vision}}\\ 0\end{array}\right],$

where E<sub>off—

vision</sub> is the off track error estimated by the vision module and E<sub>head—

vision</sub> is the heading error estimated by the vision module;

$a_{\mathrm{gps}}=\left[\begin{array}{c}{\mathrm{\alpha <br><br>}}_{\mathrm{off\_gps}}\\ {\mathrm{\alpha <br><br>}}_{\mathrm{head\_gps}}\\ {\mathrm{\alpha <br><br>}}_{\mathrm{curv\_gps}}\end{array}\right],$

where α

_gps is the aggregate location data weight matrix, α

<sub>off—

gps</sub> is the location data weight for off-track error data, α

<sub>head—

gps</sub> is the location data weight for heading error data, and α

<sub>curv—

gps</sub> is the location data weight for curvature error data;

$y_{\mathrm{gps}}=\left[\begin{array}{c}{E}_{\mathrm{off\_gps}}\\ E_{\mathrm{head\_gps}}\\ {\mathrm{\rho <br><br>}}_{\mathrm{gps}}\end{array}\right],$

where E<sub>off—

gps</sub> is the off-track error estimated by the location module, E<sub>head—

gps</sub> is the heading error estimated by the location module, and ρ

_gps is the radius of curvature associated with the location module.

11. A system for guiding a vehicle, the system comprising:

collecting location data for the vehicle based on a location-determining receiver associated with the vehicle;

collecting vision data for the vehicle based on a vision module associated with the vehicle;

estimating location quality data for the location data during an evaluation time window;

estimating vision quality data for the vision data during the evaluation time window; and

selecting at least one of a location data weight and a vision data weight based on the quality data.

12. The system according to claim 11 wherein the estimating of the location quality data comprises detecting an off-track error, a heading error, and a curvature associated with the location-determining receiver.

13. The system according to claim 11 wherein the estimating of the vision quality data comprises detecting an off-track error, a heading error, and a curvature associated with the vision module.

14. The system according to claim 11 wherein the vision data weight and the location data weight are consistent with the following equation:

α

_vision+α

_gps=1.

16. The system according to claim 11 wherein the selecting further comprises:

identifying a quality-mixing ratio relationship based on the respective input values associated with the vision quality data and the location quality data;

determining output values associated with the location data weight, and the vision data weight based on the identified quality-mixing ratio; and

applying the vision data weight and the location data weight to determine the relative contributions of the vision data and the location data to the vehicular guidance system for a time interval.

17. The system according to claim 11 wherein the selecting further comprises:

identifying a quality-mixing ratio relationship based on the respective input values associated with the vision quality data, the location quality data, and curvature data;

determining output values associated with the location data weight, the vision data weight, and curvature data weight based on the identified quality-mixing ratio; and

applying the vision data weight, the location data weight, and the curvature data weight to determine the relative contributions of the vision data and the location data to the vehicular guidance system for a time interval.

18. The system according to claim 11 wherein the selecting further comprises:

identifying a quality-mixing ratio relationship based on the respective input values associated with the vision quality data and the location quality data;

determining output values associated with the location data weight, and the vision data weight based on the identified quality-mixing ratio; and

generating an error control signal for steering the vehicle in accordance with the following equation;

y=α

_vision×

y_vision+α

_gps×

y_gps, where y is the error control signal, α

_vision is vision data weight, y_vision is the vision error signal, α

_gps is the location data weight, and y_gps is the location error signal.

19. The system according to claim 18 wherein the y, y_vision, y_gps, α

_gps, and α

_vision are multidimensional vectors in accordance with the following expressions;

$y=\left[\begin{array}{c}{E}_{\mathrm{off}}\\ E_{\mathrm{head}}\end{array}\right],$

E_off is the aggregate off-track error from the aggregation of error data from the vision module and the location module, E_head is the aggregate heading error from the aggregation of the error data from the vision module and the location module;

${\mathrm{\alpha <br><br>}}_{\mathrm{vision}}=\left[\begin{array}{c}{\mathrm{\alpha <br><br>}}_{\mathrm{off\_vision}}\\ {\mathrm{\alpha <br><br>}}_{\mathrm{head\_vision}}\end{array}\right],$

where α

_vision is the vision data weight matrix, α

<sub>off—

vision</sub> is the vision data weight for off track error data, and α

<sub>head—

vision</sub> is the vision data weight for heading error data;

${\mathrm{\alpha <br><br>}}_{\mathrm{gps}}=\left[\begin{array}{c}{\mathrm{\alpha <br><br>}}_{\mathrm{off}<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{gps}}\\ {\mathrm{\alpha <br><br>}}_{\mathrm{head}<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{gps}}\end{array}\right],$

where α

_gps is the location data weight matrix, α

<sub>off—

gps</sub> is the location data weight for off track error data, and α

<sub>head—

gps</sub> is the location data weight for heading error data;

$y_{\mathrm{gps}}=\left[\begin{array}{c}{E}_{\mathrm{off\_gps}}\\ E_{\mathrm{head}<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{gps}}\end{array}\right],$

where E<sub>off—

gps</sub> is the off-track error estimated by the location module (e.g., location-determining receiver), and E<sub>head—

gps</sub> is the heading error estimated by the location module; and

$y_{\mathrm{vision}}=\left[\begin{array}{c}{E}_{\mathrm{off\_vision}}\\ E_{\mathrm{head\_vision}}\end{array}\right],$

where E<sub>off—

vision</sub> is the off track error estimated by the vision module and E<sub>head—

vision</sub> is the heading error estimated by the vision module.

20. The system according to claim 11 wherein the selecting further comprises:

identifying a quality-mixing ratio relationship based on the respective input values associated with the vision quality data, the location quality data, and curvature quality data;

determining output values associated with the location data weight, the vision data weight, and curvature data weight based on the identified quality-mixing ratio; and

generating an error control signal for steering the vehicle in accordance with the following equation;

y=α

_vision×

y_vision+α

_gps×

y_gps, where y is the error control signal, α

_vision is vision data weight, y_vision is the vision error signal, α

_gps is the location data weight, and y_gps is the location error signal.

21. The system according to claim 20 wherein the y, y_vision, y_gps, α

_vision, and α

_gps are multidimensional vectors in accordance with the following expressions;

$y=\left[\begin{array}{c}{E}_{\mathrm{off}}\\ E_{\mathrm{head}}\\ \mathrm{\rho <br><br>}\end{array}\right],$

E_off is the aggregate off-track error from the aggregation of error data from the vision module and the location module, E_head is the aggregate heading error from the aggregation of the error data from the vision module and the location module and ρ

is the radius of curvature based on the vision module and the location module;

${\mathrm{\alpha <br><br>}}_{\mathrm{vision}}=\left[\begin{array}{c}{\mathrm{\alpha <br><br>}}_{\mathrm{off\_vision}}\\ {\mathrm{\alpha <br><br>}}_{\mathrm{head\_vision}}\\ {\mathrm{\alpha <br><br>}}_{\mathrm{curv\_vision}}\end{array}\right],$

where α

_vision is the aggregate vision data weight matrix, α

<sub>off—

vision</sub> is the vision data weight for off-track error data, α

<sub>head—

vision</sub> is the vision data weight for heading error data, and α

<sub>curv—

vision</sub> is vision data weight for curvature error data;

$y_{\mathrm{vision}}=\left[\begin{array}{c}{E}_{\mathrm{off\_vision}}\\ E_{\mathrm{head\_vision}}\\ {\mathrm{\rho <br><br>}}_{\mathrm{vision}}\end{array}\right],$

where E<sub>off—

vision</sub>, is the off track error estimated by the vision module and E<sub>head—

vision</sub> is the heading error estimated by the vision module, and ρ

_v is the radius of curvature associated with the vision module;

${\mathrm{\alpha <br><br>}}_{\mathrm{gps}}=\left[\begin{array}{c}{\mathrm{\alpha <br><br>}}_{\mathrm{off\_gps}}\\ {\mathrm{\alpha <br><br>}}_{\mathrm{head\_gps}}\\ {\mathrm{\alpha <br><br>}}_{\mathrm{curv\_gps}}\end{array}\right],$

where α

_gps is the aggregate location data weight matrix, α

<sub>off—

gps</sub> is the location data weight for off-track error data, α

<sub>head—

gps</sub> is the location data weight for heading error data, and α

<sub>curv—

gps</sub> is the location data weight for curvature error data;

$y_{\mathrm{gps}}=\left[\begin{array}{c}{E}_{\mathrm{off\_gps}}\\ E_{\mathrm{head\_gps}}\\ {\mathrm{\rho <br><br>}}_{\mathrm{gps}}\end{array}\right],$

where E<sub>off—

gps</sub> is the off-track error estimated by the location module, E<sub>head—

gps</sub> is the heading error estimated by the location module, and ρ

_gps is the radius of curvature associated with the location module.

15. (canceled)