Method and system for vehicular guidance using a crop image

• US 7,792,622 B2
• Filed: 12/15/2005
• Issued: 09/07/2010
• Est. Priority Date: 07/01/2005
• Status: Active Grant

• Current Assignee: Deere & Company, Kansas State University Research Foundation
• Original Assignee: Deere & Company, Kansas State University Research Foundation
• Inventors: Han, Shufeng, Wei, Jiantao
• Primary Examiner(s): Tran; Khoi
• Assistant Examiner(s): Peche; Jorge O
• Application Number:
  US11/305,651
• Publication Number:
  US 20070001097A1
• Time in Patent Office: 1,727 Days
• Field of Search: 701/23, 701/28, 701/50, 701/301, 700/50, 700/57, 700/58, 700/59, 700/61, 700/192, 700/259, 700/279, 382/100, 382/103, 382/104, 382/106, 382/107, 382/153, 382/162, 382/164, 382/165, 382/168, 382/173, 382/181, 382193-203, 382/224, 382/232, 382/256, 382/274, 382/276, 382/286, 382/312, 382/305, 56/135, 56/145, 56/146, 56/164.R, 345/158, 345/581, 345/589, 345/591, 345/597, 345/598, 345/599, 345/600, 345/634, 345/635
• US Class Current: 701/50
• CPC Class Codes: A01B 69/001 Steering by means of optical assistance, e.g. television cameras steering devices for road marking vehicles E01C23/163
• CPC Class Codes: A01B 69/008 automatic

Abstract

The method and system for vehicular guidance comprises an imaging device for collecting color image data to facilitate distinguishing crop image data (e.g., crop rows) from background data. A definer defines a series of scan line segments generally perpendicular to a transverse axis of the vehicle or of the imaging device. An intensity evaluator determines scan line intensity data for each of the scan line segments. An alignment detector (e.g., search engine) identifies a preferential heading of the vehicle that is generally aligned with respect to a crop feature, associated with the crop image data, based on the determined scan line intensity meeting or exceeding a maximum value or minimum threshold value.

Claims

1. A method of guiding a vehicle, the method comprising:

collecting color image data to distinguish crop image data having one or more crop rows from background image data;

defining a series of scan line segments generally perpendicular to a transverse axis of the vehicle or of an imaging system using a data processor;

determining a scan line intensity for each of the scan line segments with respect to corresponding crop image data, wherein determining the scan line intensity comprises at least one of determining an average or mean scan line intensity for a corresponding scan line segment and determining a mode scan line intensity for the corresponding scan line segment;

identifying a preferential heading of the vehicle that is generally aligned with respect to a crop feature, associated with the crop image data, based on the determined scan line intensity meeting or exceeding a maximum value or minimum threshold value, wherein the minimum threshold value refers to a substantial alignment of the vehicle with the crop feature;

estimating a reliability of the vehicle heading based on compliance with an intensity level criteria associated with one or more crop rows;

determining a first intensity value associated with a first crop row and a second intensity value associated with a second crop row;

considering vision guidance data unreliable for a minimum time period if the first intensity value differs from the second intensity value by more than a minimum threshold;

determining a spacing between a first crop row and a second crop row;

determining a maximum intensity value for the first crop row and the second crop row;

considering vision guidance data reliable if the first intensity value does not differ from the second intensity value by more than the minimum threshold;

determining whether the vision guidance data is reliable;

responsive to a determination that the vision guidance data is reliable, using the vision guidance data to guide a vehicle; and

controlling a path of the vehicle in accordance with the identified preferential heading of the vehicle.

2. The method according to claim 1 wherein the identifying further comprises searching candidate headings to select the preferential heading based on the determined scan line intensity data.

3. The method according to claim 1 wherein the crop feature comprises one or more crop rows.

4. The method according to claim 1 wherein the crop feature comprises a crop edge between harvested and unharvested crop.

5. The method according to claim 1 further comprising:

deriving a segmented image of crop rows as the crop image data from the collected color image.

6. The method according to claim 1 wherein the collecting further comprises classifying image data as crop image data if a green-red difference associated with a pixel is greater than a first threshold value and if a green-blue difference associated with the pixel is greater than a second threshold value.

7. The method according to claim 6 wherein the green-red difference (D_g−

r) is defined as D<sub>g−

r</sub>=G−

R and the green-blue difference (D<sub>g−

b</sub>) is defined as D<sub>g−

b</sub>=G−

B, where R,G,B are the pixel'"'"'s red, green and blue component, respectively, in RGB color space.

8. The method according to claim 1 wherein the collecting further comprises classifying image data as soil data if a red-green difference associated with a pixel is less than or equal to first threshold value or if a green-blue difference associated with the pixel is less than or equal to the second threshold value.

9. The method according to claim 1 wherein the determining of the scan line intensity comprises determining an aggregate scan line intensity for a corresponding scan line segment.

10. The method according to claim 1 wherein the identifying the preferential heading comprises limiting a search of candidate heading angles (φ

) from −

φ

_max to φ

_max, to select the preferential heading angle, and limiting a search of a lateral view along the transverse axis from −

x_max to x_max.

11. The method according to claim 1 wherein the average pixel intensity, I(φ

,x), on the scan line segment at lateral position x with heading angle φ

, is calculated as;

$I<br><br>\left(\mathrm{\phi <br><br>},x\right)=\frac{1}{N<br><br>\left(\mathrm{\phi <br><br>},x\right)}<br><br>\underset{y=y_{\min }}{\overset{y_{\max }}{\sum <br><br>}}<br><br>P<br><br>\left(x,y\right)$

where N(φ

, x) is the number of pixels on the scan line segment x, P(x,y) is the pixel'"'"'s intensity value at coordinate (x,y), and wherein the evaluation is limited to scan line segments between y_min and y_max.

12. The method according to claim 1 wherein the identifying further comprises determining an intensity variation or derivative of I(φ

, x) as a judging function, σ

(φ

) to search for the vehicle'"'"'s preferential heading angle;

$\mathrm{\sigma <br><br>}<br><br>\left(\mathrm{\phi <br><br>}\right)=\frac{1}{N_x}<br><br>\underset{x=-x_{\max }}{\overset{x_{\max }-1}{\sum <br><br>}}<br><br><br><br><br><br>I<br><br>\left(\mathrm{\phi <br><br>},x\right)-I<br><br>\left(\mathrm{\phi <br><br>},x+1\right)<br><br>$

where N_x is the number of scan lines.

13. The method according to claim 12 wherein the identifying further comprises averaging the intensity variation or derivative of I(φ

, x) to produce a refined judging function, σ

(φ

,ω

) to search for the vehicle'"'"'s preferential heading angle;

$\mathrm{\sigma <br><br>}<br><br>\left(\mathrm{\phi <br><br>}\right)=\frac{1}{N_{\mathrm{\omega <br><br>}}}<br><br>\underset{\mathrm{\omega <br><br>}=-{\mathrm{\omega <br><br>}}_{\max }}{\overset{{\mathrm{\omega <br><br>}}_{\max }}{\sum <br><br>}}<br><br><br><br>\mathrm{\sigma <br><br>}<br><br>\left(\mathrm{\phi <br><br>},\mathrm{\omega <br><br>}\right)$

where N_w is the number of scan lines, and wherein ω

represents the pitch angle that ranges from a minimum pitch angle (−

ω

_max) to a maximum pitch angle (ω

_max).

14. The method according to claim 1 further comprises:

determining an offset for the vehicle along the transverse axis.

15. The method according to claim 14 further comprises:

controlling a path of the vehicle in accordance with the identified vehicle heading and the determined offset.

16. A system of guiding a vehicle, the system comprising:

an imaging device for collecting color image data to distinguish crop image data having one or more crop rows from background image data;

a definer for defining a series of scan line segments generally perpendicular to a transverse axis of the vehicle or of an imaging system;

an intensity evaluator for determining a scan line intensity for each of the scan line segments with respect to corresponding crop image data, wherein the intensity evaluator determines at least one of an average or mean scan line intensity as the scan line intensity determined for a corresponding scan line segment and a mode scan line intensity as the scan line intensity determined for the corresponding scan line segment, and wherein the intensity evaluator determines a first intensity value associated with a first crop row and a second intensity value associated with a second crop row, the intensity evaluator determining a maximum intensity value for the first crop row and the second crop row;

an alignment detector for identifying a preferential heading of the vehicle that is generally aligned with respect to a crop feature, associated with the crop image data, based on the determined scan line intensity meeting or exceeding a maximum value or minimum threshold value, wherein the minimum threshold value refers to a substantial alignment of the vehicle with the crop feature;

a vehicle guidance controller for controlling a path of the vehicle in accordance with the identified preferential heading of the vehicle;

a reliability estimator for estimating a reliability of the vehicle heading based on compliance with an intensity level criteria associated with one or more crop rows, wherein the reliability estimator considers vision guidance data unreliable for a minimum time if the first intensity value differs from the second intensity value by more than a minimum threshold, and wherein the reliability estimator considers vision guidance data reliable if the first intensity value does not differ from the second intensity value by more than the minimum threshold; and

a data processor for determining a spacing between a first crop row and a second crop row and for determining whether the vision guidance data is reliable, wherein responsive to a determination that the vision guidance data is reliable, the vision guidance data is used to guide the vehicle.

17. The system according to claim 16 wherein the alignment detector further comprises searching candidate headings to select the preferential heading based on the determined scan line intensity data.

18. The system according to claim 16 wherein the crop feature comprises one or more crop rows.

19. The system according to claim 16 wherein the crop feature comprises a crop edge between harvested and unharvested crop.

20. The system according to claim 16 further comprising:

an image segmenter for deriving a segmented image of crop rows as the crop image data from the collected color image.

21. The system according to claim 16 wherein the discriminator classifies image data as crop image data if a green-red difference associated with a pixel is greater than a first threshold value and if a green-blue difference associated with the pixel is greater than a second threshold value.

22. The system according to claim 16 wherein the discriminator classifies image data as soil data if a red-green difference associated with a pixel is less than or equal to first threshold value or if a green-blue difference associated with the pixel is less than or equal to the second threshold value.

23. The system according to claim 16 wherein the intensity evaluator determines an aggregate scan line intensity as the determined scan line intensity for a corresponding scan line segment.

24. The system according to claim 16 wherein the identifying the preferential heading comprises limiting a search of candidate heading angles (φ

) to −

φ

_max to φ

_max, to select the preferential heading angle, and limiting a search of a lateral view along the transverse axis from −

x_max to x_max.

25. The system according to claim 16 wherein the average pixel intensity, I(φ

, x), on the scan line at lateral position x with heading angle φ

, is calculated as;

$I<br><br>\left(\mathrm{\phi <br><br>},x\right)=\frac{1}{N<br><br>\left(\mathrm{\phi <br><br>},x\right)}<br><br>\underset{y=y_{\min }}{\overset{y_{\max }}{\sum <br><br>}}<br><br>P<br><br>\left(x,y\right)$

where N(φ

, x) is the number of pixels on the scan line x and P(x,y) is the pixel'"'"'s intensity value at coordinate (x,y).

26. The system according to claim 19 wherein the alignment detector uses an intensity variation (derivative) of I(φ

, x) as a judging function (σ

(φ

) to expedite searching of scan line intensity, lines is defined as a judging function to search for the vehicle'"'"'s heading angle;

$\mathrm{\sigma <br><br>}<br><br>\left(\mathrm{\phi <br><br>}\right)=\frac{1}{N_x}<br><br>\underset{x=-x_{\max }}{\overset{x_{\max }-1}{\sum <br><br>}}<br><br><br><br><br><br>I<br><br>\left(\mathrm{\phi <br><br>},x\right)-I<br><br>\left(\mathrm{\phi <br><br>},x+1\right)<br><br>$

where N_x is the number of scan lines.

27. The system according to claim 16 further comprises:

an offset calculator for determining an offset for the vehicle along the transverse axis.

28. The system according to claim 27 further comprises:

a vehicle guidance controller for controlling a path of the vehicle in accordance with the identified vehicle heading and the determined offset.