Method and system for vehicular guidance using a crop image

• US 7,580,549 B2
• Filed: 12/16/2005
• Issued: 08/25/2009
• Est. Priority Date: 07/01/2005
• Status: Active Grant

• Current Assignee: Deere & Company, Kansas State University Research Foundation
• Original Assignee: Deere & Company, Kansas State University Research Foundation
• Inventors: Han, Shufeng, Wei, Jiantao
• Primary Examiner(s): AZARIAN, SEYED H
• Application Number:
  US11/303,696
• Publication Number:
  US 20070014434A1
• Time in Patent Office: 1,348 Days
• Field of Search: 382/100, 382/103, 382/104, 382/106, 382/107, 382/153, 382/162, 382/164, 382/165, 382/168, 382/173, 382/181, 382193-203, 382/224, 382/232, 382/256, 382/274, 382/286, 382/312, 382/305, 382/276, 700/50, 701/23, 701/28, 701/301
• US Class Current: 382/104
• CPC Class Codes: A01B 69/001 Steering by means of optical assistance, e.g. television cameras steering devices for road marking vehicles E01C23/163
• CPC Class Codes: G06V 10/243 by compensating for image skew or non-uniform image deformations
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

The method and system for vehicular guidance comprises an imaging device for collecting color image data. A definer defines a series of scan line segments generally perpendicular to a transverse axis of the vehicle or of the imaging device. An image parameter evaluator determines scan line image parameter data for each of the scan line segments. An alignment detector (e.g., search engine) identifies a preferential heading of the vehicle that is generally aligned with respect to a crop feature, associated with the image data, based on the determined scan line image parameter meeting or exceeding a maximum value or minimum threshold value.

Claims

1. A method of guiding a vehicle, the method comprising:

collecting color image data to distinguish crop image data from background image data;

wherein the collecting further comprises classifying image data as crop image data if a green-red difference associated with a pixel is greater than a first threshold value and if a green-blue difference associated with the pixel is greater than a second threshold value, and wherein the green-red difference (D<sub>g−

r</sub>) is defined as D<sub>g−

r</sub>=G−

R and the green-blue difference (D<sub>g−

b</sub>) is defined as D<sub>g−

b</sub>=G−

B, where R,G,B are the pixel'"'"'s red, green and blue component, respectively, in RGB color space;

defining a series of scan line segments generally perpendicular to a transverse axis of the vehicle or of an imaging system;

determining, by a data processor, a scan-line-image-parameter level for each of the scan line segments, the scan-line-image-parameter level comprising a color-related parameter level; and

identifying a preferential heading of the vehicle that is generally aligned with respect to a crop feature, associated with crop image data, based on the determined scan-line-image-parameter level meeting or exceeding a maximum value or minimum threshold value.

2. The method according to claim 1 wherein image parameter level is associated with or derived from one or more of the following:

a color value of a red component, green component, or blue component in RGB color space;

a composite color value of multiple components in RGB color space, a hue value in the HSV color space;

a hue value in the HSL color space;

a saturation value associated with vibrancy or purity of a color, a saturation value in HSV or HSL color space, a value indicative of observed or reflected light in the green visible light spectrum or for another plant color;

a value indicative of observed or reflected light with certain green hue value or another plant color; and

a value indicative of observed or reflected light in multiple spectrums (e.g., green light and infra-red or near infra-red light), an NDVI (normalized difference vegetation index), a greenness level, a plant color level, and a green-red difference level.

3. The method according to claim 1 wherein the identifying further comprises searching candidate headings to select the preferential heading based on the determined scan line image parameter data.

4. The method according to claim 1 wherein the crop feature comprises one or more crop rows.

5. The method according to claim 1 wherein the crop feature comprises a crop edge between harvested and unharvested crop.

6. The method according to claim 1 further comprising:

deriving a segmented image of crop rows as the crop image data from the collected color image.

7. The method according to claim 1 wherein the collecting further comprises classifying image data as soil data if a red-green difference associated with a pixel is less than or equal to a first threshold value or if a green-blue difference associated with the pixel is less than or equal to a second threshold value.

8. The method according to claim 1 wherein the determining of the scan line image parameter comprises determining an average or mean scan line image parameter for a corresponding scan line segment.

9. The method according to claim 1 wherein the determining of the scan line image parameter comprises determining a mode scan line image parameter for a corresponding scan line segment.

10. The method according to claim 1 wherein the determining of the scan line image parameter comprises determining an aggregate scan line image parameter for a corresponding scan line segment.

11. The method according to claim 1 wherein the identifying the preferential heading comprises limiting a search of candidate heading angles (φ

) from −

φ

_max to φ

_max, to select the preferential heading angle, and limiting a search of a lateral view along the transverse axis from −

x_max to x_max.

12. The method according to claim 1 wherein the average pixel image parameter, I(φ

, x), on the scan line segment at lateral position x with heading angle φ

, is calculated as;

$I<br><br>\left(\mathrm{\phi <br><br>},x\right)=\frac{1}{N<br><br>\left(\mathrm{\phi <br><br>},x\right)}<br><br>\underset{y=y_{\min }}{\overset{y_{\max }}{\sum <br><br>}}<br><br>P<br><br>\left(x,y\right)$

where N(φ

, x) is the number of pixels on the scan line segment x, P(x,y) is the pixel'"'"'s image parameter value at coordinate (x,y), and wherein the evaluation is limited to scan line segments between y_min and y_max.

13. The method according to claim 1 wherein the identifying further comprises determining an image parameter variation or derivative of I(φ

, x) as a judging function, σ

(φ

) to search for the vehicle'"'"'s preferential heading angle;

$\mathrm{\sigma <br><br>}<br><br>\left(\mathrm{\phi <br><br>}\right)=\frac{1}{N_x}<br><br>\underset{x=-x_{\max }}{\overset{x_{\max }-1}{\sum <br><br>}}<br><br><br><br>I<br><br>\left(\mathrm{\phi <br><br>},x\right)-I<br><br>\left(\mathrm{\phi <br><br>},x+1\right)<br><br>$

where N_x is the number of scan lines.

14. The method according to claim 1 wherein the identifying further comprises averaging the image parameter variation or derivative of I(φ

, x) to produce a refined judging function, σ

(φ

,ω

) to search for the vehicle'"'"'s preferential heading angle;

$\mathrm{\sigma <br><br>}<br><br>\left(\mathrm{\phi <br><br>}\right)=\frac{1}{N_{\mathrm{\omega <br><br>}}}<br><br>\underset{\mathrm{\omega <br><br>}=-{\mathrm{\omega <br><br>}}_{\max }}{\overset{{\mathrm{\omega <br><br>}}_{\max }}{\sum <br><br>}}<br><br>\mathrm{\sigma <br><br>}<br><br>\left(\mathrm{\phi <br><br>},\mathrm{\omega <br><br>}\right)$

where N_w is the number of scan lines, and wherein ω

represents the pitch angle that ranges from a minimum pitch angle (−

ω

_max) to a maximum pitch angle (ω

_max).

15. The method according to claim 1 comprises controlling a path of the vehicle in accordance with the identified vehicle heading.

16. The method according to claim 1 further comprises:

determining an offset for the vehicle along the transverse axis.

17. The method according to claim 1 further comprises:

controlling a path of the vehicle in accordance with the identified vehicle heading and the determined offset.

18. A system of guiding a vehicle, the system comprising:

an imaging device for collecting color image data;

a discriminator for classifying image data as crop image data if a green-red difference associated with a pixel is greater than a first threshold value and if a green-blue difference associated with the pixel is greater than a second threshold value, wherein the green-red difference (D<sub>g−

r</sub>) is defined as D<sub>g−

r</sub>=G−

R and the green-blue difference (D<sub>g−

b</sub>) is defined as D<sub>g−

b</sub>=G−

B, where R,G,B are the pixel'"'"'s red, green and blue component, respectively, in RGB color space;

a definer for defining a series of scan line segments generally perpendicular to a transverse axis of the vehicle or of an imaging system;

an image parameter evaluator for determining a scan-line-image-parameter level for each of the scan line segments, the image parameter level comprising a color-related parameter; and

an alignment detector for identifying a preferential heading of the vehicle that is generally aligned with respect to a crop feature, associated with the collected image data, based on the determined scan-line-image-parameter level meeting or exceeding a maximum value or minimum threshold value.

19. The system according to claim 18 wherein image parameter level is associated with or derived from one or more of the following:

a color value of a red component, green component, or blue component in RGB color space;

a composite color value of multiple components in RGB color space, a hue value in the HSV color space;

a hue value in the HSL color space;

a saturation value associated with vibrancy or purity of a color, a saturation value in HSV or HSL color space, a value indicative of observed or reflected light in the green visible light spectrum or for another plant color;

a value indicative of observed or reflected light with certain green hue value or another plant color; and

a value indicative of observed or reflected light in multiple spectrums (e.g., green light and infra-red or near infra-red light), an NDVI (normalized difference vegetation index), a greenness level, a plant color level, and a green-red difference level.

20. The system according to claim 18 wherein the alignment detector searches candidate headings to select the preferential heading based on the determined scan line image parameter data.

21. The system according to claim 18 wherein the crop feature comprises one or more crop rows.

22. The system according to claim 18 wherein the crop feature comprises a crop edge between harvested and unharvested crop.

23. The system according to claim 18 further comprising:

an image segmenter for deriving a segmented image of crop rows as crop image data from the collected color image.

24. The system according to claim 18 wherein the discriminator classifies image data as soil data if a red-green difference associated with a pixel is less than or equal to first threshold value or if a green-blue difference associated with the pixel is less than or equal to the second threshold value.

25. The system according to claim 18 further comprising:

a vehicle guidance controller for controlling a path of the vehicle in accordance with the identified vehicle heading.

26. The system according to claim 18 further comprises:

an offset calculator for determining an offset for the vehicle along the transverse axis.

27. The system according to claim 18 further comprises:

a vehicle guidance controller for controlling a path of the vehicle in accordance with the identified vehicle heading and the determined offset.