Obstacle detection using stereo vision

• US 7,248,968 B2
• Filed: 10/29/2004
• Issued: 07/24/2007
• Est. Priority Date: 10/29/2004
• Status: Active Grant

• Current Assignee: Deere & Company
• Original Assignee: Deere & Company
• Inventors: Reid, John Franklin
• Primary Examiner(s): Tran; Dalena
• Application Number:
  US10/977,675
• Publication Number:
  US 20060095207A1
• Time in Patent Office: 998 Days
• Field of Search: 701/301, 701/116, 701/117, 340/435, 340/436, 340/901, 340/903, 340/461, 340/438, 340/439, 340/462, 348/118, 348/148
• US Class Current: 701/301
• CPC Class Codes: G01S 11/12 using electromagnetic waves other than radio waves
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/188 Vegetation
• CPC Class Codes: G06V 20/38 Outdoor scenes

Abstract

A system and method for detecting an obstacle comprises determining a vegetation height of a vegetation canopy at corresponding coordinates within a field. An object height is detected, where the object height of an object exceeds the vegetation height. A range of the object is estimated. The range of the object may be references to a vehicle location, a reference location, or absolute coordinates. Dimensions associated with the object are estimated. An obstacle avoidance zone is established for the vehicle based on the estimated range and dimensions.

Claims

1. A method for detecting an obstacle, the method comprising:

identifying an object in a field by analyzing one or more stereo images in color space;

determining a vegetation height of a cultivated vegetation canopy at corresponding coordinates within the field;

detecting an object height of the object to identify a screened object having an object height exceeding a vegetation height of the cultivated vegetation canopy;

estimating a range associated with the screened object with respect to a vehicle;

estimating dimensions associated with the screened object; and

establishing an obstacle avoidance zone for the vehicle based on the estimated range and dimensions.

2. The method according to claim 1 further comprising:

classifying the screened object based on the dimensions and compliance with a dimension reference profile.

3. The method according to claim 2 further wherein the dimension reference profile comprises one or more of the following:

a height range and width range for a front human profile, a height range and width range for a side human profile, a height and width range for a front animal profile, a height and width range for a side animal profile, a height and width range for front of a machine, a height and width range for a side of a machine, a height and width range for a tree, a dimension range for a corresponding front outline of human, a dimension range for a corresponding side outline of a human, a dimension range for a corresponding front outline of an animal, dimension range for a front outline of a machine, dimension range for a side outline of a machine, and dimension range for a tree, a three dimensional model human profile, a three dimensional model animal profile, a three dimensional model tree profile, and a two-dimensional representation of an object associated with a uniform depth zone of a three dimensional representation of the object.

4. The method according to claim 1 wherein the identifying of the object first analyzes a zone associated with a candidate path plan and secondarily analyzes a remainder of a field of view of the image.

5. The method according to claim 1 further comprising:

collecting image data associated with at least one of the field and the object;

deriving a derivative data set of image data based on the collected image data;

establishing reference image data associated with a reference image representative of the field;

deriving a reference derivative data set based on the established reference image data;

comparing the derivative data set to the reference derivative data set to determine a region of the collected image data that is distinct from the reference image and indicative of the potential presence of an object in the field.

6. The method according to claim 5 wherein the reference image data is collected in an obstacle-free portion of a field.

7. The method according to claim 6 wherein establishing of the reference image comprises collecting the reference image for a particular crop at a particular growth stage with defined illumination characteristics substantially similar to those applicable to the field.

8. The method according to claim 1 wherein the determining of the vegetation height comprises determining the vegetation height of a crop row canopy as the vegetation canopy.

9. The method according to claim 1 wherein the determining of the vegetation height comprises determining the vegetation height of a weed canopy as a vegetation canopy.

10. The method according to claim 1 wherein estimating of the range is based on depth data of the object, where the depth data is derived from at least one of stereo vision data and a disparity map.

11. The method according to claim 1 wherein the estimating of the range and the dimensions comprises:

gathering a pair of raw stereo scene images of the object from spatially offset perspectives;

combining the raw stereo scene images to produce a disparity map image that represents disparity between the raw stereo scene images; and

creating a three dimensional representation of the object based on at least one of the disparity map image and the raw stereo scene images.

12. The method according to claim 1 wherein the estimating of the dimensions is based on the estimated range and at least one stereo image.

13. The method according to claim 1 further comprising:

determining a corresponding position and contour of crop rows defining the vegetation canopy of a field.

14. The method according to claim 13 further comprising:

establishing a path plan for a vehicle to travel in one or more travel rows interspersed among the crop rows.

15. The method according to claim 1 wherein the determining of the vegetation height of the vegetation canopy further comprises determining a corresponding position and contour of crop rows in the field.

16. A system for detecting an obstacle, the system comprising:

a row detection module for determining a vegetation height of a cultivated vegetation canopy at corresponding coordinates within a field;

an obstacle detection module for detecting a screened object with an object height that exceeds the vegetation height of the cultivated vegetation canopy;

an object estimator for estimating a range associated with the screened object with respect to a vehicle and estimating dimensions associated with the object via one or more stereo images; and

a vehicular guidance system for establishing an obstacle avoidance zone for the vehicle based on the estimated range and dimensions.

17. The system according to claim 16 further comprising:

a left imaging unit and right imaging unit for gathering a pair of raw stereo scene images of the object from spatially offset perspectives;

a disparity map module for forming a disparity map image that represents disparity between the raw stereo scene images; and

a three dimensional image former for creating a three dimensional representation of the object based on the disparity map image and at least one of the raw stereo scene images.

18. The system according to claim 17 wherein the obstacle detection module is arranged to first analyzes a zone associated with a candidate path plan and secondarily analyze a remainder of a field of view of the left imaging unit and the right imaging unit.

19. The system according to claim 16 further comprising:

a classifier for classifying the object based on the dimensions and compliance with a dimension reference profile.

20. The system according to claim 19 further wherein the dimension reference profile comprises one or more of the following:

a height range and width range for a front human profile, a height range and width range for a side human profile, a height and width range for a front animal profile, a height and width range for a side animal profile, a height and width range for front of a machine, a height and width range for a side of a machine, a height and width range for a tree, a dimension range for a corresponding front outline of human, a dimension range for a corresponding side outline of a human, a dimension range for a corresponding front outline of an animal, dimension range for a front outline of a machine, dimension range for a side outline of a machine, and dimension range for a tree, a three dimensional model human profile, a three dimensional model animal profile, a three dimensional model tree profile, and a two-dimensional representation of an object associated with a uniform depth zone of a three dimensional representation of the object.

21. The system according to claim 16 wherein the obstacle detection module compares collected derivative data set of image data to a reference derivative data set to determine a region of the collected image data that is distinct from the reference image and indicative of the potential presence of an object.

22. The system according to claim 21 wherein the reference image is collected in an obstacle-free portion of a field.

23. The system according to claim 16 wherein the vegetation height comprises a height of a crop row canopy as the vegetation canopy.

24. The system according to claim 16 wherein the vegetation height comprises a height of a weed canopy as the vegetation canopy.

25. The system according to claim 16 wherein the range is based on depth data of the object, where the depth data is derived from at least one of stereo vision data and a disparity map.

26. The system according to claim 25 further comprising:

a crop row detection module for determining a corresponding position and contour of crop rows of the field based on the disparity map.

27. The system according to claim 16 wherein the object estimator estimates the dimensions and range ob art abject based on the estimated range and at least one stereo image.

28. The system according to claim 27 further comprising:

a path planner for establishing a path plan for a vehicle consistent with the obstacle avoidance zone.