System and method for detecting and analyzing features in an agricultural field for vehicle guidance

• US 8,855,405 B2
• Filed: 04/06/2004
• Issued: 10/07/2014
• Est. Priority Date: 04/30/2003
• Status: Active Grant

• Current Assignee: Deere & Company, University of Illinois
• Original Assignee: Deere & Company, University of Illinois
• Inventors: Mas, Francisco Rovira, Zhang, Qin, Reid, John Franklin
• Primary Examiner(s): Akhavannik, Hadi
• Assistant Examiner(s): RASHIDIAN, MOHAMMAD M
• Application Number:
  US10/817,906
• Publication Number:
  US 20040264763A1
• Time in Patent Office: 3,836 Days
• Field of Search: 382/154, 382/162, 382/110, 382/100, 382103-107, 382/115, 354419-427, 354/139, 354/437, 348 42- 43, 348143-161, 701/36, 701/200, 70/23, 70/36, 702/00, 703/1, 703/6, 703/8, 345419-427, 356 12- 22, 356419-427, 700/1
• US Class Current: 382/154
• CPC Class Codes: G01C 23/00 Combined instruments indicating more than one navigational value, e.g. for aircraft; Combined measuring devices for measuring two or more variables of movement, e.g. distance, speed or acceleration
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30128 Food products
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

A system and method for creating 3-dimensional agricultural field scene maps are disclosed comprising producing a pair of images using a stereo camera and creating a disparity image based on the pair of images, the disparity image being a 3-dimensional representation of the stereo images. Coordinate arrays can be produced from the disparity image and the coordinate arrays can be used to render a 3-dimensional local map of the agricultural field scene. Global maps can also be made by using geographic location information associated with various local maps to fuse together multiple local maps into a 3-dimensional global representation of the field scene. The disparity images can be used to analyze a variety of agricultural features.

Claims

1. An apparatus for using data from a 3-dimensional field scene map to aid in the navigation of a vehicle, comprising:

at least one camera for producing a pair of images of a field scene;

means for creating a disparity image from the pair of images and analyzing features within the disparity image, wherein the creating comprises determining at least one of a camera to ground distance and a camera tilt angle; and

means for operatively connecting the creating means to a navigation system of a vehicle, wherein information gathered from the at least one camera is transmitted to the navigation system for use in guiding the vehicle.

2. The apparatus of claim 1, wherein the at least one camera comprises a stereo camera.

3. The apparatus of claim 1, wherein the field scene is an agricultural field scene of a plurality of rows of individual plants.

4. The apparatus of claim 3, wherein the disparity image is analyzed to determine a distance between individual rows of plants.

5. The apparatus of claim 3, wherein the disparity image is analyzed to determine a distance between individual plants within the same row of plants.

6. The apparatus of claim 3, wherein the disparity image is analyzed to determine the height of individual plants.

7. The apparatus of claim 3 wherein the disparity images is analyzed to identify agricultural infrastructure other than individual plants.

8. The apparatus of claim 1, wherein the means for creating a disparity image comprises a computer running a software program.

9. The apparatus of claim 1, wherein multiple 3-dimensional local maps of the field scene are produced from generated disparity images, and further comprising means for fusing the multiple 3-dimensional local maps into a 3-dimensional global image.

10. The apparatus of claim 1, wherein the at least one camera is programmed to create a three-dimensional representation of the field scene, and wherein the three-dimensional representation is divided into a plurality of layers substantially parallel to the ground, and the plurality of layers are analyzed to determine a path to be followed by the vehicle.

11. The apparatus of claim 1, further comprising a location tracking device configured for providing geographic location information.

12. The apparatus of claim 1, wherein the disparity image is segmented to separate items of interest from background information, and wherein the disparity image is also used to determine a target location in the disparity image for the vehicle.

13. The apparatus of claim 12, wherein the disparity image is divided into multiple regions of interest.

14. The apparatus of claim 13, wherein a linear regression analysis is used to reduce the number of pixels identified in the multiple regions of interest.

15. The apparatus of claim 14, wherein data generated from the linear regression analysis is used to define a central path for vehicle navigation.

16. A method guiding a vehicle by making and analyzing a 3-dimensional map of a field scene, the method comprising the steps of:

acquiring a set of images of the field scene using one or more cameras;

creating a disparity image based on the at least one set of images, wherein;

the disparity image including depth information for objects captured in the at least one set of images; and

the creating comprises determining at least one of a camera to ground distance and a camera tilt angle;

analyzing the disparity image to generate information concerning at least one feature captured in the at least one set of images;

transmitting the information to navigation system of a vehicle.

17. The method of claim 16, wherein the at least one camera comprises a stereo camera.

18. The method of claim 17, wherein the disparity image is segmented to separate items of interest from background information, and wherein the disparity image is also used to determine a target location in the disparity image for the vehicle.

19. The method of claim 18, wherein the disparity image is divided into multiple regions of interest.

20. The method of claim 19, wherein a linear regression analysis is used to reduce the number of pixels identified in the multiple regions of interest.

21. The method of claim 20, wherein data generated from the linear regression analysis is used to define a central path for vehicle navigation.

22. The method of claim 16, wherein the image is used to create a three-dimensional representation of the field scene, and wherein the three-dimensional representation is divided into a plurality of layers substantially parallel to the ground, and the plurality of layers are analyzed to determine a path to be followed by the vehicle.

23. The method of claim 16, wherein the field scene is an agricultural field scene of a plurality of rows of individual plants.

24. The method of claim 16, wherein the disparity image is analyzed to determine a distance between individual rows of plants.

25. The method of claim 16, wherein the disparity image is analyzed to determine a distance between individual plants within the same row of plants.

26. The method of claim 16, wherein the disparity image is analyzed to determine the height of individual plants.

27. The method of claim 16 wherein the disparity images is analyzed to identify agricultural infrastructure other than individual plants.

28. The method of claim 16, further comprising the step of filtering the 3-dimensional coordinates to remove noise.

29. A system for creating and analyzing a 3-dimensional agricultural field scene map, comprising:

a stereo camera for producing a pair of stereo images of an agricultural field scene;

means for creating a disparity image from the pair of stereo images and analyzing features in the disparity image;

wherein;

the disparity image including a three-dimensional image encoded with x, y and z coordinates relative to the stereo camera; and

the creating comprises determining at least one of a camera to ground distance and a camera tilt angle; and

means for providing information gathered from the disparity image to a vehicle navigation system for use in guiding the vehicle.

30. The system of claim 29, wherein the disparity image is segmented to separate items of interest from background information, and wherein the disparity image is also used to determine a target location in the disparity image for the vehicle.

31. The system of claim 30, wherein the disparity image is divided into multiple regions of interest, and wherein a linear regression analysis is used to reduce the number of pixels identified in the multiple regions of interest.

32. The method of claim 31, wherein data generated from the linear regression analysis is used to define a central path for vehicle navigation.

33. The system of claim 29, wherein the image is used to create a three-dimensional representation of the field scene, and wherein the three-dimensional representation is divided into a plurality of layers substantially parallel to the ground, and the plurality of layers are analyzed to determine a path to be followed by the vehicle.

34. The system of claim 29, wherein the disparity image is analyzed to determine a distance between individual rows of plants.

35. The system of claim 29, wherein the disparity image is analyzed to determine a distance between individual plants.

36. The system of claim 29 wherein the disparity image is analyzed to identify agricultural infrastructure other than individual plants.

37. The system of claim 29, wherein the means for creating a disparity image and analyzing features in the disparity image comprises a computer running a software program.

38. The system of claim 29, wherein the means for providing information to the vehicle navigation system comprises a computer running a software program.