Method and stereo vision system for facilitating the unloading of agricultural material from a vehicle

• US 8,868,304 B2
• Filed: 02/08/2013
• Issued: 10/21/2014
• Est. Priority Date: 02/10/2012
• Status: Active Grant

• Current Assignee: Deere & Company
• Original Assignee: Deere & Company
• Inventors: Bonefas, Zachary T.
• Primary Examiner(s): Black, Thomas G
• Assistant Examiner(s): NELSON, SARA J
• Application Number:
  US13/762,508
• Publication Number:
  US 20130211658A1
• Time in Patent Office: 620 Days
• Field of Search: 701/50, 701/117, 701/408, 701/534, 701/300, 701/28, 701/41, 701/49
• US Class Current: 701/50
• CPC Class Codes: A01D 41/1217 Unloading mechanisms
• CPC Class Codes: A01D 43/087 with controllable discharge spout
• CPC Class Codes: B65G 67/24 Unloading land vehicles
• CPC Class Codes: G05D 1/0251 extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593

Abstract

A mode controller determines whether to use an automated control mode of the spout or operator-directed manual control mode of the spout, based on a first operational status of a first location determining receiver associated with the propelling vehicle, a second operational status of a second location determining receiver associated with the harvesting vehicle, a third operational status of the imaging device or devices. In the automated control mode, an image processing module is adapted to facilitate the determination of the relative position of the spout and the storage portion and to generate command data place the storage portion and spout in relative cooperative alignment for transferring of material into the storage portion.

Claims

1. A system for facilitating the transfer of agricultural material from a transferring vehicle to a receiving vehicle, the system comprising:

a receiving vehicle comprising a propelled portion for propelling the receiving vehicle and a storage portion for storing agricultural material;

a transferring vehicle having a spout for directing agricultural material toward the storage portion of the receiving vehicle;

a first location-determining receiver on or associated with the transferring vehicle;

a second location-determining receiver on or associated with the receiving vehicle;

a first imaging device facing towards the storage portion of the receiving vehicle, the first imaging device collecting first image data;

a mode controller for determining whether to use an automated control mode of the spout position with respect to the storage portion or an operator-directed manual control mode of the spout based on a first operational status of the first location-determining receiver, a second operational status of the second location-determining receiver, and a third operational status of the first imaging device, an image processing module, or both; and

the mode controller adapted to authorize operation in the automated control mode if the first, second and third operational statuses are acceptable, the automated control mode comprising an operational mode where an image processing module processes the collected image data to facilitate determination of the relative position of the spout and the storage portion and where the image processing module generates command data

to place the storage portion and spout in relative cooperative alignment such that the spout is aligned for transferring of material into storage portion.

2. The system according to claim 1 wherein the command data may comprise one or more of the following:

steering command data for the receiving vehicle, steering command data for the transferring vehicle, or actuator command data for rotating a spout end of a spout of the transferring vehicle.

3. The system according to claim 1 wherein the first operational status is acceptable if the first location determining receiver provides reliable position data that meets or exceeds a dilution of precision threshold during a sampling period, wherein the second operational status is acceptable if the second location determining receiver provides reliable position data that meets or exceeds a dilution of precision threshold during a sampling period, and wherein the third operational status is acceptable if the first imaging device provides reliable image data in which the image processing module is capable of reliably identifying one or more edges of the spout, a container perimeter of the storage portion, or both in the collected image data during a sampling time period.

4. The system according to claim 1 wherein the first operational status is acceptable if the first location-determining receiver provides reliable position data that meets or exceeds a dilution of precision threshold during a sampling period, wherein the second operational status is acceptable if the second location-determining receiver provides reliable position data that meets or exceeds a dilution of precision threshold during a sampling period, and wherein the third operational status is acceptable if the image processing module is capable of reliably identifying one or more edges of a spout end of the spout and one or more edges of the container perimeter of the storage portion.

5. The system according to claim 4 wherein the first location-determining receiver provides a first indicator message that indicates that the first location-determining receiver meets or exceeds a precision dilution threshold, wherein the second location-determining receiver provides a second indicator message that indicates that the second location-determining receiving exceeds a precision dilution threshold, and wherein the image processing module provides a third indicator message that indicates that the image processing module is capable of reliably identifying one or more edges of the spout end and one or more edges of the container perimeter of the storage portion during a sampling period.

6. The system according to claim 1 wherein the mode controller is adapted to authorize operation in a semi-automated control mode if the first and second operational statuses are acceptable, the semi-automated control mode comprising an operational mode where a master controller coordinates the movement of the vehicles from motion data provided by the first location-determining receiver and the second location-determining receiver to place the vehicles in general alignment along substantially parallel paths such that an operator of the transferring vehicle can manually adjust or fine tune the relative position and velocity of the transferring vehicle to the receiving vehicle or the relative position of the spout with respect to the receiving vehicle for transferring of material into the storage portion, where the semi-automated mode does not use image data of the image processing module for any alignment of the spout or the vehicles.

7. The system according to claim 1 further comprising:

a steering controller associated with a steering system of the propelled portion for steering the receiving vehicle in accordance with the cooperative alignment based on location data and motion data from the first location-determining receiver and the second location-determining receiver in the automated control mode.

8. The system according to claim 1 further comprising:

a steering controller associated with a steering system of the transferring vehicle for steering the transferring vehicle in accordance with the cooperative alignment based on location data and motion data from the first location-determining receiver and the second location determining receiver in the automated control mode.

9. The system according to claim 1 further comprising:

an actuator for actuating or rotating the spout with respect to the transferring vehicle to place the spout and storage portion in cooperative alignment based on the collected image data.

10. The system according to claim 1 wherein if the first, second and third operational statuses are not acceptable, the system reverts to manual mode control for control of the spout and storage-portion alignment.

11. The system according to claim 1 wherein the first imaging device is mounted on the receiving vehicle.

12. The system according to claim 1 wherein the first imaging device is mounted on the transferring vehicle.

13. The system according to claim 1 wherein the mode controller uses the first location-determining receiver, the second location-determining receiver and the imaging device to track relative position of the receiving vehicle and the transferring vehicle if the first operational status, the second operational status and the third operational status are acceptable or indicative of properly functioning.

14. The system according to claim 1 further comprising:

a first wireless communications device on the transferring vehicle;

a second wireless communications device on the receiving vehicle;

the mode controller using the first location-determining receiver and the second location-determining receiver to track relative position of the receiving vehicle and the transferring vehicle if the first operational status and the second operation status are acceptable and if the third operational status is unacceptable;

wherein the first wireless communications device wirelessly transmits first location data and first motion data from the first location determining receiver to the second wireless communications device.

15. The system according to claim 1 further comprising:

an odometer for providing odometry data;

an inertial sensor for providing inertial data, wherein the mode controller uses the image data, odometry data and inertial data if the first operational status and the second operational status are unacceptable and if the third operational status is acceptable.

16. The system according to claim 15 wherein the steering controller operates in a partially automated mode in which an operator supervises the steering of the transferring vehicle and the receiving vehicle;

where the operator can override the steering control based on the odometry and inertial data.

17. A method for facilitating the transfer of agricultural material from a transferring vehicle to a receiving vehicle, the transferring vehicle having a spout for transferring the agricultural material to the receiving vehicle, the method comprising:

collecting first image data by a first imaging device facing towards a storage portion of the receiving vehicle;

determining whether to use an automated control mode of the spout or an operator-directed manual control mode of the spout based on a first operational status of a first location determining receiver associated with the transferring vehicle, a second operational status of a second location determining receiver associated with the receiving vehicle, and a third operational status of the first imaging device; and

in the automated control mode, determining the relative position of the spout and the storage portion; and

generating command data to place the spout and storage portion in relative cooperative alignment in the automated control mode such that the spout is aligned with the storage portion if the first, second and third operational statuses are acceptable.

18. The system according to claim 17 wherein the first operational status is acceptable if the first location determining receiver provides reliable position data that meets or exceeds a dilution of precision threshold during a sampling period, wherein the second operational status is acceptable if the second location determining receiver provides reliable position data that meets or exceeds a dilution of precision threshold during a sampling period, and wherein the third operational status is acceptable if the first imaging device provides reliable image data in which the container module or spout module are capable of reliably identifying one or more edges of the spout or a container perimeter of the storage portion in the collected image data during a sampling time period.

19. The method according to claim 17 further comprising:

steering the receiving vehicle, by a steering controller, in accordance with the cooperative alignment based on location data and motion data from the first location determining receiver and the second location determining receiver in the automated control mode.

20. The method according to claim 17 further comprising:

a steering the transferring vehicle, by a steering controller, in accordance with the cooperative alignment based on location data and motion data from the first location determining receiver and the second location determining receiver in the automated control mode.

21. The system according to claim 17 further comprising:

actuating or rotating the spout with respect to the transferring vehicle to place the spout and storage portion in cooperative alignment.

22. The method according to claim 17 wherein if the first, second and third operational statuses are not acceptable, the method reverts to the manual mode control for control of the spout and storage-portion alignment.

23. The method according to claim 17 wherein the first imaging device is mounted on the receiving vehicle.

24. The method according to claim 18 further comprising:

tracking, by the first location determining receiver, the second location determining receiver and the imaging device, relative position of the receiving vehicle and the transferring vehicle if the first operational status, the second operational status and the third operational status are acceptable or indicative of proper functionality.

25. The method according to claim 17 further comprising:

tracking, by the first location determining receiver and the second location determining receiver, relative position of the receiving vehicle and the transferring vehicle if the first operational status and the second operation status are acceptable and if the third operational status is unacceptable, wherein first location and first motion data is wirelessly transmitted from the transferring vehicle to the receiving vehicle to determine the relative position.

26. The method according to claim 17 further comprising:

estimating, via first image data, odometry data and inertial data, the relative position of the receiving vehicle and the transferring vehicle if the first operational status and the second operational status are unacceptable and if the third operational status is acceptable.

27. The method according to claim 26 further comprising:

operating the transferring vehicle and the receiving vehicle in a partially automated mode in which an operator supervises the steering of the transferring vehicle and the receiving vehicle;

where the operator can override the steering control based on the odometry and inertial data.