Software architecture for autonomous earthmoving machinery

• US 6,223,110 B1
• Filed: 10/14/1998
• Issued: 04/24/2001
• Est. Priority Date: 12/19/1997
• Status: Expired due to Term

• Current Assignee: Carnegie Mellon University
• Original Assignee: Carnegie Mellon University
• Inventors: Pedersen, Jorgen, Stentz, Anthony, Rowe, Patrick
• Primary Examiner(s): LOUIS JACQUES, JACQUES H
• Application Number:
  US09/172,420
• Time in Patent Office: 923 Days
• Field of Search: 701/50, 250/202, 239/1, 239/11, 239/155, 372/31, 372/32, 373/48, 180/168, 340/901, 340/903, 340/988, 340/994, 340/686.1, 340/687, 340/679, 340/684, 111/178, 111/903, 111/170, 111/177, 702/2, 702/5, 56/102.R, 561/02.A, 561/02.B, 56/102.H, 56/102.G, 56/102.F, 172/2, 172/7, 172/10
• US Class Current: 701/50
• CPC Class Codes: E02F 3/435 for dipper-arms, backhoes or the like
• CPC Class Codes: E02F 9/2045 Guiding machines along a predetermined path for graders E02F3/841; machines for construction of roads E01C19/004

Abstract

In accordance with the present invention, a modular architecture to organize and coordinate components that are needed to automate earthmoving tasks, and to coordinate the flow of data between the components is disclosed. The architecture includes three main subdivisions: a sensor pipeline, sensor data consumers, and motion planners and executors. The sensor pipeline receives raw sensor data from perceptual sensors such as a laser rangefinder or radar system, and converts the data into a form which is usable by the other system components. Sensor data can also be represented in the form of an elevation map of the surrounding terrain for other software components to use. Any number and types of sensor systems may be added to the software architecture depending on requirements and the capabilities of the system. The sensor data consumers use the sensor data as input to specific algorithms to produce information regarding the machine'"'"'s environment for use by other system components. A motion planner receives information provided by the sensor data consumers, and delivers output commands go to controllers on the machine. The motion planner also computes and delivers commands to the sensor systems on the machine. Additional planners may be added at this level to coordinate other system behaviors and actions.

Claims

1. A software architecture for autonomous control of earthmoving machinery comprising:

at least one sensor system operable to provide data regarding pertinent regions of the environment of the earthmoving machinery;

a sensor pipeline operable to receive data from the at least one sensor system and to distribute the data to components in the software architecture;

at least one sensor data consumer operable to plan an earthmoving task using the data including recognizing objects in the environment, selecting a location to excavate, and selecting a location to place the excavated material; and

at least one motion planner operable to generate commands to independently move components of each of the at least one sensor system and the machinery to concurrently plan and execute phases of the earthmoving task.

2. The software architecture as set forth in claim 1 wherein the sensor pipeline further comprises a terrain map server, the terrain map server being operable to receive at least a portion of the data, to process at least a portion of the received data to generate a terrain map of at least a portion of the earthmoving environment, and to distribute at least a portion of the terrain map upon request to at least one component in the software architecture.

3. The software architecture as set forth in claim 2 wherein the sensor pipeline includes at least one sensor interface operable to receive data from the at least one sensor system and to transmit commands for controlling movement of the at least one sensor system to the at least one sensor system.

4. The software architecture as set forth in claim 3 wherein the sensor pipeline includes at least one scan line processor operable to receive data from the at least one sensor interface, to transform the data from one coordinate system to another, and to transmit the transformed data to the terrain map server.

5. The software architecture as set forth in claim 4 further comprising:

a position system operable to transmit data regarding the position of the earthmoving machine to the at least one scan line processor.

6. The software architecture as set forth in claim 3 wherein the motion planner further comprises at least one sub-task, the at least one sub-task being operable to receive data from the at least one sensor interface and to transmit commands to the at least one sensor interface to control the motion of the at least one sensor system.

7. The software architecture as set forth in claim 2 wherein the at least one sensor data consumer is operable to use terrain map data to determine a location for executing the earthmoving task.

8. The software architecture as set forth in claim 2 wherein the at least one sensor data consumer is operable to use terrain map data to determine a location for unloading excavated material collected by the earthmoving machinery.

9. The software architecture as set forth in claim 2 wherein the at least one sensor data consumer is operable to use terrain map data to determine if an object is present in the earthmoving environment.

10. The software architecture as set forth in claim 2 further comprising a machine controller interface operable to receive a command from the motion planner and data from a machine controller, and to transmit the command to the machine controller and the data to the motion planner.

11. The software architecture as set forth in claim 10 wherein the motion planner is further operable to receive data from the at least one sensor data consumer and to transmit commands to the machine controller interface to control the motion of the earthmoving machine.

12. The software architecture as set forth in claim 11 further comprising:

an obstacle detection planner being operable to receive data from the terrain map server, the machine controller interface, and the motion planner, the obstacle detection planner being further operable to transmit data to the machine controller interface and the motion planner.

13. The software architecture as set forth in claim 1 wherein the motion planner uses operational constraints of the earthmoving machinery to generate commands to move components of the machinery to perform the earthmoving task.

14. The software architecture as set forth in claim 1 wherein the motion planner uses geographic constraints associated with the earthmoving environment to generate commands to move components of the earthmoving machinery to perform the earthmoving task.

15. A software architecture for autonomous control of earthmoving machinery including at least one sensor system comprising:

a motion planner including a plurality of sub-tasks, the sub-tasks including at least a sensor motion planner, an earthmoving motion planner, and a loading motion planner, the sub-tasks being operable to pre-plan and coordinate phases of an earthmoving task and to generate commands to independently move components of each of the at least one sensor system and the earthmoving machinery to perform the earthmoving task.

16. The software architecture as set forth in claim 15 further comprising:

an obstacle detection planner operable to coordinate with the plurality of sub-tasks to control the earthmoving machinery to prevent interference with any obstacles detected in an earthmoving environment.

17. A software architecture for autonomous control of earthmoving machinery comprising:

at least one sensor system operable to provide data regarding pertinent regions of the environment of the earthmoving machinery;

a sensor pipeline operable to receive data from the at least one sensor system and to distribute the data to components in the software architecture;

at least one sensor data consumer operable to plan an earthmoving task using the data including recognizing objects in the environment, selecting a location to excavate, and selecting a location to place the excavated material; and

at least one motion planner operable to generate commands to independently move components of each of the at least one sensor system and the machinery to concurrently plan and execute phases of the earthmoving task;

said sensor pipeline including at least one sensor interface operable to receive data from said at least one sensor system and to transmit commands for controlling movement of said at least one sensor system to said at least one sensor system.