Simulation system implementing real-time machine data

• US 8,139,108 B2
• Filed: 01/31/2007
• Issued: 03/20/2012
• Est. Priority Date: 01/31/2007
• Status: Active Grant

• Current Assignee: Caterpillar Incorporated
• Original Assignee: Caterpillar Incorporated
• Inventors: Stratton, Kenneth L., Shults, Jamie, Clar, Jean-Jacques, Darby, Derrick, Opdenbosch, Augusto J., Santamaria, Juan Carlos
• Primary Examiner(s): Walsh, John B.
• Application Number:
  US11/700,015
• Publication Number:
  US 20080180523A1
• Time in Patent Office: 1,875 Days
• Field of Search: None
• US Class Current: 348/114
• CPC Class Codes: G05D 1/0044 by providing the operator with a computer generated representation of the environment of the vehicle, e.g. virtual reality, maps maps used for automatic navigation G05D1/0274; flight directors G01C23/005
• CPC Class Codes: G09B 9/048 a model being viewed and manoeuvred from a remote point G09B9/052, G09B9/058 take precedence

Abstract

A simulation and control system for a machine is disclosed. The simulation and control system may have a user interface configured to display a simulated environment. The machine simulation and control system may also have a controller in communication with the user interface and a remotely located machine. The controller may be configured to receive from the machine real-time information related to operation of the machine at a worksite. The controller may also be configured to simulate the worksite, operation of the machine, and movement of a machine tool based on the received information. The controller may further be configured to provide to the user interface the simulated worksite, operation, and movement in the simulated environment.

Claims

1. A machine simulation and control system, comprising:

a user interface configured to display a simulated environment; and

a controller in communication with the user interface and a remotely located machine, the controller being configured to;

receive, from the machine, real-time information related to operation of the machine at a site, the received information including at least a position of an actuator associated with a tool of the machine;

simulate the site, operation of the machine, and movement and positioning of the machine tool based on the received information; and

provide to the user interface the simulated site, operation, and movement and positioning of the machine tool in the simulated environment.

2. The machine simulation and control system of claim 1, wherein the simulated environment is in three dimensions.

3. The machine simulation and control system of claim 1, wherein the simulated environment includes a plurality of selectable onboard and offboard view reference locations.

4. The machine simulation and control system of claim 1, wherein the simulated environment further includes a display of at least one parameter value included in the real-time information.

5. The machine simulation and control system of claim 1, wherein the real-time information further includes a machine position parameter, and the controller is further configured to indicate in the simulated environment a position of the machine on the site based on the machine position parameter.

6. The machine simulation and control system of claim 5, wherein the real-time information further includes at least one of a travel speed of the machine, a steering command of the machine, or a load parameter indicating a load on the tool, and the controller is further configured to indicate in the simulated environment at least one of the travel speed of the machine, the steering command, or the load on the tool, respectively, based on the real-time information.

7. The machine simulation and control system of claim 1, wherein the real-time information further includes roll and pitch parameters, and the controller is further configured to indicate in the simulated environment a roll and pitch of the machine based on the roll and pitch parameters.

8. The machine simulation and control system of claim 1, wherein the real-time information further includes at least one of a slip parameter indicating a slip associated with a traction device of the machine, a transmission state, or an engine speed, and the controller is further configured to indicate in the simulated environment at least one of the slip, the transmission state, or the engine speed, respectively, based on the real-time information.

9. The machine simulation and control system of claim 1, wherein:

the user interface is further configured to receive real-time machine operation instructions from a user; and

the controller is further configured to communicate the instructions to the machine for remote control of the machine.

10. A method for remotely controlling a machine on a site from a remote location, comprising:

monitoring machine operation, including monitoring a position of an actuator of a tool associated with the machine;

simulating in real-time the site, machine movement within the site, machine tool position and movement within the site based on the monitored machine operation;

providing the simulation via an operator interface at the remote location, the simulation indicating at least the machine tool position and movement;

receiving machine control instructions via the operator interface; and

affecting operation of the machine based on the received instructions.

11. The method of claim 10, wherein providing the simulation includes displaying at the remote location the simulated site, machine movement, and machine tool movement in three dimensions from a plurality of selectable onboard and offboard view reference locations.

12. The method of claim 10, wherein monitoring machine operation includes determining a global position of the machine, and the method further includes providing in the simulation an indication of a position of the machine on the site based on the global position of the machine.

13. The method of claim 12, wherein monitoring machine operation includes detecting an engine operating condition, and the method further includes providing in the simulation an indication of the detected engine operating condition.

14. The method of claim 12, wherein monitoring machine operation further includes monitoring at least one of a machine travel speed, a steering angle, or a load on the machine tool, and the method further includes providing in the simulation an indication of the at least one of the machine travel speed, the steering angle, or the load on the machine tool.

15. The method of claim 12, wherein simulating includes determining the position and movement of the machine tool based on the monitored position of the actuator.

16. A system for simulating machine operation at a site, comprising:

a plurality of sensing elements for producing one or more signals indicative of real-time information related to operation of the machine and operation of a machine tool, the real-time information including positioning of an actuator associated with the machine tool;

a machine data module located on the machine to collect and transmit the one or more signals offboard the machine;

a user interface configured to receive operational instructions from an offboard user; and

an offboard controller in communication with the machine data module and the user interface, the offboard controller being configured to;

receive the one or more signals;

simulate in real-time a three dimensional machine environment based on the one or more signals, the simulated environment including the site, the machine operation, and the machine tool operation;

provide to the user interface for display the simulated machine environment; and

transmit to the machine the operational instructions for remote control of the machine and machine tool operations.

17. The system of claim 16, wherein the simulated environment includes a plurality of selectable onboard and offboard view reference locations.

18. The system of claim 16, wherein the simulated environment further includes a display of at least one parameter value included in the one or more signals.

19. The system of claim 16, wherein the real-time information further includes at least one of a machine position parameter, a travel speed, a steering command, or a load parameter indicating a load on the machine tool, and the controller is further configured to provide in the simulated environment an indication of the at least one of a position of the machine on the site, the steering command, or the load on the machine tool, based on the real-time information.

20. A method for remotely controlling a machine on a site, comprising:

receiving, at a location remote to the machine, real-time information relating to operation of the machine on the site, the real-time information including loading of a tool associated with the machine;

providing, by simulation system controller, a simulation of the operation of the machine on the site based on the received real-time information and a terrain map associated with the site; and

in the simulation, providing an indication of the loading of the tool.

21. The method of claim 20, further comprising:

receiving machine control instructions at the location remote from the machine; and

enabling control of operation of the machine based on the received instructions.

22. The method of claim 20, wherein the received real-time information further includes at least one of an engine speed, ground speed, steering angle, transmission state, or pitch and roll of the machine, and the method further includes providing, in the simulation, an indication of the at least one of the engine speed, ground speed, steering angle, transmission state, or pitch and roll of the machine.

23. The method of claim 20, wherein the received real-time information further includes positioning information related to a positioning of an actuator of the tool, and the method further includes providing, in the simulation, an indication of a positioning of the tool based on the actuator positioning information.

24. A simulation system for controlling a machine on a site remote from the simulation system, comprising:

an operator interface for presenting a simulated environment; and

a simulation controller configured to;

receive real-time information relating to operation of the machine on the site, the real-time information including loading of a tool associated with the machine;

provide, via the operator interface, a simulation of the operation of the machine on the site based on the received real-time information and a terrain map associated with the site; and

in the simulation, provide an indication of the loading of the tool.

25. The system of claim 24, the simulation controller being further configured to:

receive, via the operator interface, machine control instructions; and

enable control of operation of the machine based on the received instructions.

26. The system of claim 24, wherein the received real-time information further includes at least one of an engine speed, ground speed, steering angle, transmission state, or pitch and roll of the machine, and the simulation controller is further configured to provide, in the simulation, an indication of the at least one of the engine speed, ground speed, steering angle, transmission state, or pitch and roll of the machine.

27. The system of claim 24, wherein the received real-time information further includes positioning information related to a positioning of an actuator of the tool, and the simulation controller is further configured to provide, in the simulation, an indication of positioning of the tool based on the actuator positioning information.

28. A method for remotely controlling a machine on a site, comprising:

receiving, at a location remote to the machine, real-time information relating to operation of the machine on the site, the real-time information including a slip parameter indicating slip of a traction device associated with the machine;

providing, by simulation system controller, a simulation of the operation of the machine on the site based at least on the received real-time information and a terrain map associated with the site; and

in the simulation, providing an indication of the slip of the traction device based on the slip parameter.

29. The method of claim 28, further comprising:

receiving machine control instructions at the location remote from the machine; and

enabling control of operation of the machine based on the received instructions.

30. The method of claim 28, wherein the received real-time information further includes at least one of an engine speed, ground speed, or transmission state of the machine, and the method further includes providing, in the simulation, an indication of the at least one of the engine speed, ground speed, or transmission state of the machine.

31. The method of claim 28, wherein the received real-time information further includes at least one of a positioning of an actuator of a tool associated with the machine or loading information indicating loading of the tool, and the method further includes providing, in the simulation, an indication of positioning of the tool, based on the actuator positioning information, or of the loading of the tool, based on the loading information.

32. The method of claim 28, wherein the received real-time information further includes at least one of a roll and pitch of the machine or a steering command of the machine, and the method further includes providing in the simulation an indication of the at least one of the roll and pitch of the machine or the steering command of the machine.

33. A simulation system for controlling a machine on a site remote from the simulation system, comprising:

an operator interface for presenting a simulated environment; and

a simulation controller configured to;

receive real-time information relating to operation of the machine on the site, the real-time information including a slip parameter indicating slip of a traction device associated with the machine;

provide, via the operator interface, a simulation of the operation of the machine on the site based at least on the received real-time information and a terrain map associated with the site; and

in the simulation, provide an indication of the slip of the traction device based on the slip parameter.

34. The system of claim 33, the simulation controller being further configured to:

receive, via the operator interface, machine control instructions; and

enable control of operation of the machine based on the received instructions.

35. The system of claim 33, wherein the received real-time information further includes at least one of an engine speed, ground speed, or transmission state of the machine, and the simulation controller is further configured to provide, in the simulation, an indication of the at least one of the engine speed, ground speed, or transmission state of the machine.

36. The system of claim 33, wherein the received real-time information further includes at least one of a positioning of an actuator of a tool associated with the machine or loading information indicating loading of the tool, and the simulation controller is further configured to provide, in the simulation, an indication of at least one of positioning of the tool, based on the actuator positioning information, or of the loading of the tool, based on the loading information.

37. The system of claim 33, wherein the received real-time information further includes at least one of a roll and pitch or a steering command of the machine, and the simulation controller is further configured to provide, in the simulation, an indication of the at least one of the roll and pitch or the steering command of the machine.