System and method for controlling machines remotely

• US 9,206,589 B2
• Filed: 03/30/2010
• Issued: 12/08/2015
• Est. Priority Date: 03/31/2009
• Status: Active Grant

• Current Assignee: Caterpillar Incorporated
• Original Assignee: Caterpillar Incorporated
• Inventors: Price, Robert J.
• Primary Examiner(s): KASENGE, CHARLES R
• Application Number:
  US12/750,698
• Publication Number:
  US 20100249957A1
• Time in Patent Office: 2,079 Days
• Field of Search: 700/15, 700/17, 700/83, 700/85, 700/275, 701/2, 701/50, 340/679, 340/686.1, 373/48, 715/772
• US Class Current: 1/1
• CPC Class Codes: E02F 9/205 Remotely operated machines, e.g. unmanned vehicles E02F3/8866 takes precedence
• CPC Class Codes: G08C 17/02 using a radio link

Abstract

Systems and methods for remotely controlling machines includes generating, on a display device associated with a remote control console, a first image associated with a position of the machine at a first time period. A virtual position of the machine is estimated based at least on the first position and at least one operating parameter associated with the machine. A virtual image of the machine relative to the first image is generated on the display device, the virtual image of the machine corresponding to the estimated virtual position of the machine.

Claims

1. A method for controlling a machine remotely, the method comprising:

generating, on a display device associated with a remote control console, a first image of the machine in a first position at a first time period;

estimating a virtual position of the machine based at least on the first position and at least one operating parameter associated with the machine, wherein the at least one operating parameter includes a pitch and roll of the machine; and

generating, on the display device, a virtual image of the machine superimposed on the first image, the virtual image of the machine corresponding to the estimated virtual position of the machine.

2. The method of claim 1, further including:

receiving information indicative of a second position of the machine at a second time period; and

updating the first image based on the information indicative of the second position of the machine.

3. The method of claim 1, further including:

receiving, at the remote control console associated with the display device, a command for controlling an operational aspect of the machine;

updating the virtual image of the machine based on at least one of the first image and the received command.

4. The method of claim 1, wherein the first image is indicative of an actual location of the machine and the virtual image is indicative of an estimated location of the machine, wherein the estimated location of the machine is determined by predicting behavior of the machine based on a software model adapted to predict machine performance based on the at least one operating parameter associated with the machine.

5. The method of claim 1, wherein generating the first image associated with the first position of the machine at the first time period includes:

receiving, at the first time period, information indicative of a coordinate location of the machine and an orientation of the machine;

determining a location of the machine within a worksite based on the received coordinate location of the machine and map information associated with the worksite; and

generating the first image associated with the position of the machine based on the determined location of the machine within the worksite.

6. The method of claim 5, wherein generating the virtual image superimposed on the first image of the machine includes:

receiving the at least one operating parameter associated with the machine;

predicting the virtual position of the machine within the worksite based on the coordinate location of the machine received at the first time period, an amount of time elapsed relative to the first time period, and the at least one operating parameter associated with the machine; and

generating the virtual image of the machine superimposed on the first image based on the predicted virtual position of the machine.

7. The method of claim 1, wherein the at least one operating parameter further includes at least one of a velocity of the machine, an acceleration of the machine, and an angular position of the machine.

8. A method for controlling a machine remotely, the method comprising:

receiving, at a first time period, information indicative of a coordinate location of the machine, an orientation of the machine, and at least one operating parameter associated with the machine, wherein the at least one operating parameter includes a pitch and roll of the machine;

generating, on a display device associated with a remote control console, a first image associated with a position of the machine within a worksite at a first time period;

predicting a virtual position of the machine within the worksite based on the coordinate location of the machine received at the first time period, a time delay associated with controlling the machine remotely, and the at least one operating parameter associated with the machine; and

generating on the display device, a virtual image of the machine relative to the first image, the virtual image of the machine based on the predicted virtual position of the machine.

9. The method of claim 8, further including:

receiving, at the remote control console associated with the display device, a command for controlling an operational aspect of the machine; and

updating the virtual image of the machine based on at least one of an updated first image and the received command.

10. The method of claim 8, further including:

receiving information indicative of a second position of the machine at a second time period; and

updating the first image based on the information indicative of the second position of the machine.

11. The method of claim 8, wherein the first image is indicative of an actual location of the machine and the virtual image is indicative of an estimated location of the machine, wherein the estimated location of the machine is determined by predicting behavior of the machine based on a software model adapted to predict machine performance based at least on the actual location of the machine and the at least one operating parameter associated with the machine.

12. The method of claim 8, wherein generating the first image associated with the position of the machine at the first time period includes:

determining a location of the machine within a worksite based on the received coordinate location of the machine and map information associated with the worksite; and

generating the first image associated with the position of the machine based on the determined location of the machine within the worksite.

13. The method of claim 8, wherein the at least one operating parameter further includes at least one of a velocity of the machine, an acceleration of the machine, and an angular position of the machine.

14. A remote control console configured to control a machine remotely, the remote control console comprising:

an operator interface configured to receive an input from an operator corresponding to a desired location of the machine; and

a processor, configured to;

generate, on a display device associated with a remote control console, a first image of the machine in a first position at a first time period;

estimate a virtual position of the machine based at least on the first position and at least one operating parameter associated with the machine, wherein the at least one operating parameter includes a pitch and roll of the machine; and

generate, on the display device, a virtual image of the machine superimposed on the first image, the virtual image of the machine corresponding to the estimated virtual position of the machine.

15. The remote control console of claim 14, wherein the processor is further configured to:

receive information indicative of a second position of the machine at a second time period; and

update the first image based on the information indicative of the second position of the machine.

16. The remote control console of claim 14, wherein the processor is further configured to:

receive, at the remote control console associated with the display device, a command for controlling an operational aspect of the machine; and

update the virtual image of the machine based on at least one of the first image and the received command.

17. The remote control console of claim 14, wherein the first image is indicative of an actual location of the machine and the virtual image is indicative of an estimated location of the machine, wherein the estimated location of the machine is determined by predicting behavior of the machine based on a software model adapted to predict machine performance based at least on the first position of the machine and the at least one operating parameter associated with the machine.

18. The remote control console of claim 14, wherein generating the first image associated with the first position of the machine at the first time period includes:

receiving, at the first time period, information indicative of a coordinate location of the machine and an orientation of the machine;

determining a location of the machine within a worksite based on the received coordinate location of the machine and map information associated with the worksite; and

generating the first image associated with the position of the machine based on the determined location of the machine within the worksite.

19. The remote control console of claim 18, wherein generating the virtual image superimposed on the first image of the machine includes:

receiving the at least one operating parameter associated with the machine;

predicting the virtual position of the machine within the worksite based on the coordinate location of the machine received at the first time period, an amount of time elapsed relative to the first time period, and the at least one operating parameter associated with the machine; and

generating the virtual image of the machine superimposed on the first image based on the predicted virtual position of the machine.

20. The remote control console of claim 14, wherein the at least one operating parameter further includes at least one of a velocity of the machine, an acceleration of the machine, and an angular position of the machine.