Machine control over the web

• US 20030036817A1
• Filed: 08/09/2002
• Published: 02/20/2003
• Est. Priority Date: 08/16/2001
• Status: Active Grant

• Current Assignee: R Morley Incorporated
• Original Assignee: R Morley Incorporated
• Inventors: Grondstra, Jan W., Morley, Richard E., Lajoie, James
• Granted Patent: US 6,739,078 B2
• Time in Patent Office: Days
• US Class Current: 700/245
• CPC Class Codes: B25J 9/1656 characterised by programming, planning systems for manipulators
• CPC Class Codes: E02F 3/435 for dipper-arms, backhoes or the like
• CPC Class Codes: E02F 9/2045 Guiding machines along a predetermined path for graders E02F3/841; machines for construction of roads E01C19/004
• CPC Class Codes: E02F 9/205 Remotely operated machines, e.g. unmanned vehicles E02F3/8866 takes precedence
• CPC Class Codes: G05B 2219/31241 Remote control by a proxy or echo server, internet - intranet
• CPC Class Codes: G05B 2219/33192 Radio link, wireless
• CPC Class Codes: G05B 2219/34279 Pc, personal computer as controller
• CPC Class Codes: G05B 2219/36169 Remote, host controlled, operated manual data input, keyboard
• CPC Class Codes: G05B 2219/40099 Graphical user interface for robotics, visual robot user interface
• CPC Class Codes: G05B 2219/45012 Excavator

Abstract

A system to control a piece of construction equipment or other piece of heavy equipment at a remote location via a data network is disclosed in which a user provides movement instructions via a graphical user interface at a user PC to a programmable controller interfaced to the data network and the hydraulic movement systems of the heavy equipment. The graphical user interface includes a visual representation portion and a user control portion. A visual representation of the moveable elements of the heavy equipment is provided to the user via the graphical user interface, and the user inputs movement instructions via the user control portion of the graphical user interface.

Claims

1. An apparatus for a user to remotely controlling a backhoe across a data network, the backhoe including a boom arm, a dipper arm, and a bucket each being controlled by a corresponding electrically controlled hydraulic movement system, the apparatus comprising:

a user personal computer having a graphical user interface, the graphical user interface operable to receive movement instructions from the user and to display a visual representation of the backhoe in response to received position data, the user personal computer coupled to the data network and operable to transmit movement instructions over the data network and to receive position data;

first and second wireless modems capable of transmitting and receiving data over a wireless connection to one another;

a remote personal computer coupled to the data network and operable to receive movement instructions therefrom, the remote personal computer coupled to the first wireless modem and operable to transmit the movement instructions;

a programmable controller coupled to the second wireless modem and operable to receive the movement instructions, the programmable controller further operable to translate the received user instructions into electrical control signals, the programmable controller coupled to the boom arm, dipper arm, and bucket hydraulic movement control systems, wherein the electrical control signals are routed to the hydraulic movement control system corresponding to the movement instructions to provide for actuation of the corresponding hydraulic movement system; and

a data generator coupled to the user personal computer, the data generator operable to provide position data corresponding to the movement instructions.

2. The apparatus of claim 1 wherein movement instructions include one from the group consisting of boom swivel left or right, boom up or down, dipper out or in, and bucket scoop or drop.

3. The apparatus of claim 1 wherein the visual representation of the backhoe is a schematic representation of the side view and a schematic representation of the top view of the backhoe.

4. The apparatus of claim 1 wherein the data generator is a mathematical model of the boom arm, dipper arm, and bucket in which the movement instructions are used as input and that provides position data as an output.

5. The apparatus of claim 4 wherein the data generator is included in the remote personal computer and the data generator provides the position data to the remote personal computer for transmission over the data network to the user personal computer.

6. The apparatus of claim 4 wherein the data generator is included in the user personal computer and provides position data thereto.

7. The apparatus of claim 1 further including a server coupled in series between the data network and the remote personal computer, wherein the data generator is included in the server and the data generator provides the position data to the server for transmission over the data network to the user personal computer.

8. The apparatus of claim 4 wherein the data generator is included in the programmable controller and the data generator provides the position data to the programmable controller for transmission to the user personal computer.

9. The apparatus of claim 1 wherein the data generator includes first, second, and third position sensors for sensing the boom position, the dipper position, and the bucket position respectively, the first, second, and third position sensors are coupled to the remote personal computer and provide measurement data thereto, the remote personal computer being operative to transmit the position data over the data network to the user personal computer.

10. The apparatus of claim 9 wherein the data generator is a mathematical model of the boom arm, dipper arm, and bucket in which the movement instructions are used as input and that provides position data as an output, and wherein the measurement data provided the by the first, second, and third position sensors is periodically provided to the user PC to update the position data provided by the mathematical model.

11. The apparatus of claim 1 wherein the data generator includes a video camera providing video images of the backhoe to the remote personal computer for transmission over the data network to the user personal computer, and wherein the visual representation of the backhoe is a video image displayed on the user personal computer.

12. The apparatus of claim 1 further comprising the programmable controller being coupled to a steering hydraulic movement system and a backhoe movement hydraulic movement system and providing electrical control signals thereto, wherein the backhoe may be moved by the application of appropriate control signals.

13. The apparatus of claim 1 wherein one of the group of the user PC, the remote PC, or the programmable controller autonomously generates one or more movement commands.

14. The apparatus of claim 13 wherein the autonomously generated movement commands are generated in response to a feedback signal.

15. The apparatus of claim 14 wherein the feedback signal is a predetermined time-stamp.

16. The apparatus of claim 14 wherein the feedback signal is a predetermined position.

17. The apparatus of claim 1 further including a server connected in series between the data network and the remote PC, and wherein the server generates one or more movement commands.

18. The apparatus of claim 17 wherein the autonomously generated movement commands are generated in response to a feedback signal.

19. The apparatus of claim 18 wherein the feedback signal is a predetermined time-stamp.

20. The apparatus of claim 18 wherein the feedback signal is a predetermined position.

21. The apparatus of claim 1 wherein the user personal computer is coupled to the data network using a low bandwidth connection.

22. The apparatus of claim 21 wherein the low bandwidth connection is a dial up line

23. An apparatus for a user to remotely controlling a heavy equipment across a data network, the heavy equipment including one or more moveable elements being controlled by a corresponding electrically controlled hydraulic movement system, the apparatus comprising:

a user personal computer having a graphical user interface, the graphical user interface operable to receive movement instructions from the user and to display a visual representation of the heavy equipment in response to received position data, the user personal computer coupled to the data network and operable to transmit movement instructions over the data network and to receive position data;

first and second wireless modems capable of transmitting and receiving data over a wireless connection to one another;

a remote personal computer coupled to the data network and operable to receive movement instructions therefrom, the remote personal computer coupled to the first wireless modem and operable to transmit the movement instructions;

a programmable controller coupled to the second wireless modem and operable to receive the movement instructions, the programmable controller further operable to translate the received user instructions into electrical control signals, the programmable controller coupled to the one or more moveable elements hydraulic movement control systems, wherein the electrical control signals are routed to the hydraulic movement control system corresponding to the movement instructions to provide for actuation of the corresponding hydraulic movement system; and

a data generator coupled to the user personal computer, the data generator operable to provide position data corresponding to the movement instructions.

24. The apparatus of claim 23 wherein the visual representation of the backhoe is a schematic representation of the side view and a schematic representation of the top view of the heavy equipment.

25. The apparatus of claim 23 wherein the data generator is a mathematical model of the one or more moveable elements in which the movement instructions are used as input and that provides position data as an output.

26. The apparatus of claim 23 wherein the data generator is included in the remote personal computer and the data generator provides the position data to the remote personal computer for transmission over the data network to the user personal computer.

27. The apparatus of claim 23 wherein the data generator is included in the user personal computer and provides position data thereto.

28. The apparatus of claim 23 further including a server coupled in series between the data network and the remote personal computer, wherein the data generator is included in the server and the data generator provides the position data to the server for transmission over the data network to the user personal computer.

29. The apparatus of claim 23 wherein the data generator is included in the programmable controller and the data generator provides the position data to the programmable controller for transmission to the user personal computer.

30. The apparatus of claim 23 wherein the data generator includes a plurality of position sensors for sensing the position of the one or more moveable elements and providing measurement data thereof, the plurality of position sensors being coupled to the remote personal computer and provide the measurement data thereto, the remote personal computer being operative to transmit the measurement data over the data network to the user personal computer.

31. The apparatus of claim 30 wherein the data generator is a mathematical model of the one or more moveable elements in which the movement instructions are used as input and that provides position data as an output, and wherein the measurement data provided the by plurality of position sensors is periodically provided to the user PC to update the position data provided by the mathematical model.

32. The apparatus of claim 23 wherein the data generator includes a video camera providing video images of the heavy equipment to the remote personal computer for transmission over the data network to the user personal computer, and wherein the visual representation of the heavy equipment is a video image displayed on the user personal computer.

33. The apparatus of claim 23 further comprising the programmable controller being coupled to a steering hydraulic movement system and an equipment movement hydraulic movement system and providing electrical control signals thereto, wherein the heavy equipment may be moved by the application of appropriate control signals.

34. The apparatus of claim 23 wherein one of the group of the user PC, the remote PC, or the programmable controller autonomously generates one or more movement commands.

35. The apparatus of claim 34 wherein the autonomously generated movement commands are generated in response to a feedback signal.

36. The apparatus of claim 35 wherein the feedback signal is a predetermined time-stamp.

37. The apparatus of claim 35 wherein the feedback signal is a predetermined position.

38. The apparatus of claim 23 further including a server connected in series between the data network and the remote PC, and wherein the server generates one or more movement commands.

39. The apparatus of claim 38 wherein the autonomously generated movement commands are generated in response to a feedback signal.

40. The apparatus of claim 39 wherein the feedback signal is a predetermined time-stamp.

41. The apparatus of claim 39 wherein the feedback signal is a predetermined position.

42. The apparatus of claim 23 wherein the user personal computer is coupled to the data network using a low bandwidth connection.

43. The apparatus of claim 23 wherein the low bandwidth connection is a dial up line

44. An apparatus for controlling a plurality of pieces of heavy equipment at one or more remote locations from a user via a data network, the heavy equipment including one or more moveable elements being controlled by a corresponding electrically controlled hydraulic movement system, the apparatus comprising:

a user personal computer having a graphical user interface, the graphical user interface operable to receive movement instructions from the user and to display a visual representation of the heavy equipment in response to received position data, the user personal computer coupled to the data network and operable to transmit movement instructions over the data network and to receive position data;

a plurality of first and second wireless modems each of the first wireless modems corresponding to an associated one of the plurality of second modems forming a plurality of modem pairs, each of the plurality of modem pairs corresponding to one of the plurality of pieces of heavy equipment, each of the modem pairs capable of transmitting and receiving data over a wireless connection to one another;

a remote personal computer coupled to the data network and operable to receive movement instructions therefrom, the remote personal computer coupled to the first wireless modem and operable to transmit the movement instructions;

a programmable controller coupled to the second wireless modem and operable to receive the movement instructions, the programmable controller further operable to translate the received user instructions into electrical control signals, the programmable controller coupled to the one or more moveable elements hydraulic movement control systems, wherein the electrical control signals are routed to the hydraulic movement control system corresponding to the movement instructions to provide for actuation of the corresponding hydraulic movement system; and

a data generator coupled to the user personal computer, the data generator operable to provide position data corresponding to the movement instructions.

45. The apparatus of claim 44 wherein the visual representation of the backhoe is a schematic representation of the side view and a schematic representation of the top view of the heavy equipment.

46. The apparatus of claim 44 wherein the data generator is a mathematical model of the one or more moveable elements in which the movement instructions are used as input and that provides position data as an output.

47. The apparatus of claim 44 further including a server coupled in series between the data network and each of the plurality of remote personal computers, wherein the data generator is included in the server and the data generator provides the position data to the server for transmission over the data network to the user personal computer.

48. The apparatus of claim 44 wherein the data generator includes a plurality of position sensors corresponding to each of the plurality of heavy equipment for sensing the position of the one or more moveable elements of the corresponding piece of heavy equipment and providing measurement data thereof, the plurality of position sensors being coupled to the corresponding remote personal computer and provide the measurement data thereto, the corresponding remote personal computer being operative to transmit the measurement data over the data network to the user personal computer.

49. The apparatus of claim 48 wherein the data generator is a mathematical model of the one or more moveable elements corresponding to each of the plurality of pieces of heavy equipment in which the movement instructions are used as input and that provides position data as an output, and wherein the measurement data provided the by plurality of position sensors is periodically provided to the user PC to update the position data provided by the mathematical model.

50. The apparatus of claim 44 wherein the data generator includes a video camera providing video images of the heavy equipment to the remote personal computer for transmission over the data network to the user personal computer, and wherein the visual representation of the heavy equipment is a video image displayed on the user personal computer.

51. The apparatus of claim 44 wherein one of the group of the user PC, each of the plurality of remote PCs, or the programmable controller autonomously generates one or more movement commands for one or more pieces of heavy equipment.

52. The apparatus of claim 51 wherein the autonomously generated movement commands are generated in response to a feedback signal.

53. The apparatus of claim 51 wherein the feedback signal is a predetermined time-stamp.

54. The apparatus of claim 51 wherein the feedback signal is a predetermined position.

55. The apparatus of claim 44 further including a server connected in series between the data network and the plurality of remote PCs, and wherein the server generates one or more movement commands for one or more of the pieces of heavy equipment.

56. The apparatus of claim 55 wherein the autonomously generated movement commands are generated in response to a feedback signal.

57. The apparatus of claim 56 wherein the feedback signal is a predetermined time-stamp.

58. The apparatus of claim 56 wherein the feedback signal is a predetermined position.

59. The apparatus of claim 44 wherein the user personal computer is coupled to the data network using a low bandwidth connection.

60. The apparatus of claim 44 wherein the low bandwidth connection is a dial up line