Computer interface system for a robotic system

• US 6,292,712 B1
• Filed: 01/29/1998
• Issued: 09/18/2001
• Est. Priority Date: 01/29/1998
• Status: Expired due to Term

• Current Assignee: Northrop Grumman Systems Corporation (Northrop Grumman Corporation)
• Original Assignee: Northrop Grumman Corporation
• Inventors: Bullen, George Nicholas
• Primary Examiner(s): Grant, William
• Assistant Examiner(s): Bahta, Kidest
• Application Number:
  US09/016,797
• Time in Patent Office: 1,328 Days
• Field of Search: 700/17, 700/245, 700/264, 700/246, 700/253, 700/257, 901/3, 901/4, 901/5
• US Class Current: 700/245
• CPC Class Codes: G05B 19/409 characterised by using manual data input [MDI] or by using control panel, e.g. controlling functions with the panel; characterised by control panel details or by setting parameters G05B19/408, G05B19/4093 take precedence
• CPC Class Codes: G05B 2219/35481 Display, panel
• CPC Class Codes: G05B 2219/35513 Setting tool condition, tool set in tool exchanger, present or not
• CPC Class Codes: G05B 2219/35515 Workpiece set condition, workpiece present or not

Abstract

The present invention is an interactive multi-media interface system which incorporates text, audio, and video graphics animation and other tools to interface a user with a device in an outside environment, such as a robotic device, machining or other tool. The interface system includes a computer workstation to allow inputs from a user, such as a machine operator, computer memory, and an interface computer program operating on the computer. The computer has a graphical user interface for interfacing the operator with the workstation and to allow interactive operation and control of the machine tool. The operator can provide inputs to the interface system by various input devices such as a keyboard, cursor, mouse, a stylus pointing device or a touch sensitive screen. The interface software shares access to and exchanges data with the computer memory.

Claims

1. A computer implemented method for interfacing an operator with a robotic system, comprising the steps of:

(a) providing an interactive computer environment with interface software operating thereon;

(b) processing engineering data stored on a computer memory by said interface software, wherein said engineering data contains operating information relating to the robotic system;

(c) interfacing the operator with said interactive computer environment with a graphical user interface;

(d) displaying a menu of said graphical user interface with at least one interface module having preprogrammed sensory media associated with actual robotic events of the robotic system;

(e) operating, controlling, and monitoring the robotic system interactively in real-time by said operator through said preprogrammed sensory media of at least one interface module;

(f) performing continuous motion control interaction of the robotic system instantaneously and performing specific tasks and operations associated with the robotic system by said engineering data processed by said interface software; and

(g) adjusting control of the robotic system automatically in real-time by said interface software in response to predetermined varying conditions surrounding and affecting motion and control of said robot system and controls from said operator.

2. The method of claim 1, wherein said step (d) further comprises:

(d1) displaying a login menu of said graphical user interface for requiring the operator to login to said interactive computer environment in order to obtain usable access of said interactive computer environment.

3. The method of claim 2, wherein said step (d1) further comprises:

requiring the operator to login to said interactive computer environment with an identifying login name and password.

4. The method of claim 1, wherein said step (d) further comprises:

(d1) displaying a main menu of said graphical user interface for allowing the operator to select one of a plurality of operating series, each operating series having a submenu and being associated with an actual operation event of the robotic system.

5. The method of claim 1, wherein said step (g) further comprises:

(g1) performing an operator login for requiring the operator to login to said interactive computer environment in order to obtain usable access of said interactive computer environment;

(g2) performing idle operations for providing management and coordination of idle states of the robotic system and said interactive computer environment;

(g3) performing robotic system operations for providing management of movements and control operations of the robotic system; and

(g4) performing completion operations for providing general management, control and monitoring of the robotic system.

6. The method of claim 1 wherein step (d) further comprises displaying a text window for providing textual information and operator selectable hypertext and a graphical window for providing graphical information and operator selectable hypergraphics associated with actual events of an external environment outside said computer environment.

7. The method of claim 1 further comprising linking said computer memory of said interactive computer environment with a remote station for monitoring and supervising said operator in real time, wherein said remote station shares access to and exchanges data with said computer memory.

8. The method of claim 1 further comprising providing training software operating on said interactive computer environment for training said operator, wherein said training software shares access to and exchange of data with said computer memory.

9. The method of claim 8 wherein said step of providing training software further comprises providing an operator selectable simulation training module for viewing a preprogrammed simulation of training operations.

10. The method of claim 9 wherein said step of providing an operator selectable simulation training module further comprises displaying a preprogrammed animated sequence of steps replicating actual steps of said training operation in a graphical window.

11. The method of claim 1 wherein the step (d) further comprises displaying a preprogrammed animated sequence of steps replicating actual specific tasks and operations to be performed by the robotic system in a graphical window.

12. The method of claim 1 wherein the step (d) further comprises displaying a preprogrammed video sequence of steps replicating actual specific tasks and operations to be performed by the robotic system in a graphical window.

13. The method of claim 1 wherein the step (d) further comprises displaying error modules prompted by preprogrammed events occurring associated with operator error input.

14. An interface system for interactively operating and controlling a robotic system by an operator, comprising:

an interactive computer environment having;

interface software operating on the interactive computer environment and having engineering data stored on a computer memory and containing operating information relating to the robotic system, said interface software being adapted to automatically adjust control of the robotic system in real-time in response to predetermined varying conditions surrounding and affecting motion and control of the robotic system and controls from said operator in response to the engineering data; and

a graphical user interface for interfacing said operator with said computer environment, said graphical user interface having a menu with at least one interface module with preprogrammed sensory media associated with robotic events of said robotic system, said sensory media operative to provide said operator with instantaneous continuous motion control interaction of the robotic system and specific tasks and operations associated with the robotic system.

15. The interface system of claim 14, further comprising:

a login menu of said graphical user interface for requiring the operator to login to said interactive computer environment in order to obtain usable access of said interactive computer environment.

16. The interface system of claim 14, further comprising:

a login menu of said graphical user interface requiring the operator to login to said interactive computer environment with an identifying login name and password.

17. The interface system of claim 14, further comprising:

a main menu of said graphical user interface for allowing the operator to select one of a plurality of operating series, each operating series having a submenu and being associated with an actual operation event of the robotic system.

18. The interface system of claim 14, further comprising:

an operator login menu for requiring the operator to login to said interactive computer environment in order to obtain usable access of said interactive computer environment;

idle operation menus for providing management and coordination of idle states of the robotic system and said interactive computer environment;

robotic system operation menus for providing management of movements and control operations of the robotic system; and

completion operation menus for providing general management, control and monitoring of the robotic system.

19. The interface system of claim 14, further comprising a text window for providing textual information and operator selectable hypertext and a graphical window for providing graphical information and operator selectable hypergraphics associated with actual events of an external environment outside said computer environment.

20. The interface system of claim 14, further comprising a link from said computer memory of said interactive computer environment to a remote station for monitoring and supervising said operator in real time, wherein said remote station shares access to and exchanges data with said computer memory.

21. The interface system of claim 14, further comprising training software operating on said interactive computer environment for training said operator, wherein said training software shares access to and exchange of data with said computer memory.

22. The interface system of claim 21, wherein said training software further comprises an operator selectable simulation training module for viewing a preprogrammed simulation of training operations.

23. The interface system of claim 22, wherein said operator selectable simulation training module further comprises a preprogrammed animated sequence of steps replicating actual steps of said training operation in a graphical window.

24. The interface system of claim 14, further comprising a preprogrammed animated sequence of steps replicating actual specific tasks and operations to be performed by the robotic system in a graphical window.

25. The interface system of claim 14, further comprising a preprogrammed video sequence of steps replicating actual specific tasks and operations to be performed by the robotic system in a graphical window.

26. The interface system of claim 14, further comprising error modules prompted by preprogrammed events occurring associated with operator error input.

27. A computer-readable medium for causing an interactive computer to function as an interface system for interactively operating and controlling a robotic system by an operator, comprising:

a computer-readable storage medium; and

a computer program stored on said storage medium, said computer program preprogrammed to process engineering data stored on a computer memory and having operation information relating to the robotic system, said computer program adapted to access in real-time a database containing preprogrammed sensory media associated with said engineering data and actual robotic events of the robotic system;

said computer program being preprogrammed to automatically adjust control of the robotic system in real-time in response to predetermined varying conditions surrounding and affecting motion and control of the robot system and controls from said operator and in response to the engineering data;

said computer program having a graphical user interface module for interfacing said operator with said computer program and said sensory media, said graphical user interface module including a menu with interface modules associated with said preprogrammed sensory media;

said preprogrammed sensory media being adapted to interface said operator with said robotic system to interactively operate and control said robotic system in real-time and to instantaneously perform continuous motion control interaction of the robotic system and to perform specific tasks and operations associated with the robotic system.

28. The computer-readable medium of claim 27, further comprising:

a login menu of said graphical user interface for requiring the operator to login to said interactive computer environment in order to obtain usable access of said interactive computer environment.

29. The computer-readable medium of claim 27, further comprising:

a login menu of said graphical user interface requiring the operator to login to said interactive computer environment with an identifying login name and password.

30. The computer-readable medium of claim 27, further comprising:

a main menu of said graphical user interface for allowing the operator to select one of a plurality of operating series, each operating series having a submenu and being associated with an actual operation event of the robotic system.

31. The computer-readable medium of claim 27, further comprising:

an operator login menu for requiring the operator to login to said interactive computer environment in order to obtain usable access of said interactive computer environment;

idle operation menus for providing management and coordination of idle states of the robotic system and said interactive computer environment;

robotic system operation menus for providing management of movements and control operations of the robotic system; and

completion operation menus for providing general management, control and monitoring of the robotic system.

32. The computer-readable medium of claim 27, further comprising a text window for providing textual information and operator selectable hypertext and a graphical window for providing graphical information and operator selectable hypergraphics associated with actual events of an external environment outside said computer environment.

33. The computer-readable medium of claim 27, further comprising a link from said computer memory of said interactive computer environment to a remote station for monitoring and supervising said operator in real time, wherein said remote station shares access to and exchanges data with said computer memory.

34. The computer-readable medium of claim 27, further comprising training software operating on said interactive computer environment for training said operator, wherein said training software shares access to and exchange of data with said computer memory.

35. The computer-readable medium of claim 34, wherein said training software further comprises an operator selectable simulation training module for viewing a preprogrammed simulation of training operations.

36. The computer-readable medium of claim 35, wherein said operator selectable simulation training module further comprises a preprogrammed animated sequence of steps replicating actual steps of said training operation in a graphical window.

37. The computer-readable medium of claim 27, further comprising a preprogrammed animated sequence of steps replicating actual specific tasks and operations to be performed by the robotic system in a graphical window.

38. The computer-readable medium of claim 27, further comprising a preprogrammed video sequence of steps replicating actual specific tasks and operations to be performed by the robotic system in a graphical window.

39. The computer-readable medium of claim 27, further comprising error modules prompted by preprogrammed events occurring associated with operator error input.