Method and system for integrating visual controls with legacy applications

• US 9,141,443 B2
• Filed: 01/07/2013
• Issued: 09/22/2015
• Est. Priority Date: 01/07/2013
• Status: Active Grant

• Current Assignee: GE Digital Holdings LLC (GE Aerospace)
• Original Assignee: General Electric Company
• Inventors: Iyer, Giridhar, Tadepalli, Venu
• Primary Examiner(s): Ho, Andy
• Application Number:
  US13/735,794
• Publication Number:
  US 20140196061A1
• Time in Patent Office: 988 Days
• Field of Search: 719/320, 715/863
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 9/541 via adapters, e.g. between incompatible applications

Abstract

A method is provided for controlling a legacy application. The method includes visually capturing a series of movements. The method also includes recording the series of movements at a first time as a recorded series of movements. Moreover, the recorded series of movements includes at least a command that is configured to control a legacy application. Furthermore, the method includes invoking the recorded series of movements at a second time.

Claims

1. A method for controlling a legacy application, comprising:

visually capturing a series of movements;

recording the series of movements at a first time as a recorded series of movements, wherein the recorded series of movements comprises at least a command that is configured to control a legacy application;

editing the recorded series of movements, wherein the recorded series of movements comprise visually captured movements of a user comprising two or more recorded gesture types; and

invoking the recorded series of movements at a second time.

2. The method of claim 1, wherein editing the recorded series of movements is performed at a third time.

3. The method of claim 2, wherein editing of the recorded series of movements comprises using a mobile computing device.

4. The method of claim 1, wherein recording the series of movements comprises receiving user input to record a name for the recorded series of movements.

5. The method of claim 1, wherein recording the series of movements comprises using an image capturing device.

6. The method of claim 5, wherein using the image capturing device comprises using a wand configured to assist the image capturing device in capturing the series of movements.

7. The method of claim 1, wherein recording the series of movements comprises capturing voice commands and integrating the voice commands into the recorded series of movements.

8. The method of claim 1, comprising controlling a legacy application by using the recorded series of movements and without using a legacy application programming interface configured to control the legacy application, wherein the legacy application is configured to manage an electrical network.

9. A system, comprising:

at least one processor;

a memory communicatively coupled to the at least one processor;

an image capturing device configured to record a series of movements in the memory at a first time as a recorded series of movements, wherein the recorded series of movements comprises at least a command that is configured to control a legacy application;

an editing interface configured to enable a user to edit the recorded series of movements, wherein the recorded series of movements comprise visually captured movements of a user comprising two or more recorded gesture types; and

an input interface configured to invoke the recorded series of movements at a second time.

10. The system of claim 9, wherein the input interface comprises a mobile computing device remote from the at least one processor.

11. The system of claim 10, wherein the mobile computing device comprises an editing interface configured to edit the recorded series of movements.

12. The system of claim 9, comprising a microphone configured to capture voice commands associated with the series of movements.

13. The system of claim 9, wherein the legacy application is configured to manage an electrical network.

14. The system of claim 9, wherein the at least one processor derives a plurality of user interface commands configured to navigate through a user interface included in the legacy application based on the recorded series of movements.

15. The system of claim 14, wherein an execution of the plurality of user interface commands by the at least one processor is faster than a playback of the recorded series of movements by the at least one processor.

16. The system of claim 9, comprising a display, wherein the series of movements comprises movements where a user is not touching the display.

17. A visual control apparatus, comprising:

an image capture device configured to capture a series of movements;

a memory configured to record the series of movements as a recorded series of movements, wherein the recorded series of the movements comprises at least a command that is configured to control a legacy application;

an editing interface configured to enable a user to edit the recorded series of movements, wherein the recorded series of movements comprise visually captured movements of a user comprising two or more recorded gesture types; and

a user interface application configured to invoke or to edit the recorded series of movements.

18. The apparatus of claim 17, wherein the legacy application is configured to manage an electrical network.

19. The apparatus of claim 17, comprising a communication interface configured to transport the recorded series of movements to the legacy application through one or more pools as a protein.

20. The apparatus of claim 19, wherein each pool is a queue implemented as a ring buffer, and wherein the protein is a self describing data structure unit.