Method for aligning demonstrated user actions with existing documentation

• US 20060112322A1
• Filed: 11/19/2004
• Published: 05/25/2006
• Est. Priority Date: 11/19/2004
• Status: Active Grant

• Current Assignee: Activision Publishing Incorporated (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Castelli, Vittorio, Bergman, Lawrence D., Lau, Tessa A., Oblinger, Daniel A., Creswick, Eugene Rogan
• Granted Patent: US 8,176,422 B2
• Time in Patent Office: Days
• US Class Current: 715/229
• CPC Class Codes: G06F 40/10 Text processing natural language analysis G06F40/20; semantic analysis G06F40/30; processing or translation of natural language G06F40/40

Abstract

There is provided an apparatus for aligning document actions specified in a documented procedure with demonstrated user actions. The apparatus includes an alignment device for computing an alignment between the document actions and the demonstrated user actions based on at least one similarity metric.

Claims

1. An apparatus for aligning document actions specified in a documented procedure with demonstrated user actions, comprising:

an alignment device for computing an alignment between the document actions and the demonstrated user actions based on at least one similarity metric.

2. The apparatus of claim 1, wherein said alignment device identifies a current step in the documented procedure based on the alignment.

3. The apparatus of claim 2, wherein the documented procedure is displayed on a user interface, and said alignment device provides an identification of the current step to the user interface for use in highlighting the current step on the user interface.

4. The apparatus of claim 2, further comprising a user interface, in signal communication with said alignment device, for displaying the documented procedure, and wherein said alignment device outputs an identification of the current step to the user interface, and said user interface highlights the current step based on the identification.

5. The apparatus of claim 1, wherein said alignment device identifies a next step in the documented procedure based on the alignment.

6. The apparatus of claim 5, wherein the documented procedure is displayed on a user interface, and said alignment device provides an identification of the next step to the user interface for use in highlighting the next step on the user interface.

7. The apparatus of claim 5, further comprising a user interface, in signal communication with said alignment device, for displaying the documented procedure, and wherein said alignment device outputs an identification of the next step to the user interface, and said user interface highlights the next step based on the identification.

8. The apparatus of claim 1, wherein the at least one similarity metric is based on at least one of a number of words in common between a textual description of a demonstrated user action and a document action, a similarity of an action type involved in both a demonstrated user action and a document action, a probability distribution over at least two steps of the documented procedure.

9. The apparatus of claim 1, wherein the demonstrated user actions are performed with respect to a user interface, and the at least one similarity metric is based on common strings in the user interface and the documented procedure.

10. The apparatus of claim 1, further comprising a camera, connected in signal communication with said alignment device, for capturing images of at least some of the demonstrated user actions, and wherein said alignment device uses at least some of the captured images to calculate the alignment.

11. The apparatus of claim 1, wherein said alignment device computes the alignment further based on previous user experiences with respect to performing the documented procedure.

12. The apparatus of claim 1, wherein the documented procedure has a plurality of steps specified therein, and the apparatus further comprises a document action extractor for extracting at least some of the plurality of steps.

13. An apparatus for guiding user actions with respect to a documented procedure displayed on a user interface, the documented procedure having a plurality of steps specified therein, the apparatus comprising:

an alignment device for computing an alignment between the document actions and the demonstrated user actions based on at least one similarity metric;

a highlighting device, connected in signal communication with said alignment device, for visually highlighting on the user interface at least a portion of a step in the documented procedure based on the alignment computed by said alignment device; and

a scrolling device, connected in signal communication with said alignment device, for automatically scrolling through each of the plurality of steps of the documented procedure on the user interface based on the alignment computed by said alignment device.

14. The apparatus of claim 13, wherein the at least one similarity metric is based on at least one of a number of words in common between a textual description of a demonstrated user action and a document action, a similarity of an action type involved in both a demonstrated user action and a document action, a probability distribution over at least two steps of the documented procedure, and common strings in the user interface and the documented procedure.

15. The apparatus of claim 13, further comprising a camera, connected in signal communication with said alignment device, for capturing images of at least some of the demonstrated user actions, and wherein said alignment device uses at least some of the captured images to calculate the alignment.

16. The apparatus of claim 13, wherein said alignment device computes the alignment further based on previous user experiences with respect to performing the documented procedure.

17. A method for aligning document actions specified in a documented procedure with demonstrated user actions, comprising the steps of:

computing an alignment between the document actions and the demonstrated user actions based on at least one similarity metric.

18. The method of claim 17, further comprising the step of identifying a current step in the documented procedure based on the alignment.

19. The method of claim 18, wherein the documented procedure is displayed on a user interface, and the method further comprises the step of providing an identification of the current step to the user interface for use in highlighting the current step on the user interface.

20. The method of claim 17, further comprising the step of identifying a next step in the documented procedure based on the alignment.

21. The method of claim 20, wherein the documented procedure is displayed on a user interface, and the method further comprises the step of providing an identification of the next step to the user interface for use in highlighting the next step on the user interface.

22. The method of claim 17, wherein the at least one similarity metric is based on at least one of a number of words in common between a textual description of a demonstrated user action and a document action, a similarity of an action type involved in both a demonstrated user action and a document action, a probability distribution over at least two steps of the documented procedure.

23. The method of claim 17, wherein the demonstrated user actions are performed with respect to a user interface, and the at least one similarity metric is based on common strings in the user interface and the documented procedure.

24. The method of claim 17, wherein said computing step computes the alignment based on images of at least some of the demonstrated user actions captured by an imaging device.

25. The method of claim 17, wherein said computing step computes the alignment further based on previous user experiences with respect to performing the documented procedure.

26. The method of claim 17, wherein the documented procedure has a plurality of steps specified therein, and the method further comprises the step of extracting at least some of the plurality of steps from the documented procedure.

27. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for aligning document actions specified in a documented procedure with demonstrated user actions as recited in claim 17.

28. A method for guiding user actions with respect to a documented procedure displayed on a user interface, the documented procedure having a plurality of steps specified therein, the method comprising the steps of:

computing an alignment between the document actions and the demonstrated user actions based on at least one similarity metric;

visually highlighting on the user interface at least a portion of a step in the documented procedure based on the computed alignment; and

automatically scrolling through each of the plurality of steps of the documented procedure on the user interface based on the computed alignment.

29. The method of claim 28, wherein the at least one similarity metric is based on at least one of a number of words in common between a textual description of a demonstrated user action and a document action, a similarity of an action type involved in both a demonstrated user action and a document action, a probability distribution over at least two steps of the documented procedure, and common strings in the user interface and the documented procedure.

30. The method of claim 28, wherein said computing step computes the alignment based on images of at least some of the demonstrated user actions captured by an imaging device.

31. The method of claim 28, wherein said computing step computes the alignment further based on previous user experiences with respect to performing the documented procedure.

32. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for guiding user actions with respect to a documented procedure displayed on a user interface as recited in claim 28.