INDEXING AND ANNOTATING A USABILITY TEST RECORDING

• US 20150356062A1
• Filed: 06/06/2014
• Published: 12/10/2015
• Est. Priority Date: 06/06/2014
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: DE PAULA, ROGERIO ABREU, PINHANEZ, CLAUDIO SANTOS, DE SANTANA, VAGNER FIGUEREDO
• Granted Patent: US 10,649,634 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/3438 monitoring of user actions tracking the activity of the user H04L67/535
• CPC Class Codes: G06F 17/40 Data acquisition and logging for input to computer G06F3/00
• CPC Class Codes: G06F 2201/835 Timestamp
• CPC Class Codes: G06F 2201/86 Event-based monitoring
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence

Abstract

A method for indexing a user interface test recording includes capturing, during a session, a recording of a plurality of interactions with the user interface, capturing, during the session, an event stream including a plurality of user interface events, synchronizing, in time, the plurality of interactions and the plurality of user interface events, identifying a point of interest in the event stream, wherein the point of interest is correlated to a time in the recording by the synchronization, and annotating the recording at a time correlated to when the point of interest occurred.

Claims

1. A method for indexing a user interface test recording comprising:

capturing, during a session, a recording of a plurality of interactions with the user interface;

capturing, during the session, an event stream including a plurality of user interface events;

synchronizing, in time, the plurality of interactions and the plurality of user interface events;

identifying a point of interest in the event stream, wherein the point of interest is correlated to a time in the recording by the synchronization; and

annotating the recording at a time correlated to when the point of interest occurred.

2. The method of claim 1, wherein the point of interest corresponds to an activity of a user.

3. The method of claim 1, wherein the point of interest corresponds to a period of inactivity of a user.

4. The method of claim 1, wherein identifying the point of interest in the event stream further comprises building an event digraph comprising a plurality of nodes corresponding to individual ones of the plurality of user interface events and a plurality of edges representing a sequence of the plurality of user interface events over time, wherein the event digraph models the plurality of user interface events over time.

5. The method of claim 4, wherein building the event digraph comprises labeling each of the plurality of nodes based on a corresponding user interface event type and a user interface element identification.

6. The method of claim 4, wherein the digraph comprises a root node representing a beginning of the session and having a mean timestamp.

7. The method of claim 4, wherein the point of interest corresponds to one or more of the following:

a number of nodes to reach a certain node in the event digraph greater than a threshold number of nodes;

an amount of time to reach a certain node in the event digraph greater than a threshold time; and

an elapsed time corresponding to a single node above a threshold elapsed time.

8. A computer program product for indexing a user interface test recording, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform a method comprising:

capturing, during a session, a recording of a plurality of interactions with the user interface;

capturing, during the session, an event stream including a plurality of user interface events;

synchronizing, in time, the plurality of interactions and the plurality of user interface events;

identifying a point of interest in the event stream, wherein the point of interest is correlated to a time in the recording by the synchronization; and

annotating the recording at a time correlated to when the point of interest occurred.

9. The computer program product of claim 8, wherein the point of interest corresponds to an activity of a user.

10. The computer program product of claim 8, wherein the point of interest corresponds to a period of inactivity of a user.

11. The computer program product of claim 8, wherein identifying the point of interest in the event stream further comprises building an event digraph comprising a plurality of nodes corresponding to individual ones of the plurality of user interface events and a plurality of edges representing a sequence of the plurality of user interface events over time, wherein the event digraph models the plurality of user interface events over time.

12. The computer program product of claim 11, wherein building the event digraph comprises labeling each of the plurality of nodes based on a corresponding user interface event type and a user interface element identification.

13. The computer program product of claim 11, wherein the digraph comprises a root node representing a beginning of the session and having a mean timestamp.

14. The computer program product of claim 11, wherein the point of interest corresponds to one or more of the following:

a number of nodes to reach a certain node in the event digraph greater than a threshold number of nodes;

an amount of time to reach a certain node in the event digraph greater than a threshold time; and

an elapsed time corresponding to a single node above a threshold elapsed time.

21. The apparatus of claim 14, wherein the event logger generates a signal used by the synchronizer to synchronize, in time, the plurality of interactions and the plurality of user interface events.

15. An apparatus configured to index a user interface test recording comprising:

a recording device configured to capture, during a session, a video of a plurality of interactions with the user interface; and

a computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to embody;

an event logger configured to capture, during the session, an event stream including a plurality of user interface events;

a synchronizer configured to synchronize, in time, the plurality of interactions and the plurality of user interface events;

a user interface modeler configured to generate a model of the user interface events;

a model analyzer configured to identify a point of interest in the event stream, wherein the point of interest is correlated to a time in the recording by the synchronization; and

an annotator configured to annotate the video at a time correlated to when the point of interest occurred.

16. The apparatus of claim 15, further comprising a display displaying the video including at least one annotation correlated to the time in the video the point of interest occurred.

17. The apparatus of claim 15, wherein the recording device records audio corresponding to at least one of the plurality of interactions with the user interface.

18. The apparatus of claim 15, wherein identifying the point of interest in the event stream further comprises building an event digraph comprising a plurality of nodes corresponding to individual ones of the plurality of user interface events and a plurality of edges representing a sequence of the plurality of user interface events over time, wherein the event digraph models the plurality of user interface events over time, and the diagraph is stored in memory accessible to the processor.

19. The apparatus of claim 18, wherein building the event digraph comprises labeling each of the plurality of nodes based on a corresponding user interface event type and a user interface element identification.

20. The apparatus of claim 18, wherein the digraph comprises a root node representing a beginning of the session and having a mean timestamp.