Recording touch information

• US 10,423,515 B2
• Filed: 11/29/2011
• Issued: 09/24/2019
• Est. Priority Date: 11/29/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Thiruvillamalai, Varadarajan (TSV), Dixit, Deepraj Sitaram, Sahoo, Tapas Ranjan, Reddy, Singireddy Neeraja
• Primary Examiner(s): Sax, Steven P
• Application Number:
  US13/306,245
• Publication Number:
  US 20130139115A1
• Time in Patent Office: 2,856 Days
• CPC Class Codes: G06F 11/3428 Benchmarking
• CPC Class Codes: G06F 11/3438 monitoring of user actions tracking the activity of the user H04L67/535
• CPC Class Codes: G06F 11/3476 Data logging G06F11/14, G06F11/2205 take precedence
• CPC Class Codes: G06F 2201/865 Monitoring of software
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

A method of recording user-driven events within a computing system includes receiving at a motion-sensitive display surface at least one user-performed gesture, which includes user movement of an object over the surface that recognizes such user interaction therewith. Touch information is generated corresponding to the at least one user-performed gesture. The touch information is configured to be provided to an application. The touch information is intercepted and recorded before it is provided to the application. The intercepted touch information is grouped into at least one chunk, and the at least one chunk is output to the application.

Claims

1. A method of recording user-driven events within a computing system, comprising:

receiving at a motion-sensitive surface at least one user-performed interaction, which includes user movement of an object over the surface that recognizes such user interaction therewith;

generating touch information corresponding to the at least one user-performed interaction, wherein the touch information comprises touch frames, and wherein the touch frames are configured to be provided directly to an application;

intercepting the touch frames and redirecting the intercepted touch frames to a test framework for testing the application, and recording the intercepted and redirected touch frames with the test framework before it is provided to the application;

generating, with the test framework, a log based on the intercepted and redirected touch frames for debugging the application;

identifying, with the test framework, a sequence of input events including gestures, that caused a bug to occur in the application based on the generated log to facilitate elimination of the bug;

processing the intercepted and redirected touch frames into at least one chunk, wherein the at least one chunk comprises an adapted stream of touch frames with a set start frame and end frame, and wherein the at least one chunk represents at least one gesture; and

outputting the at least one chunk to the application in a manner that prevents a user perceptible delay so the application can act on the at least one gesture represented by the at least one chunk.

2. The method of claim 1, wherein the at least one chunk comprises a plurality of chunks, and wherein each of the chunks corresponds to a separate user-performed gesture.

3. The method of claim 2, wherein intra-chunk timing of the touch frames in each chunk is maintained.

4. The method of claim 3, wherein timings between touch frames within each chunk are the same as timings between corresponding touch frames in the generated touch information.

5. The method of claim 2, wherein inter-chunk timings vary from corresponding timings in the generated touch information.

6. The method of claim 2, wherein inter-chunk timings vary from chunk to chunk.

7. The method of claim 1, wherein the intercepting and redirecting the touch frames, the recording the intercepted and redirected touch frames, the processing the intercepted and redirected touch frames into at least one chunk, and the outputting the at least one chunk to the application is performed in less than about 150 milliseconds.

8. The method of claim 1, and further comprising:

outputting the generated touch frames to a touch input stack; and

intercepting and redirecting the touch frames from the touch input stack, and recording the intercepted and redirected touch frames from the touch input stack before it is provided to the application.

9. The method of claim 8, and further comprising:

outputting the at least one chunk to the touch input stack; and

outputting the at least one chunk from the touch input stack to the application.

10. The method of claim 1, wherein the user-performed interaction includes at least one of tapping, touching, touching and holding for a specified time, sliding, waving, flicking or dragging.

11. The method of claim 1, wherein the object is at least one of a bodily appendage, writing utensil, or stylus.

12. A computer-readable storage device storing computer-executable instructions that when executed by at least one processor cause the at least one processor to perform a method of recording user-driven events within a computing system, the method comprising:

receiving at a touch device having a motion-sensitive surface at least one user-performed interaction, which includes user movement of an object over the surface that recognizes such user interaction therewith;

generating touch information with the touch device corresponding to the at least one user-performed interaction, wherein the touch information comprises touch frames, and wherein the touch frames are configured to be provided directly to an application;

intercepting the touch frames and redirecting the intercepted touch frames to a test framework for testing the application, and recording the intercepted and redirected touch frames with the test framework before it is provided to the application;

generating, with the test framework, a log based on the intercepted and redirected touch frames for debugging the application;

identifying, with the test framework, a sequence of input events including gestures, that caused a bug to occur in the application based on the generated log to facilitate elimination of the bug;

processing the intercepted and redirected touch frames into at least one chunk, wherein the at least one chunk comprises an adapted stream of touch frames with a set start frame and end frame, and wherein the at least one chunk represents at least one gesture; and

outputting the at least one chunk to the application in a manner that prevents a user perceptible delay so the application can act on the at least one gesture represented by the at least one chunk.

13. The computer-readable storage device of claim 12, wherein the at least one chunk comprises a plurality of chunks, and wherein each of the chunks corresponds to a separate user-performed gesture.

14. The computer-readable storage device of claim 13, wherein intra-chunk timing of the touch frames in each chunk is maintained.

15. The computer-readable storage device of claim 14, wherein inter-chunk timings vary from corresponding timings in the generated touch frames.

16. The computer-readable storage device of claim 15, wherein inter-chunk timings vary from chunk to chunk.

17. A method of recording user-driven events within a computing system, comprising:

receiving at a motion-sensitive surface a plurality of user-performed interactions, which includes user movement of an object over the surface that recognizes such user interaction therewith;

generating touch frames corresponding to the user-performed interactions, the touch frames configured to be provided directly to an application;

intercepting the touch frames with a touch frame redirect unit and redirecting the intercepted touch frames to a test framework for testing the application, and recording the intercepted and redirected touch frames with the test framework before the touch frames are provided to the application;

generating, with the test framework, a log based on the intercepted and redirected touch frames for debugging the application;

identifying, with the test framework, a sequence of input events including gestures, that caused a bug to occur in the application based on the generated log to facilitate elimination of the bug;

processing the intercepted and redirected touch frames into a plurality of chunks, wherein each chunk comprises an adapted stream of touch frames with a set start frame and end frame, and wherein each chunk corresponds to one of the interactions and represents a gesture; and

outputting the plurality of chunks to the application in a manner that prevents a user perceptible delay so the application can act on the gestures represented by the plurality of chunks.