Synthetic Gesture Trace Generator

• US 20120092286A1
• Filed: 10/19/2010
• Published: 04/19/2012
• Est. Priority Date: 10/19/2010
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Benko, Hrvoje, O'Prey, Christopher Jozef, Sol, Alisson Augusto Souza
• Application Number:
  US12/907,588
• Publication Number:
  US 20120092286A1
• Time in Patent Office: Days
• US Class Current: 345/174
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/044 by capacitive means
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A synthetic gesture trace generator is described. In an embodiment, a synthetic gesture trace is generated using a gesture synthesizer which may be implemented in software. The synthesizer receives a number of inputs, including parameters associated with a touch sensor to be used in the synthesis and a gesture defined in terms of gesture components. The synthesizer breaks each gesture component into a series of time-stamped contact co-ordinates at the frame rate of the sensor, with each time-stamped contact co-ordinate detailing the position of any touch events at a particular time. Sensor images are then generated from the time-stamped contact co-ordinates using a contact-to-sensor transformation function. Where there are multiple simultaneous contacts, there may be multiple sensor images generated having the same time-stamp and these are combined to form a single sensor image for each time-stamp. This sequence of sensor images is formatted to create the synthetic gesture trace.

Claims

1. A computer-implemented method of generating a synthetic gesture trace, the method comprising:

receiving, at a computer, gesture data defining a gesture to be synthesized with reference to one or more gesture components;

receiving, at the computer, parameters associated with a capacitive touch sensor;

loading data for each of the one or more gesture components referenced in the gesture data;

generating a series of time-stamped contact co-ordinates for each gesture component at a frame rate of the sensor;

constructing a sensor image for each time-stamp based on the series of time-stamped contact co-ordinates for each gesture component; and

outputting a synthetic gesture trace comprising the sensor images.

2. A computer-implemented method according to claim 1, wherein the synthetic gesture trace is output to a gesture recognizer.

3. A computer-implemented method according to claim 1, further comprising:

adding at least one blank sensor image after all the constructed sensor images, prior to outputting the synthetic gesture trace.

4. A computer-implemented method according to claim 1, wherein constructing a sensor image for each time-stamp based on the series of time-stamped contact co-ordinates for each gesture component comprises:

creating a blank sensor frame for each time-stamp;

selecting a first time-stamped contact co-ordinate from a series of time-stamped contact co-ordinates for a gesture component;

generating a sensor image using a contact-to-sensor function;

adding the generated sensor image to an existing sensor frame associated with the time-stamp of the selected contact co-ordinate;

repeating the generating and adding steps for each time-stamped contact co-ordinate in the series of time-stamped contact co-ordinates for each gesture component.

5. A computer-implemented method according to claim 1, wherein constructing a sensor image for each time-stamp based on the series of time-stamped contact co-ordinates for each gesture component comprises:

selecting a first time-stamped contact co-ordinate from a series of time-stamped contact co-ordinates for a gesture component;

generating a sensor image using a contact-to-sensor function;

adding the sensor image to any existing sensor image associated with the time-stamp of the selected contact co-ordinate;

repeating the generating and adding steps for each time-stamped contact co-ordinate in the series of time-stamped contact co-ordinates for each gesture component;

arranging the generated sensor images into a sequence of images in chronological order; and

inserting blank sensor images within the sequence for any time-stamps without an associated sensor image.

6. A computer-implemented method according to claim 5, wherein the capacitive touch sensor provides a multi-level output of capacitance values and contact-to-sensor function has the form:

y=(1−

(w<sup>−

2</sup>x²)).(1−

(w<sup>−

2</sup>z²)).h

where h is a maximum level of capacitance output by the sensor and w is a maximum width of the function.

7. A computer-implemented method according to claim 1, wherein generating a series of time-stamped contact co-ordinates for each gesture component at a frame rate of the sensor comprises:

generating a series of time-stamped contact co-ordinates for a gesture component;

determining if the gesture component data specifies jitter correction, and if jitter correction is specified in the data, adjusting each contact co-ordinate in the series of time-stamped contact co-ordinates for the gesture component by a random offset; and

repeating the generating, determining and adjusting steps for each gesture component.

8. A computer-implemented method according to claim 1, wherein generating a series of time-stamped contact co-ordinates for each gesture component at a frame rate of the sensor comprises:

generating a series of time-stamped contact co-ordinates for a gesture component;

determining if the gesture component data specifies a curvature correction value, and if a value is specified in the data, adjusting each contact co-ordinate in the series of time-stamped contact co-ordinates to lie on an arc defined by the specified value; and

repeating the generating, determining and adjusting steps for each gesture component.

9. A computer-implemented method according to claim 8, wherein the curvature correction value specifies an offset between a straight line and the arc at a midpoint between a start co-ordinate of the gesture component and an end co-ordinate of the gesture component.

10. A computer-implemented method according to claim 1, wherein the gesture data and the parameters associated with a capacitive touch sensor are received in a single input script.

11. A computer-implemented method according to claim 10, wherein the input script further comprises gesture data defining a second gesture to be synthesized with reference to one or more gesture components and wherein the method is repeated to generate and output a second synthetic gesture trace corresponding to the second gesture.

12. A computer-implemented method according to claim 1, wherein the synthetic gesture trace comprises a binary file of run-length encoded data and the trace is output in a file which further comprises a data block comprising information about the capacitive touch sensor.

13. A computer-implemented method according to claim 1, further comprising:

outputting a data file comprising control instructions for a robot which on execution cause the robot to replicate motion of the gesture.

14. One or more tangible device-readable media with device-executable instructions that, when executed by a computing system, direct the computing system to perform steps comprising:

receiving an input file defining at least one gesture to be synthesized, each gesture comprising at least one gesture component;

accessing a frame rate of a capacitive touch sensor;

accessing data for the at least one gesture component;

calculating a set of time-stamped contact co-ordinates at the frame rate of the sensor for each gesture component;

generating a series of sensor images at the frame rate of the sensor based on the set of time-stamped contact co-ordinates for each gesture component and a contact-to-sensor function;

formatting the series of sensor images to form a synthetic gesture trace; and

outputting the synthetic gesture trace.

15. One or more tangible device-readable media according to claim 14, further comprising device-executable instructions that, when executed by a computing system, direct the computing system to perform steps comprising:

adding at least one empty sensor image onto the end of the series of sensor images, prior to formatting the series of sensor images.

16. One or more tangible device-readable media according to claim 14, wherein formatting the series of sensor images to form a synthetic gesture trace comprises:

creating a series of USB data packets wherein each USB data packet comprises data defining a part or a whole of a sensor image.

17. One or more tangible device-readable media according to claim 14, wherein the input file further comprises the frame rate of the sensor and physical dimensions of the sensor.

18. A method comprising:

receiving, at a computer, an input file comprising parameters associated with a capacitive touch sensor and data defining a gesture in terms of gesture components;

accessing data associated with the gesture components;

dividing each gesture component into a series of time-stamped contact co-ordinates at a frame rate of the touch sensor;

if the input file specifies jitter correction for a gesture component, adjusting the contact co-ordinates in the corresponding series of time-stamped contact co-ordinates by a random offset;

if the input file specifies a non-zero curvature correction parameter, adjusting the contact co-ordinates in the corresponding series of time-stamped contact co-ordinates to lie on an arc defined by the non-zero curvature correction parameter;

automatically constructing a sensor image corresponding to each frame of the touch sensor using the series of time-stamped contact co-ordinates and a contact-to-sensor function;

adding at least one blank sensor image following a last of the constructed sensor images;

formatting the sensor images to form a synthetic gesture trace; and

outputting the synthetic gesture trace for input into a gesture recognizer.

19. A method according to claim 18, wherein the input file further comprises data defining a second gesture in terms of gesture components and wherein the method is repeated to form and output a second synthetic gesture trace for input into a gesture recognizer.

20. A method according to claim 18, wherein the capacitive touch sensor comprises a touch sensor for a multi-touch mouse.