Gesture Recognition Training

• US 20120131513A1
• Filed: 11/19/2010
• Published: 05/24/2012
• Est. Priority Date: 11/19/2010
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Ansell, Peter John
• Application Number:
  US12/950,551
• Publication Number:
  US 20120131513A1
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

Gesture recognition training is described. In an example, a gesture recognizer is trained to detect gestures performed by a user on an input device. Example gesture records, each showing data describing movement of a finger on the input device when performing an identified gesture are retrieved. A parameter set that defines spatial triggers used to detect gestures from data describing movement on the input device is also retrieved. A processor determines a value for each parameter in the parameter set by selecting a number of trial values, applying the example gesture records to the gesture recognizer with each trial value to determine a score for each trial value, using the score for each trial value to estimate a range of values over which the score is a maximum, and selecting the value from the range of values.

Claims

1. A computer-implemented method of training a gesture recognizer to detect gestures performed by a user on an input device, the method comprising:

loading, from a storage device, a plurality of example gesture records, each comprising data describing movement of at least one digit on the input device when performing an identified gesture;

loading, from the storage device, a parameter set that defines spatial triggers used to detect gestures from data describing movement of at least one digit on the input device; and

determining, at a processor, a value for each parameter in the parameter set by;

selecting a plurality of trial values;

applying the example gesture records to the gesture recognizer with each trial value to determine a score for each trial value;

using the score for each trial value to estimate a range of values over which the score is a maximum; and

selecting the value from the range of values.

2. A method according to claim 1, wherein the step of using the score for each trial value to estimate a range of values over which the score is a maximum comprises:

determining the extent of a maximum-score plateau by using the scores for each trial value to estimate values corresponding to upper and lower edges of the plateau, and selecting the range of values from the plateau.

3. A method according to claim 1, wherein the step of determining a value for each parameter in the parameter set further comprises, prior to selecting a plurality of trial values:

selecting an initial value;

applying the example gesture records to the gesture recognizer with the initial value to determine a score for the initial value;

setting a plateau height variable to the score for the initial value;

setting a first inside edge variable and a second inside edge variable to the initial value;

setting a first outside edge variable to a predefined minimum value for the parameter; and

setting a second outside edge variable to a predefined maximum value for the parameter.

4. A method according to claim 3, wherein the step of using the score for each trial value to estimate a range of values over which the score is a maximum comprises:

determining whether the score for an associated trial value is greater than the plateau height variable, and, if so, setting the first inside edge variable and the second inside edge variable to the associated trial value.

5. A method according to claim 3, wherein the step of using the score for each trial value to estimate a range of values over which the score is a maximum comprises:

determining whether the score for an associated trial value is equal to the plateau height variable, and, if so, selecting one of the first or second inside edge variables, and setting it to the associated trial value.

6. A method according to claim 3, wherein the step of using the score for each trial value to estimate a range of values over which the score is a maximum comprises:

determining whether the score for an associated trial value is less than the plateau height variable and the associated trial value is between either the first inside edge variable and the first outside edge variable, or the second inside edge variable and the second outside edge variable, and, if so, selecting one of the first or second outside edge variables, and setting it to the associated trial value.

7. A method according to claim 3, wherein the step of using the score for each trial value to estimate a range of values over which the score is a maximum comprises:

determining whether the score for an associated trial value is less than the plateau height variable and the associated trial value is between the first inside edge variable and the second inside edge variable, and, if so, selecting one of the first or second inside edge variables and resetting it to a previous value.

8. A method according to claim 3, wherein the step of determining a value for each parameter in the parameter set further comprises:

repeating the steps of selecting a plurality of trial values, applying the example gesture records, and using the score for each trial value, until the difference between the first inside edge variable and the first outside edge variable is less than a predefined threshold, and the difference between the second inside edge variable and the second outside edge variable is less than a predefined threshold.

9. A method according to claim 1, wherein the parameter set that defines spatial triggers comprises parameters defining a plurality of regions corresponding to zones on a touch-sensitive portion of the input device, wherein each region in the plurality of regions is associated with a distinct set of gestures that can be initiated from that region.

10. A method according to claim 1, wherein the parameter set that defines spatial triggers comprises parameters defining a plurality of threshold vectors, wherein each threshold vector is positioned relative to a start location of a gesture, and a gesture associated with a given threshold vector is triggered when movement of a digit on the input device crosses the given threshold vector.

11. A method according to claim 1, wherein the step of selecting a plurality of trial values comprises selecting a plurality of values spaced substantially evenly between a predefined minimum and maximum for the parameter.

12. A method according to claim 3, wherein the step of selecting a plurality of trial values comprises:

selecting a first trial value between the first inside edge variable and the first outside edge variable; and

selecting a second trial value between the second inside edge variable and the second outside edge variable.

13. A method according to claim 1, wherein the step of applying the example gesture records to the gesture recognizer with each trial value to determine a score for each trial value comprises, for each trial value:

i) selecting an example gesture record from the plurality of example gesture records;

ii) passing the example gesture record through the gesture recognizer with the trial value;

iii) comparing the gesture recognizer output to the identified gesture for the example gesture record;

iv) reading a weighting factor the identified gesture and adding the weighting factor to the score if the gesture recognizer output and the identified gesture match; and

repeating steps i) to iv) for each example gesture record in the plurality of example gesture records.

14. A method according to claim 1, wherein the step of selecting a value from the a range of values comprises:

selecting a minimum value from the range as the value;

selecting a maximum value from the range as the value;

or selecting a mid-point value from the range as the value;

15. A method according to claim 1, further comprising repeating the step of determining a value for each parameter in the parameter set, until the value for each parameter differs by less than a predefined threshold between consecutive repetitions.

16. A method according to claim 1, wherein the step of determining is performed for each parameter in a first sequence, and the method further comprises repeating the step of determining in an opposite sequence to the first sequence.

17. A computer system for training a gesture recognizer to detect gestures performed by a user on an input device, comprising:

a memory arranged to store a parameter set that defines spatial triggers used to detect gestures from data describing movement of at least one digit on the input device, and a plurality of example gesture records, each comprising pre-recorded data describing movement of at least one digit on the input device when performing an identified gesture; and

a processor executing an optimization engine arranged to determine a value for each parameter in the parameter set, wherein the optimization engine is configured to;

select a plurality of trial values;

retrieve the example gesture records from the memory;

apply the example gesture records to the gesture recognizer with each trial value to determine a score for each trial value;

use the score for each trial value to estimate a range of values over which the score is a maximum; and

select the value from the range of values and store the value in the parameter set at the memory.

18. A computer system according to claim 17, wherein the input device is a multi-touch mouse device.

19. A computer system according to claim 17, further comprising an input interface arranged to receive data from the input device, the data describing movement of at least one digit of a user on a touch-sensitive portion of the input device, and wherein the processor further executes a gesture recognition engine arranged to compare the data from the input device to the spatial triggers defined in the parameter set to detect a gesture applicable to the data, and execute a command associated with the gesture detected.

20. One or more tangible device-readable media with device-executable instructions that, when executed by a computing system, direct the computing system to perform steps comprising:

loading, from a memory, a plurality of example gesture records, each comprising pre-recorded data describing movement of at least one digit of one or more users on a touch-sensitive portion of a mouse device when performing an identified gesture;

loading, from the memory, a gesture recognizer parameter set that defines a plurality of regions corresponding to zones on the touch-sensitive portion of the mouse device, wherein each region in the plurality of regions is associated with a distinct set of gestures that can be initiated from that region;

determining a value for each parameter in the parameter set by;

selecting a plurality of trial values;

applying the example gesture records to the gesture recognizer with each trial value to determine a score for each trial value;

using the score for each trial value to estimate a range of values over which the score is a maximum; and

selecting the value from the range of values; and

storing the value for each parameter in the parameter set at the memory.