RECOGNIZING INPUT GESTURES
First Claim
1. At a computer system including a multi-touch input display surface, a method for training the computer system to recognize input gesture data entered at the multi-touch input display surface as a corresponding specified input pattern, the method comprising:
- an act of accessing input gesture data for at least one example of a specified input pattern, the input gesture data representing detected contact on the multi-touch input display surface over a period of time, the input gesture data including for at least the one example of the specified input pattern;
first direction movement data, the first direction movement data indicating the position of detected contact on the multi-touch input display surface relative to a first axis over the time period; and
second direction movement data, the second direction movement data indicating the position of detected contact on the multi-touch input display surface relative to a second different axis over the time period;
an act of encoding the input gesture data into an input vector of real values, each real value corresponding to a sample on the first or second axis;
an act of saving an indication that the real values contained in the input vector represent an example of the specified input pattern;
an act of accessing a fully connected neural network, the neural network including input nodes, hidden nodes, and output nodes, the input nodes being fully connected to the hidden nodes and the hidden nodes being fully connected to the output nodes;
an act of creating a training set to supply to the neural network to train the neural network to recognize the specified input pattern, the training set including at least the input vector and a corresponding output vector, the corresponding output vector indicating the desired output of the neural network that is to correspond to the specified input pattern; and
an act of using a back propagation algorithm to train the neural network to recognize the specified pattern of input by training the neural network to generate output node values approximating the corresponding output vector in response to receiving the input vector.
2 Assignments
0 Petitions
Accused Products
Abstract
The present invention extends to methods, systems, and computer program products for recognizing input gestures. A neural network is trained using example inputs and backpropagation to recognize specified input patterns. Input gesture data is representative of movements in contact on a multi-touch input display surface relative to one or more axes over time. Example inputs used for training the neural network to recognize a specified input pattern can be created from sampling input gesture data for example input gestures known to represent the specified input pattern. Trained neural networks can subsequently be used to recognize input gestures that are similar to known input gestures as the specified input pattern corresponding to the known input gestures.
-
Citations
20 Claims
-
1. At a computer system including a multi-touch input display surface, a method for training the computer system to recognize input gesture data entered at the multi-touch input display surface as a corresponding specified input pattern, the method comprising:
-
an act of accessing input gesture data for at least one example of a specified input pattern, the input gesture data representing detected contact on the multi-touch input display surface over a period of time, the input gesture data including for at least the one example of the specified input pattern; first direction movement data, the first direction movement data indicating the position of detected contact on the multi-touch input display surface relative to a first axis over the time period; and second direction movement data, the second direction movement data indicating the position of detected contact on the multi-touch input display surface relative to a second different axis over the time period; an act of encoding the input gesture data into an input vector of real values, each real value corresponding to a sample on the first or second axis; an act of saving an indication that the real values contained in the input vector represent an example of the specified input pattern; an act of accessing a fully connected neural network, the neural network including input nodes, hidden nodes, and output nodes, the input nodes being fully connected to the hidden nodes and the hidden nodes being fully connected to the output nodes; an act of creating a training set to supply to the neural network to train the neural network to recognize the specified input pattern, the training set including at least the input vector and a corresponding output vector, the corresponding output vector indicating the desired output of the neural network that is to correspond to the specified input pattern; and an act of using a back propagation algorithm to train the neural network to recognize the specified pattern of input by training the neural network to generate output node values approximating the corresponding output vector in response to receiving the input vector. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13)
-
-
11. The method as recited in 10, further comprising:
-
an act of continuing to calculate further error values and propagate the further error values backwards from the plurality of output nodes into the neural network to further adjust link weights within the neural network until adjusted link weights result in a specified output node value that is least the specified threshold value; and an act of, upon detecting that the specified output node value is at least the specified threshold value, indicating that the neural network is trained to recognize the specified symbol.
-
-
14. At a computer system including a multi-touch input display surface, a method for recognizing input gesture data entered at the multi-touch input display surface as a specified symbol, the method comprising:
-
an act of accessing input gesture data representing detected contact on the multi-touch input display surface over a period of time, the input gesture data including at least; first direction movement data, the first direction movement data indicating the position of detected contact on the multi-touch input display surface relative to a first axis over the time period; and second direction movement data, the second direction movement data indicating the position of detected contact on the multi-touch input display surface relative to a second different axis over the time period; an act of taking a plurality of samples of each of the first direction movement data and the second direction movement data at a plurality of designated intervals between the beginning of the time period and the end of the time period; an act of submitting the plurality of samples to a corresponding plurality of input nodes of a neural network, the neural network having previously trained link weights from the input nodes to a plurality of hidden nodes and the neural network having previously trained link weights from the plurality of hidden nodes to a plurality of output nodes, each output node assigned to a specified symbol such that an output node being activated to a specified threshold value is indicative of the neural network recognizing input gesture data as the specified symbol; an act of the neural network processing the plurality of samples based on the previously trained link weights to activate values at each of the plurality of output nodes; an act of determining that the activated value at the specified output node assigned to the specified symbol is at least the specified threshold value; and an act of indicating that the specified symbol has been recognized from the input gesture data. - View Dependent Claims (15, 16, 17, 18, 19)
-
-
20. A computer program product for use at a computer system including a multi-touch input display surface, the computer program product form implementing a method for training the computer system to recognize input gesture data entered at the multi-touch input display surface as a corresponding specified input pattern, the computer program product comprising one or more computer-readable media having stored thereon computer-executable instructions that, when executed at a processor, cause the computer system to perform the method, including performing the following:
-
access input gesture data for at least one example of a specified input pattern, the input gesture data representing detected contact on the multi-touch input display surface over a period of time, the input gesture data including for at least the one example of the specified input pattern; first direction movement data, the first direction movement data indicating the position of detected contact on the multi-touch input display surface relative to a first axis over the time period; and second direction movement data, the second direction movement data indicating the position of detected contact on the multi-touch input display surface relative to a second different axis over the time period; encode the input gesture data into an input vector of real values, each real value corresponding to a sample on the first or second axis; save an indication that the real values contained in the input vector represent an example of the specified input pattern; access a fully connected neural network, the neural network including input nodes, hidden nodes, and output nodes, the input nodes being fully connected to the hidden nodes and the hidden nodes being fully connected to the output nodes; create a training set to supply to the neural network to train the neural network to recognize the specified input pattern, the training set including at least the input vector and a corresponding output vector, the corresponding output vector indicating the desired output of the neural network that is to correspond to the specified input pattern; and use a back propagation algorithm to train the neural network to recognize the specified pattern of input by training the neural network to generate output node values approximating the corresponding output vector in response to receiving the input vector.
-
Specification