Speak and touch auto correction interface
First Claim
Patent Images
1. A speak and touch auto correction interface, comprising:
- a key-input module configured to collect one or more key inputs entered by a user using a hardware input mechanism associated with a device, wherein a key input is associated with one of a plurality of classification types and wherein one classification type includes an end of utterance indicator that is not a letter;
an audio-input module configured to collect, in parallel with the key inputs, one or more speech samples spoken by the user using a hardware audio unit associated with the device;
a multimodal module that dynamically reduces a vocabulary as key inputs are entered, but waits for performing speech recognition until the end of word indicator is entered, the vocabulary is dynamically reduced based on the key inputs and on the classification type associated with the key inputs, and, upon receiving the end of word indicator, the multimodal module obtains and stores the key inputs and an utterance detected segment from the one or more speech samples, wherein the utterance detected segment comprises speech samples spoken by the user corresponding to the word, and performs speech recognition when the end of word indicator is received on the utterance detected segment using a current state of the dynamically reduced vocabulary, wherein if the key inputs includes at least one letter, the multimodal module applies an ambiguity filter when dynamically reducing the vocabulary, the ambiguity filter reflecting ambiguities caused by one or more potential typing errors, and wherein if the key inputs includes a symbol, the multimodal module dynamically activates an associated vocabulary associated with the symbol, the associated vocabulary becomes the vocabulary that undergoes dynamic reduction, the symbol is not letters and is not the end of word indicator.
0 Assignments
0 Petitions
Accused Products
Abstract
The disclosure describes an overall system/method for developing a “speak and touch auto correction interface” referred to as STACI which is far more superior to existing user interfaces including the widely adopted qwerty. Using STACI a user speaks and types a word at the same time. The redundant information from the two modes, namely speech and the letters typed, enables the user to sloppily and partially type the words. The result is a very fast and accurate enhanced keyboard interface enabling document production on computing devices like phones and tablets.
-
Citations
22 Claims
-
1. A speak and touch auto correction interface, comprising:
-
a key-input module configured to collect one or more key inputs entered by a user using a hardware input mechanism associated with a device, wherein a key input is associated with one of a plurality of classification types and wherein one classification type includes an end of utterance indicator that is not a letter; an audio-input module configured to collect, in parallel with the key inputs, one or more speech samples spoken by the user using a hardware audio unit associated with the device; a multimodal module that dynamically reduces a vocabulary as key inputs are entered, but waits for performing speech recognition until the end of word indicator is entered, the vocabulary is dynamically reduced based on the key inputs and on the classification type associated with the key inputs, and, upon receiving the end of word indicator, the multimodal module obtains and stores the key inputs and an utterance detected segment from the one or more speech samples, wherein the utterance detected segment comprises speech samples spoken by the user corresponding to the word, and performs speech recognition when the end of word indicator is received on the utterance detected segment using a current state of the dynamically reduced vocabulary, wherein if the key inputs includes at least one letter, the multimodal module applies an ambiguity filter when dynamically reducing the vocabulary, the ambiguity filter reflecting ambiguities caused by one or more potential typing errors, and wherein if the key inputs includes a symbol, the multimodal module dynamically activates an associated vocabulary associated with the symbol, the associated vocabulary becomes the vocabulary that undergoes dynamic reduction, the symbol is not letters and is not the end of word indicator. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 19, 20)
-
-
9. A computing device configured to provide a multimodal interface for entering input, the computing device comprising:
-
a hardware input mechanism; a hardware audio unit; a computer storage medium storing computer-readable components comprising computer-readable instructions; a processor configured to execute the computer-readable instructions, the computer-readable components comprising; a key-input module configured to collect one or more key inputs entered by a user using the hardware input mechanism; an audio-input module configured to collect, in parallel with the key inputs, one or more speech samples spoken by the user using the hardware audio unit; a multimodal module that dynamically reduces a vocabulary as key inputs are entered, but waits for performing speech recognition until an end of utterance indicator is entered, the vocabulary is dynamically reduced based on the key inputs and on a classification type associated with the key inputs, wherein one classification type includes the end of word indicator that is not a letter, and, upon receiving the end of word indicator, obtains and stores the key inputs and an utterance detected segment from the one or more speech samples, wherein the utterance detected segment comprises speech samples spoken by the user corresponding to the word, and performs speech recognition when the end of word indicator is received on the utterance detected segment using a current state of the dynamically reduced vocabulary. - View Dependent Claims (10, 11, 12, 13, 14, 15, 16, 17, 18, 21, 22)
-
Specification