Combined speech and touch input for observation symbol mappings

• US 9,519,353 B2
• Filed: 03/30/2009
• Issued: 12/13/2016
• Est. Priority Date: 03/30/2009
• Status: Active Grant

• Current Assignee: Symbol Technologies, LLC (Zebra Technologies Corporation)
• Original Assignee: Symbol Technologies, LLC (Zebra Technologies Corporation)
• Inventors: Willins, Bruce
• Primary Examiner(s): He, Jialong
• Application Number:
  US12/413,683
• Publication Number:
  US 20100250248A1
• Time in Patent Office: 2,815 Days
• Field of Search: 704/3, 704/256, 704/270
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs
• CPC Class Codes: G10L 15/142 Hidden Markov Models [HMMs]
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/226 using non-speech characteristics
• CPC Class Codes: H04M 1/271 controlled by voice recognition
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00

Abstract

The invention relates to systems and or methodologies for enabling combined speech and touch inputs for observation symbol mappings. More particularly, the current innovation leverages the commonality of touch screen display text entry and speech recognition based text entry to increase the speed and accuracy of text entry via mobile devices. Touch screen devices often contain small and closely grouped keypads that can make it difficult for a user to press the intended character, by combining touch screen based text entry with speech recognition based text entry the aforementioned limitation can be overcome efficiently and conveniently.

Claims

1. A system for mobile device text entry, comprising:

memory storing computer readable instructions configured for executing a plurality of components; and

a processor, coupled to the memory and operable to execute the instructions, the plurality of components comprising;

a text recognition component that obtains a set of inputs from a text entry component, determines a set of digitized coordinates for the set of inputs, and produces a first set of observations based on a set of possible intended user inputs and the digitized coordinates, wherein the text entry component is a touch screen display;

a voice recognition component that obtains audio data from an audio capture component, and produces a second set of observations based on the audio data, wherein the audio capture component obtains the audio data concurrently as the text entry component obtains the set of inputs; and

a predictive component that combines the first set of observations and the second set of observations into a concatenated observation sequence, and generates a textual output based on the concatenated observation sequence, wherein the generated textual output includes a first predetermined character that is predicted by the predictive component based on (a) a substitute character received by the text entry component, the substitute character is different from the first predetermined character, and (b) pronunciation of the first predetermined character received by the audio capture component.

2. The system of claim 1, wherein the touch screen display provides a virtual set of keys for a user to input a set of characters.

3. The system of claim 1, wherein the user triggers an input via the text entry component that enables a secondary input mode, wherein the secondary input mode includes speech recognition.

4. The system of claim 1 wherein the predictive component generates the textual output based on the sets in the concatenated observation sequence by employing at least one of a Hidden Markov model and a Viterbi decoder that processes the concatenated observation sequence.

5. The system of claim 1 wherein the first predetermined character is a special character.

6. A method for mobile device text entry, comprising:

generating a first set of observations based on a set of possible intended user inputs and a determined set of digitized coordinates associated with the set of possible intended user inputs obtained via a text input device, wherein the text input device is a touch screen display;

generating a second set of observations based on user inputs obtained via an audio input device, wherein user inputs via the text input device and the audio input device are obtained concurrently;

combining the first set of observations and the second set of observations to produce a set of combined observations; and

producing a textual output by processing the set of combined observations, wherein the produced textual output includes a first predetermined character that is predicted based on (a) a substitute character received by the text input device, the substitute character is different from the first predetermined character, and (b) pronunciation of the first predetermined character received by the audio input device.

7. The method of claim 6, wherein the touch screen display provides a graphical key wedge which enables users to select at least one character by touching the character on the display.

8. The method of claim 6, further comprising triggering an input that enables an alternative input mode.

9. The method of claim 8, wherein the alternative input mode includes speech recognition.

10. The method of claim 8, further comprising enabling the user to access inputs via the alternative input mode that are not currently available.

11. The method of claim 6 wherein the textual output is produced by processing the set of combined observations employing at least one of a Hidden Markov model and a Viterbi decoder.

12. The method of claim 6 wherein the first predetermined character is a special character.

13. A system for mobile device text entry, comprising:

means for generating a first set of observations based on a set of possible intended user inputs and a determined set of digitized coordinates associated with the set of possible intended user inputs obtained via a touch screen display that provides a graphical key interface, wherein users select at least one unintended character by pressing at a location on the graphical key interface proximate to the location of the unintended character;

means for generating a second set of observations based on user inputs obtained via an audio capture device, wherein user inputs via the touch screen display and the audio capture device are obtained concurrently; and

means for combining the first set of observations and the second set of observations to produce a set of concatenated observations; and

means for producing a textual output based on the set of concatenated observations, wherein the produced textual output includes a first predetermined character that is predicted based on (a) a substitute character received by the graphical key interface, the substitute character is different from the first predetermined character, and (b) pronunciation of the first predetermined character received by the audio capture device.

14. The system of claim 13, further comprising means for triggering an input that enables an alternative input mode, wherein the alternative input mode provides the user with input options that are not presently available.

15. The system of claim 13 wherein the textual output is produced by using at least one of a Viterbi decoder and a Hidden-Markov Model.

16. The system of claim 13 wherein the first predetermined character is a special character.