Method and apparatus utilizing voice input to resolve ambiguous manually entered text input

• US 20060190256A1
• Filed: 02/07/2006
• Published: 08/24/2006
• Est. Priority Date: 12/04/1998
• Status: Active Grant

• Current Assignee: Cerence Operating Company (Cerence Inc.)
• Original Assignee: Tegic Communications, Inc. (Microsoft Corporation)
• Inventors: Eyraud, Richard, Longe, Michael R., Kay, David Jon, Meurs, Pim Van, Bradford, Ethan, Stephanick, James
• Granted Patent: US 7,720,682 B2
• Time in Patent Office: Days
• US Class Current: 704/252
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs
• CPC Class Codes: G06V 10/987 with the intervention of an operator
• CPC Class Codes: G06V 30/1423 the instrument generating sequences of position coordinates corresponding to handwriting preprocessing or recognising digital ink G06V30/32
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

From a text entry tool, a digital data processing device receives inherently ambiguous user input. Independent of any other user input, the device interprets the received user input against a vocabulary to yield candidates such as words (of which the user input forms the entire word or part such as a root, stem, syllable, affix), or phrases having the user input as one word. The device displays the candidates and applies speech recognition to spoken user input. If the recognized speech comprises one of the candidates, that candidate is selected. If the recognized speech forms an extension of a candidate, the extended candidate is selected. If the recognized speech comprises other input, various other actions are taken.

Claims

1. A digital data processing device programmed to perform operations of resolving inherently ambiguous user input received via a manually operated text entry tool, the operations comprising:

via manually operated text entry tool, receiving ambiguous user input representing multiple different possible combinations of text;

independent of any other user input, interpreting the received user input against a vocabulary to yield multiple candidates of at least one of the following types;

(1) a word of which the user input forms one of;

a root, stem, syllable, affix, (2) a phrase of which the user input forms a word;

(3) a word represented by the user input;

visibly presenting a list of the candidates for viewing by the user;

responsive to the device receiving spoken user input, performing speech recognition of the spoken user input; and

performing one or more actions of a group of actions including;

responsive to the recognized speech comprising an utterance of one of the candidates, providing an output comprising that candidate.

2. The device of claim 1, wherein the group of actions further comprises:

responsive to the recognized speech comprising an extension of a candidate, providing an output comprising the extension of said candidate.

3. The device of claim 1, where the group of actions further comprises at least one of the following:

responsive to the recognized speech comprising a command to expand one of the candidates, searching a vocabulary for entries that include said candidate as a subpart, and visibly presenting one or more entries found by the search;

responsive to the recognized speech forming an expand command, visibly presenting at least one of the following as to one or more candidates in the list;

word completion, affix addition, phrase completion, additional words having the same root as the candidate.

4. The device of claim 1, where the group of actions further comprises:

comparing the list of the candidates with a list of possible outcomes from the speech recognition operation to identify any candidates occurring in both lists;

visibly presenting a list of the identified candidates.

5. The device of claim 1, the group of actions further including:

responsive to recognized speech comprising an utterance potentially pronouncing any of a subset of the candidates, visibly presenting a list of candidates of the subset.

6. The device of claim 1, where the operation of performing speech recognition comprises:

performing speech recognition of the spoken user input utilizing a vocabulary;

redefining the candidates to omit candidates not represented by results of the speech recognition operation;

visibly presenting a list of the redefined candidates.

7. The device of claim 1, where the operation of performing speech recognition comprises:

performing speech recognition of the spoken user input utilizing a vocabulary substantially limited to said candidates.

8. The device of claim 1, the interpreting operation performed such that each candidate begins with letters corresponding to the user input.

9. The device of claim 1, the interpreting operation performed such that a number of the candidates are words including letters representing the user input in other than starting and ending positions in the words.

10. The device of claim 1, the interpreting operation conducted such that the types of candidates further include strings of alphanumeric text.

11. The device of claim 1, the interpreting operation conducted such that the types further include at least one of:

ideographic characters, phrases of ideographic characters.

12. A digital data processing device, comprising:

user-operated means for manual text entry;

display means for visibly presenting computer generated images;

processing means for performing operations comprising;

via the user-operated means, receiving ambiguous user input representing multiple different possible combinations of text;

independent of any other user input, interpreting the received user input against a vocabulary to yield a number of candidates of at least one of the following types;

(1) a word of which the user input forms one of;

a root, stem, syllable, affix, (2) a phrase of which the user input forms a word;

(3) a word represented by the user input;

operating the display means to visibly present a list of the candidates for viewing by the user;

responsive to receiving spoken user input, performing speech recognition of the spoken user input;

performing one or more actions of a group of actions including;

responsive to the recognized speech comprising an utterance of one of the candidates, providing an output comprising that candidate.

13. Circuitry of multiple interconnected electrically conductive elements configured to operate a digital data processing device to perform operations for resolving inherently ambiguous user input received via manually operated text entry tool, the operations comprising:

via manually operated text entry tool, receiving ambiguous user input representing multiple different possible combinations of text;

independent of any other user input, interpreting the received user input against a vocabulary to yield a number of candidates of at least one of the following types;

(1) a word of which the user input forms one of;

a root, stem, syllable, affix, (2) a phrase of which the user input forms a word;

(3) a word represented by the user input;

visibly presenting a list of the candidates for viewing by the user;

responsive to receiving spoken user input, performing speech recognition of the spoken user input;

performing one or more actions of a group of actions including;

responsive to the recognized speech comprising an utterance of one of the candidates, providing an output comprising that candidate.

14. A digital data processing device programmed to perform operations for resolving inherently ambiguous user input received via manually operated text entry tool, the operations comprising:

via manually operated text entry tool, receiving ambiguous user input representing at least one of the following;

handwritten strokes, categories of handwritten strokes, phonetic spelling, tonal input;

interpreting the user input to yield multiple candidates possibly formed by the user input, where each candidate comprises one or more of the following;

one or more ideographic characters, one or more ideographic radicals of ideographic characters;

visibly presenting a list of the candidates for viewing by the user;

responsive to receiving spoken user input, performing speech recognition of the spoken user input;

performing one or more actions of a group of actions including;

responsive to the recognized speech comprising an utterance including pronunciation of one of the candidates, providing an output comprising that candidate.

15. The device of claim 14, where the group of actions further comprises:

responsive to the recognized speech comprising an extension of a candidate, providing an output comprising the extension of said candidate.

16. The device of claim 14, where the group of actions further comprises:

responsive to the recognized speech comprising a command to expand one of the candidates, searching a vocabulary for entries that include said candidate as a subpart, and visibly presenting one or more entries found by the search.

17. The device of claim 14, the group of actions further including:

determining if the recognized speech includes one of the following;

a pronunciation including one of the candidates along with other vocalizations, an expansion of one of the candidates, a variation of one of the candidates;

if so, visibly presenting a corresponding one of at least one of the following;

expansions of the candidate, variations of the candidate.

18. The device of claim 14, where the group of actions further comprises:

comparing a list of the candidates with a list of possible outcomes from the speech recognition operation to identify any candidates occurring in both lists;

visibly presenting a list of the identified candidates.

19. The device of claim 14, the group of actions further including:

responsive to recognized speech comprising an utterance potentially pronouncing any of a subset of the candidates, visibly presenting a list of candidates of the subset.

20. The device of claim 14, the group of actions further including:

responsive to recognized speech comprising a phonetic input exclusively corresponding to a subset of the candidates, visibly presenting a list of candidates of the subset.

21. The device of claim 14, where:

the device further includes digital data storage including at least one data structure including multiple items of phonetic information and cross-referencing each item of phonetic information with one or more ideographic items, each ideographic item including at least one of the following;

one or more ideographic characters, one or more ideographic radicals;

where each item of phonetic information comprises one of the following;

pronunciation of one or more ideographic items, pronunciation of one or more tones associated with one or more ideographic items;

the operation of performing speech recognition of the spoken user input further comprises searching the data structure according to phonetic information of the recognized speech in order to identify one or more cross-referenced ideographic items.

22. The device of claim 14, where the operation of performing speech recognition comprises:

performing speech recognition of the spoken user input utilizing a vocabulary substantially limited to said candidates.

23. A digital data processing device, comprising:

user-operated means for manual text entry;

display means for visibly presenting computer generated images;

processing means for performing operations comprising;

via the user-operated means, receiving ambiguous user input representing at least one of the following;

handwritten strokes, categories of handwritten strokes, phonetic spelling, tonal input;

interpreting the user input to yield multiple candidates possibly formed by the user input, where each candidate comprises one or more of the following;

one or more ideographic characters, one or more ideographic radicals of ideographic characters;

causing the display means to present a list of the candidates for viewing by the user;

responsive to the speech entry equipment receiving spoken user input, performing speech recognition of the spoken user input;

performing one or more actions of a group of actions including;

responsive to the recognized speech comprising an utterance including pronunciation of one of the candidates, providing an output comprising that candidate.

24. Circuitry of multiple interconnected electrically conductive elements configured to operate a digital data processing device to perform operations for resolving inherently ambiguous user input received via manually operated text entry tool, the operations comprising:

via manually operated text entry tool, receiving ambiguous user input representing at least one of the following;

handwritten strokes, categories of handwritten strokes, phonetic spelling, tonal input;

interpreting the user input to yield multiple candidates possibly formed by the user input, where each candidate comprises one or more of the following;

one or more ideographic characters, one or more ideographic radicals of ideographic characters;

visibly presenting a list of the candidates for viewing by the user;

responsive to the speech entry equipment receiving spoken user input, performing speech recognition of the spoken user input;

performing one or more actions of a group of actions including;

responsive to the recognized speech comprising an utterance including pronunciation of one of the candidates, providing an output comprising that candidate.