Speech and gesture recognition enhancement

• US 9,093,072 B2
• Filed: 07/20/2012
• Issued: 07/28/2015
• Est. Priority Date: 07/20/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Bathiche, Steven, Gupta, Anoop
• Primary Examiner(s): GIESY, ADAM
• Application Number:
  US13/554,513
• Publication Number:
  US 20140022184A1
• Time in Patent Office: 1,103 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features

Abstract

The recognition of user input to a computing device is enhanced. The user input is either speech, or handwriting data input by the user making screen-contacting gestures, or a combination of one or more prescribed words that are spoken by the user and one or more prescribed screen-contacting gestures that are made by the user, or a combination of one or more prescribed words that are spoken by the user and one or more prescribed non-screen-contacting gestures that are made by the user.

Claims

1. A computer-implemented process for enhancing the recognition of user input to a voice-enabled and touch-enabled computing device, comprising:

the computing device receiving the user input which is either speech comprising one or more words which are spoken by the user, or handwriting data comprising a series of characters which are handwritten by the user making screen-contacting gestures;

the computing device using a user-specific supplementary data context to narrow a vocabulary of a user input recognition subsystem and reduce the size of the vocabulary, wherein the user input recognition subsystem is a speech recognition subsystem whenever the user input is speech, and the user input recognition subsystem is a handwriting recognition subsystem whenever the user input is handwriting data; and

the computing device using the user input recognition subsystem and said narrowed vocabulary to translate the user input into recognizable text that forms either a word or word sequence which is predicted by the user input recognition subsystem to correspond to the user input, wherein said narrowed vocabulary serves to maximize the accuracy of said translation.

2. The process of claim 1, wherein the process action of using a user-specific supplementary data context to narrow a vocabulary of a user input recognition subsystem comprises the actions of:

analyzing the user-specific supplementary data context in order to learn a context-specific vocabulary; and

using the context-specific vocabulary to narrow the vocabulary of the user input recognition subsystem.

3. The process of claim 2, wherein the user-specific supplementary data context comprises the content of an online document that the user is currently working on, and the context-specific vocabulary comprises a current document vocabulary.

4. The process of claim 2, wherein the user-specific supplementary data context comprises the content of search results for an online search that the user performed, and the context-specific vocabulary comprises a search results vocabulary.

5. The process of claim 2, wherein the user-specific supplementary data context comprises tasks that are currently assigned to the user, and the context-specific vocabulary comprises a current tasks vocabulary.

6. The process of claim 2, wherein either,

the user-specific supplementary data context comprises calendar data for the user which is associated with an activity in which the user is currently involved, and the context-specific vocabulary comprises a current activity vocabulary, or

the user-specific supplementary data context comprises contacts data for the user, and the context-specific vocabulary comprises a contacts vocabulary.

7. The process of claim 2, wherein the user-specific supplementary data context comprises the content of one or more of messages that the user previously sent, or messages that the user previously received, and the context-specific vocabulary comprises a messages vocabulary.

8. The process of claim 2, wherein the user-specific supplementary data context comprises the content of online documents that the user previously stored, and the context-specific vocabulary comprises a previous documents vocabulary.

9. The process of claim 2, wherein the user-specific supplementary data context comprises the content of speech-based audio recordings that the user previously stored, and the context-specific vocabulary comprises a previous audio vocabulary.

10. The process of claim 2, wherein the user-specific supplementary data context comprises one or more of who the user previously sent messages to, or who the user previously received messages from, and the context-specific vocabulary comprises a recipient/sender vocabulary.

11. The process of claim 1, wherein the process action of using a user-specific supplementary data context to narrow a vocabulary of a user input recognition subsystem comprises the actions of:

narrowing the user-specific supplementary data context to comprise just data that is associated with one or more prescribed attributes;

analyzing said narrowed data context in order to learn a narrowed context-specific vocabulary; and

using the narrowed context-specific vocabulary to narrow the vocabulary of the user input recognition subsystem.