Combined input processing for a computing device

• US 20060290656A1
• Filed: 06/28/2005
• Published: 12/28/2006
• Est. Priority Date: 06/28/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Tian, Ye, Zhou, Jian-Lai, Soong, Frank Kao-Ping
• Granted Patent: US 7,496,513 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

Input is received from at least two different input sources. Information from these sources are combined together to provide a result. In a particular example, input from one source corresponds to potential recognition candidates, and input from another source corresponds to other potential candidates. These candidates are combined together to select a result.

Claims

1. A method of processing information comprising:

receiving input from two or more different input sources; and

combining the input from the two or more different sources to provide a result.

2. The method of claim 1 wherein input from one of the input sources includes handwriting input, speech input, pointer input, keyboard input and gesture input.

3. The method of claim 1 wherein input from the first input source is handwriting input and input from the second input source is speech input.

4. The method of claim 3 and further comprising providing at least one handwriting recognition candidate for the handwriting input and providing at least one speech recognition candidate for the speech input.

5. The method of claim 4 and further comprising assigning a score to the at least one handwriting recognition candidate, assigning a score to the speech recognition candidate, adjusting the score of the handwriting recognition candidate based on the score of the speech recognition candidate and comparing the score of the handwriting recognition candidate to a threshold.

6. The method of claim 1 wherein input from the first input source is keyboard input and the keyboard input is recognized using a reduced keyboard dictionary.

7. The method of claim 1 wherein the input from the first input source is gesture input that is recognized with a gesture recognizer.

8. The method of claim 1 wherein the input from the first input source corresponds to a plurality of potential candidates.

9. The method of claim 8 and further comprising:

selecting one candidate from the plurality of potential candidates as the result based on the input from the second source.

10. The method of claim 1 wherein the input from the first source corresponds to a location on a touch screen.

17. The method of claim 1 wherein the final joint determined results are used to enhance the performance of any of the original input recognizer.

18. The method of claim 1 wherein handwriting, pen, gesture, and keyboard input are used to improve the performance of speech activity detection

19. The method of claim 1 wherein handwriting, pen, gesture, and keyboard input are used to improve the estimation of signal-to-noise ratio for robust speech recognition.

11. A method of processing information comprising:

receiving a first input corresponding to a first plurality of potential candidates;

receiving a second input corresponding to a second plurality of potential candidates; and

combining the first input and the second input to select one of the plurality of potential candidates to produce a result.

12. The method of claim 11 wherein at least one of the first input and the second input includes one of handwriting input, pointer input, keyboard input, speech input and gesture input.

13. The method of claim 12 and further comprising using a recognizer to recognize at least one of the first input and the second input and produce at least one of the first and second plurality of potential candidates.

14. The method of claim 11 wherein the first input is handwriting input and the second input is speech input.

15. The method of claim 14 and further comprising providing at least one handwriting recognition candidate for the handwriting input.

16. The method of claim 11 wherein the first input is received at a particular time and the second input includes speech data and wherein the method further comprises performing speech recognition based on the particular time.

20. The method of claim 11 wherein at least one of first and second plurality of potential candidates the includes one of a word-graph and an N-Best list.