Intelligent keyboard interface with use of human language processing
First Claim
Patent Images
1. An intelligent keyboard interface having human language processing capability which is responsive to commands designated by voice input and keyboard input to change keypad displays on a keyboard, the intelligent keyboard interface comprising:
- an intelligent module automatically responsive to a combination of voice input and keystroke input to recognize and interpret the combination into command signals via a Hidden Markov model (HMM), the HMM comprising;
a first Hidden Markov model including one or more first states having one or more first arcs for connecting the one or more first states, the one or more first arcs representing output string of voice;
a second Hidden Markov model including one or more second states having one or more second arcs for connecting the one or more second states, the one or more second arcs representing output string of keys;
one of more third arcs connecting the one or more first states with the one or more second states, the one or more third arcs producing an empty output string, the one or more third arcs traversed when an interpretation state between the output string of the voice and the output string of keys are switched; and
one or more display screens embedded in one or more keypads on a keyboard capable of displacing contents responsive to the command signals transmitted from the intelligent module, wherein the displayed contents represent user selectable functionality for performing functions associated with the combination of the voice input and keyboard input.
2 Assignments
0 Petitions
Accused Products
Abstract
An intelligent use-friendly keyboard interface that is easily adaptable for wide variety of functions and features, and also adaptable to reduced size portable computers. Speech recognition and semantic processing for controlling and interpreting multiple symbols are used in conjunction with programmable switches with embedded LCD displays. Hidden Markov models are employed to interpret a combination of voice and keyboard input.
223 Citations
23 Claims
-
1. An intelligent keyboard interface having human language processing capability which is responsive to commands designated by voice input and keyboard input to change keypad displays on a keyboard, the intelligent keyboard interface comprising:
-
an intelligent module automatically responsive to a combination of voice input and keystroke input to recognize and interpret the combination into command signals via a Hidden Markov model (HMM), the HMM comprising;
a first Hidden Markov model including one or more first states having one or more first arcs for connecting the one or more first states, the one or more first arcs representing output string of voice;
a second Hidden Markov model including one or more second states having one or more second arcs for connecting the one or more second states, the one or more second arcs representing output string of keys;
one of more third arcs connecting the one or more first states with the one or more second states, the one or more third arcs producing an empty output string, the one or more third arcs traversed when an interpretation state between the output string of the voice and the output string of keys are switched; and
one or more display screens embedded in one or more keypads on a keyboard capable of displacing contents responsive to the command signals transmitted from the intelligent module, wherein the displayed contents represent user selectable functionality for performing functions associated with the combination of the voice input and keyboard input.
-
-
2. The intelligent keyboard interface as claimed in claim 1, wherein the contents include symbolic icons.
-
3. The intelligent keyboard interface as claimed in claim 1, wherein the contents include alphanumeric text.
-
4. The intelligent keyboard interface as claimed in claim 1, wherein the intelligent module includes an automatic speech recognition module for interpreting voice input.
-
5. The intelligent keyboard interface as claimed in claim 4, wherein the intelligent module further includes a semantic language processing module for interpreting combination of voice input and keyboard input.
-
6. The intelligent keyboard interface as claimed in claim 5, wherein the intelligent module further includes a synchronization module for synchronizing inputs forming a command.
-
7. The intelligent keyboard interface as claimed in claim 6, wherein the synchronization module is enabled to synchronize temporally overlapping inputs forming a command in accordance with time intervals in which the inputs of overlap.
-
8. The intelligent keyboard interface as claimed in claim 1, wherein the voice input is a phoneme.
-
9. A method for automatically changing contents of display device integrated onto a keyboard including a plurality of keypads, the method comprising:
-
grouping one or more inputs into a first continuous segment representing a command phrase, said one or more inputs including a combination of voice input and keyboard input that are recognized and interpreted into the command phrase via a Hidden Markov model (HMM), the HMM comprising;
first Hidden Markov model including one or more first states having one or more first arcs for connecting the one or more first states, the one or more first arcs representing output string of voice;
a second Hidden Markov model including one or more second states having one or more second arcs for connecting the one or more second states, the one or more second arcs representing output string of keys;
one or more third arcs connecting the one or more first states with the one or more second states, the one or more third arcs producing an empty output string, the one or more third arcs traversed when an interpretation state between the output string of the voice and the output string of keys are switched; and
;
comparing the first continuous segment with a prestored key sequence-command pairs to determine an action which matches the command phrase designating contents to display, wherein the displayed contents represent user selectable functionality for performing functions associated with the command phrase.
-
-
10. The method for automatically changing contents of display device integrated onto a keyboard including a plurality of keypads as claimed in claim 9, the method further comprising:
appending a second continuous segment accepted as input to the first continuous segment, if no match is found in the step of comparing.
-
11. The method for automatically changing contents of display device integrated onto a keyboard including a plurality of keypads as claimed in claim 10, the method further comprising:
repeating he steps of appending and matching until a match is found.
-
12. The method for automatically changing contents of display device integrated onto a keyboard including a plurality of keypads as claimed in claim 10, the method further comprising:
repeating the steps of appending and matching until a match is found or a number of appended segments exceed a predetermined number.
-
13. The method for automatically changing contents of keyboard display as claimed in claim 9, wherein the step of grouping further includes determining which one or more inputs are associated with each other based on a predetermined time interval occurring between each input.
-
14. The method for interpreting input to automatically change contents of keyboard display as claimed in claim 9, wherein said one or more input includes one or more keystroke input.
-
15. The method for automatically changing contents of keyboard display as claimed in claim 9, wherein said one or more input includes a combination of voice and one or more keystroke inputs.
-
16. The method for automatically changing contents of keyboard display as claimed in claim 15, wherein if the voice and keystroke inputs temporally overlap, the step of grouping further includes synchronizing the temporally overlapping inputs for representing the command phrase in accordance with time intervals in which the inputs overlap.
-
17. The method for automatically changing contents of keyboard display as claimed in claim 9, wherein the method further includes:
changing display contents of keyboard display based on the action matching the command phrase.
-
18. The method for automatically changing contents of keyboard display as claimed in claim 9, wherein the method further includes:
interpreting the first continuous segment employing semantic language processor before the step of comparing.
-
19. The method for automatically changing contents of keyboard display as claimed in claim 9, wherein the method further includes:
interpreting the first continuous segment employing automatic speech recognition processor before the step of comparing.
-
20. The method for automatically changing contents of display device integrated onto a keyboard including a plurality of keypads as claimed in claim 9, wherein the voice input is a phoneme.
-
21. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for automatically changing contents of keyboard display, the method steps comprising:
-
grouping one or more inputs into a first continuous segment representing a command phrase, said one or more inputs including a combination of voice input and keyboard input that are recognized and interpreted into the command phrase via a Hidden Markov model (HMM), the HMM comprising;
a first Hidden Markov model including one or more first states having one or more first arcs for connecting the one or more first states, the one or more first arcs representing output string of voice;
a second Hidden Markov model including one or more second states having one or more second arcs for connecting the one or more second states, the one or more second arcs representing output string of keys;
one or more third arcs connecting the one or more first states with the one or more second states, the one or more third arcs producing an empty output string, the one or more third arcs traversed when an interpretation state between the output string of the voice and the output string of keys are switched; and
comparing the first continuous segment with a prestored key sequence-command pairs to determine an action which matches the command phrase designating contents to display, wherein the displayed contents represent user selectable functionality for performing functions associated with the command phrase.
-
-
22. The program storage device as claimed in claim 21, the method steps further including:
concatenating a second continuous segment accepted as input to the first continuous segment, if no match is found in the step of comparing.
-
23. The program storage device readable by machine as claimed in claim 21, wherein the voice input is a phoneme.
Specification