Text entry at electronic communication device

• US 9,244,906 B2
• Filed: 06/21/2013
• Issued: 01/26/2016
• Est. Priority Date: 06/21/2013
• Status: Active Grant

• Current Assignee: Malikie Innovations Limited (Key Patent Innovations Limited)
• Original Assignee: Blackberry Limited
• Inventors: Houache, Mohammed
• Primary Examiner(s): AZAD, ABUL K
• Application Number:
  US13/923,464
• Publication Number:
  US 20140379325A1
• Time in Patent Office: 949 Days
• Field of Search: 704/9, 704/235, 704251-257, 704/270
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/226 using non-speech characteristics

Abstract

Typed input is received at a text field from a keyboard, such as a virtual keyboard displayed at an electronic communication device. Text prediction using the typed input can be performed to obtain at least one text-predicted candidate word and an associated confidence value. Speech recognition can be performed on audio input received via a microphone to obtain at least one speech-recognized candidate word and an associated confidence value, which can be adjusted based on the recency of the audio input. A candidate word having a highest confidence value can be selected from the text-predicted and speech-recognized candidate words for display as a suggestion to the user for selection by the user. The suggested candidate word can be displayed on a fret of the virtual keyboard.

Claims

1. A method of facilitating text entry at an electronic communication device, the method comprising:

receiving typed input at a text field from a keyboard of the electronic communication device;

receiving audio input at the electronic communication device;

performing text prediction using the typed input to obtain at least one text-predicted candidate word and an associated confidence value;

performing speech recognition using the audio input to obtain at least one speech-recognized candidate word and an associated confidence value;

selecting from the text-predicted and speech-recognized candidate words a candidate word having a highest associated confidence value, as a suggested candidate word; and

, one of;

receiving a portion of the audio input corresponding to a particular speech-recognized candidate word before receiving a most recent portion of the typed input; and

, determining the confidence value of the particular speech-recognized candidate word by referencing an amount of time that the particular speech-recognized candidate word is received before the most recent portion of the typed input, the confidence value of the particular speech-recognized candidate word decreasing as the amount of time increases;

or,

receiving the portion of the audio input corresponding to the particular speech-recognized candidate word after receiving the most recent portion of the typed input and, determining the confidence value of the particular speech-recognized candidate word by referencing an amount of time that the particular speech-recognized candidate word is received after receiving the most recent portion of the typed input, the confidence value of the particular speech-recognized candidate word decreasing as the amount of time increases.

2. The method of claim 1, further comprising obtaining the speech-recognized candidate word by performing a pre-selection from a plurality of speech-recognized candidate words based on at least a portion of the typed input.

3. The method of claim 2, wherein the portion of the typed input comprises an initial character of a word.

4. The method of claim 1, further comprising triggering the text prediction, the selecting, and the entering in response to receiving each character of the typed input.

5. The method of claim 1, wherein the selecting comprises selecting from the text-predicted and speech-recognized candidate words a plurality of candidate words having highest associated confidence values, as suggested candidate words.

6. The method of claim 5, further comprising displaying the suggested candidate words as user-selectable on a display of the electronic communication device.

7. The method of claim 6, further comprising displaying at least two of the suggested candidate words superimposed on a virtual keyboard.

8. The method of claim 6, further comprising receiving a user selection of one of the suggested candidate words.

9. The method of claim 1, further comprising entering the suggested candidate word into a text buffer of the electronic communication device.

10. A non-transitory computer-readable medium storing a computer program, wherein execution of the computer program is for:

facilitating text entry at an electronic communication device by;

receiving typed input at a text field from a keyboard of the electronic communication device;

receiving audio input at the electronic communication device;

performing text prediction using the typed input to obtain at least one text-predicted candidate word and an associated confidence value;

performing speech recognition using the audio input to obtain at least one speech-recognized candidate word and an associated confidence value;

selecting from the text-predicted and speech-recognized candidate words a candidate word having a highest associated confidence value, as a suggested candidate word; and

, one of;

receive a portion of the audio input corresponding to a particular speech-recognized candidate word before receiving a most recent portion of the typed input; and

determining the confidence value of the particular speech-recognized candidate word by referencing an amount of time that the particular speech-recognized candidate word is received before the most recent portion of the typed input, the confidence value of the particular speech-recognized candidate word decreasing as the amount of time increases;

or,

receive the portion of the audio input corresponding to the particular speech-recognized candidate word after receiving the most recent portion of the typed input; and

, determining the confidence value of the particular speech-recognized candidate word by referencing an amount of time that the particular speech-recognized candidate word is received after receiving the most recent portion of the typed input, the confidence value of the particular speech-recognized candidate word decreasing as the amount of time increases.

11. The non-transitory computer-readable medium of claim 10, wherein execution of the computer program is further for:

entering the suggested candidate word into a text buffer of the electronic communication device.

12. An electronic communication device comprising:

a display;

a microphone and connected analog-to-digital converter (ADC);

memory; and

a processor connected to the display, the ADC, and the memory, the processor configured to;

output a virtual keyboard to the display;

receive typed input from the keyboard, and perform text prediction using the typed input to obtain at least one text-predicted candidate word and an associated confidence value;

receive audio input from the ADC, and perform speech recognition using the audio input to obtain at least one speech-recognized candidate word and an associated confidence value;

select a candidate word having a highest confidence value from the text-predicted and speech-recognized candidate words, as a suggested candidate word; and

, one of;

receiving a portion of the audio input corresponding to a particular speech-recognized candidate word before receiving a most recent portion of the typed input; and

, determining the confidence value of the particular speech-recognized candidate word by referencing an amount of time that the particular speech-recognized candidate word is received before the most recent portion of the typed input, the confidence value of the particular speech-recognized candidate word decreasing as the amount of time increases;

or,

receiving the portion of the audio input corresponding to the particular speech-recognized candidate word after receiving the most recent portion of the typed input; and

, determining the confidence value of the particular speech-recognized candidate word by referencing an amount of time that the particular speech-recognized candidate word is received after receiving the most recent portion of the typed input, the confidence value of the particular speech-recognized candidate word decreasing as the amount of time increases.

13. The electronic communication device of claim 12, wherein the processor is further configured to store the suggested candidate word in the memory.

14. A method of facilitating text entry at an electronic communication device, the method comprising:

receiving typed input from a virtual keyboard;

receiving a stream of digitized audio input;

performing text prediction on a most recent word or word portion of the typed input to obtain at least one text-predicted candidate word and an associated confidence value;

performing speech recognition on the audio input to obtain at least one speech-recognized candidate word and an associated confidence value, the confidence value of the speech-recognized candidate word being based at least in part on a recency of a respective portion of the audio input as compared to a time of receiving the word or word portion of the typed input;

selecting from the text-predicted and speech-recognized candidate words a candidate word, as a suggested candidate word, based on the associated confidence values; and

, one of;

receiving a portion of the audio input corresponding to a particular speech-recognized candidate word before receiving a most recent portion of the typed input; and

, determining the confidence value of the particular speech-recognized candidate word by referencing an amount of time that the particular speech-recognized candidate word is received before the most recent portion of the typed input, the confidence value of the particular speech-recognized candidate word decreasing as the amount of time increases;

or,

receiving the portion of the audio input corresponding to the particular speech-recognized candidate word after receiving the most recent portion of the typed input; and

, determining the confidence value of the particular speech-recognized candidate word by referencing an amount of time that the particular speech-recognized candidate word is received after receiving the most recent portion of the typed input, the confidence value of the particular speech-recognized candidate word decreasing as the amount of time increases.

15. The method of claim 14, further comprising pre-selecting speech-recognized candidate words based on a match with the most recent word or word portion of the typed input.

16. The method of claim 14, further comprising displaying the suggested candidate word superimposed on a virtual keyboard.

17. The method of claim 14, comprising performing the speech recognition as a background process and performing the text prediction and the selecting in response to a key press at the virtual keyboard.

18. The method of claim 14, further comprising storing the suggested candidate word in a memory of the electronic communication device.