Input method editor providing language assistance

• US 10,656,957 B2
• Filed: 08/09/2013
• Issued: 05/19/2020
• Est. Priority Date: 08/09/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Jing, Kun, Liu, Weipeng, Scott, Matthew Robert, Li, Mu, Shi, Jin
• Primary Examiner(s): Long, Andrea N
• Application Number:
  US14/911,247
• Publication Number:
  US 20160196150A1
• Time in Patent Office: 2,475 Days
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 40/129 Handling non-Latin characters, e.g. kana-to-kanji conversion
• CPC Class Codes: G06F 40/232 Orthographic correction, e.g. spell checking or vowelisation
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs
• CPC Class Codes: G06F 9/453 Help systems
• CPC Class Codes: G09B 19/04 Speaking with audible presentation of the material to be studied G09B5/04
• CPC Class Codes: G09B 19/06 Foreign languages with audible presentation of material to be studied G09B5/04

Abstract

An input method editor (IME) configured to provide language assistance across a plurality of applications is disclosed. In one example, the IME is adapted for use by English-as-a-second-language (ESL) users. In a specific example, language assistance may be provided by first detecting a need to suggest a substitute word to a user who is typing within an application. The detection may be based on a probability that a current word is in error. If a need is detected, a suggestion may be obtained for the word, such as from a cloud linguistic service or from a local lexicon and language-model, if network connectivity is poor. Once obtained, the suggestion may be displayed to the user in a non-intrusive manner by user interface element(s). Interaction with the user allows the user to accept or reject the suggestion, and perform other functions, such as relocating user interface elements utilized by the display.

Claims

1. One or more computer-readable storage storing computer executable instructions that, when executed, cause one or more processors to perform acts comprising:

detecting keystrokes within an application by operation of an in-process input method editor (IME) dynamic link library (DLL);

forwarding the keystrokes detected within the application by operation of the in-process IME DLL to an out-of-process IME platform process, wherein the out-of-process IME platform process is located within a defined computer-readable memory address range separate and distinct from an address range within which the application is defined;

detecting a need for a suggestion to assist with input of a word into the application, the detecting comprising;

accessing words entered into the application and recorded; and

calculating a probability of correctness of the word input based at least in part on the recorded words;

obtaining a suggestion for replacement of the word, if indicated by the calculated probability, the obtaining comprising;

obtaining the suggestion from a cloud linguistic service (CLS);

if network capability used to communicate with the CLS is limited or non-existent, then obtaining a suggestion from a local lexicon or statistical language-model, based at least in part on the recorded words previously entered into the application; and

utilizing a confidence score for an obtained suggestion to determine whether to present the obtained suggestion;

providing the obtained suggestion for display if indicated by the confidence score, the providing comprising;

providing the obtained suggestion for display within a notification bubble; and

providing the notification bubble for display at a location based at least in part on location of an IME bar; and

moving the IME bar and/or the notification bubble in response to input signifying movement.

2. The one or more computer-readable storage as recited in claim 1, wherein detecting the need for the suggestion additionally comprises:

detecting typing speed, detecting typing stopping, detecting multiple backspaces, or detecting re-typing by the user.

3. The one or more computer-readable storage as recited in claim 1, wherein calculating the probability of correctness of the word is based on at least one of multiple factors, including:

locally checking on a client or server side checking, the checking comprising checking for the word in the local lexicon or a cloud-based lexicon, respectively;

or

examining a sequence of the recorded words to provide context for the word.

4. The one or more computer-readable storage as recited in claim 1, wherein obtaining the suggestion comprises:

querying one or more cloud-based services for correction candidates and associated probabilities, each associated probability representing a confidence score that an associated correction candidate includes a correct suggestion;

wherein the CLS is configured to arbitrate between multiple cloud-based services to collect candidates for suggestions.

5. The one or more computer-readable storage as recited in claim 1, wherein:

detecting the need for the suggestion includes monitoring to determine when text entry stops; and

providing the suggestion for display includes providing the suggestion for display after text entry stops.

6. The one or more computer-readable storage as recited in claim 1, additionally comprising:

persisting the location of the notification bubble over restarts; and

roaming the location of the notification bubble.

7. An input method editor (IME), comprising:

an out-of-process IME platform process configured to;

receive, from an in-process IME dynamic link library (DLL), keystrokes detected by an application of the in-process DLL, wherein the out-of-process IME platform process is located within a defined computer-readable memory address range separate and distinct from an address range within which the application is defined;

detect a need for a suggestion to fix a word entered into an application operating on a computing device;

obtain the suggestion from a cloud linguistic service (CLS) through a network connection on the computing device; and

obtain the suggestion from a local lexicon or language-model if network capability used to communicate with the CLS is limited or nonexistent; and

a suggestion renderer operating on the computing device and configured to prepare the suggestion for display, preparation of the suggestion for display comprising;

causing display of the suggestion within a notification bubble; and

binding the notification bubble according to received input.

8. The input method editor as recited in claim 7, wherein the IME platform process is additionally configured to:

receive instructions for use of the displayed suggestion; and

respond to input to move the notification bubble.

9. The input method editor as recited in claim 7, additionally comprising:

the DLL configured in-process and configured to relay keystrokes to the IME platform.

10. The input method editor as recited in claim 7, wherein the IME platform process and the suggestion renderer are configured out-of-process.

11. A method, comprising:

detecting keystrokes within an application by operation of an in-process input method editor (IME) dynamic link library (DLL);

forwarding the keystrokes detected within the application by operation of the in-process IME DLL to an out-of-process IME platform process, wherein the out-of-process IME platform process is located within a defined computer-readable memory address range separate and distinct from an address range within which the application is defined;

detecting, by operation of one or more processors configured with executable instructions, a need for a suggestion to an error in text entered into the application, the detecting comprising testing for whether suggestions are needed based on a probability of correctness of a word, wherein the probability is based on multiple factors, including local checks on a client or check by a server received at the client;

obtaining, by operation of the one or more processors configured with the executable instructions, a suggestion for a replacement for the word, the obtaining comprising querying services for correction candidates and probabilities, in the form of a confidence score, that the correction is likely;

causing the suggestion to be displayed if a confidence in the suggestion is greater than an threshold, the displaying comprising;

providing the suggestion in response to selection of an indicator that a suggestion is available; and

locating a display of the suggestion adjacent to an IME bar; and

operating a user interface, the operating comprising;

receiving instructions for use of the displayed suggestion; and

responding to input to move the displayed suggestion to a preferred screen location.

12. The method as recited in claim 11, wherein detecting the need for the suggestion additionally comprises:

detecting that text entry has slowed.

13. The method as recited in claim 11, wherein detecting the need for the suggestion additionally comprises:

examining a sequence of words to provide context for the word; and

using a statistical language model having input including the sequence of words.

14. The method as recited in claim 11, wherein obtaining the suggestion for the replacement additionally comprises:

obtaining the suggestion from a cloud linguistic service (CLS) if permitted by network capability; and

obtaining the suggestion from a local lexicon or language-model if network capability does not provide connection with the CLS.

15. The method as recited in claim 11, wherein obtaining the suggestion for the replacement additionally comprises:

querying multiple cloud-based services; and

aggregating results from the multiple services.

16. The method as recited in claim 11, wherein causing the suggestion to be displayed comprises:

monitoring to determine when text entry has stopped; and

displaying the suggestion after the text entry has stopped.

17. The method as recited in claim 11, wherein operating the user interface comprises:

receiving a rejection of the suggestion to replace the word;

reducing likelihood of the suggestion to replace the word in the future; and

reducing likelihood that the word will be considered problematic in the future.

18. The method as recited in claim 11, wherein obtaining the suggestion comprises:

determining if a technology of local functionality to suggest the replacement is similar to a technology of remote functionality used to suggest the replacement; and

using the remote functionality if the technologies are similar.