System and method for inputting one or more inputs associated with a multi-input target

• US 10,802,710 B2
• Filed: 04/02/2015
• Issued: 10/13/2020
• Est. Priority Date: 04/04/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Touchtype Limited (Microsoft Corporation)
• Inventors: Osborne, Joseph Hayyim Benedict, Okuno, Yoh
• Primary Examiner(s): Roy, Sanchita
• Application Number:
  US15/301,910
• Publication Number:
  US 20170192671A1
• Time in Patent Office: 2,021 Days
• CPC Class Codes: G06F 3/0236 using selection techniques to select from displayed items
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs

Abstract

A system comprising a user interface and a prediction engine is discussed. The user interface comprises a multi-input target associated with multiple input options available for input. In a first system, the user interface is configured to select the multi-input target and input of the multiple input options. The prediction engine is configured to determine the probability that the user intended to select an input of the multiple inputs other than the selected input. In a second system, the user interface is configured to identify a first input of the multiple input options and change the identified input from the first input to a second input. The prediction engine is configured to determine the time elapsed between first and second user selection events, the probability that the user intended to select the second input, and the probability that the user had intended to input the first input.

Claims

1. A system comprising a processor and memory, the memory storing computer-readable instructions that, when executed by the processor, cause the system to:

render, in response to receipt of a first user selection event that triggers output of multiple input options, a user interface comprising a multi-input keyboard target associated with the multiple input options available for input by user selection;

identify an input option of the multiple input options in response to a second user selection event that triggers one of the multiple input options, wherein a location of the second user selection event received in the user interface is spatially separated from a location of the first user selection event received in the user interface;

perform a selection of a keyboard character, using the identified input option, wherein the selection of the keyboard character corresponds to a first keyboard character of the multiple input options;

execute a prediction engine configured to determine, based on location and time characteristics from the user inputs in each of the first user selection event and the second user selection event, a probability that the user had intended to select an input option of the multiple input options other than the identified input option, wherein the input option that the user had intended to select corresponds to a second keyboard character;

change the selection of the keyboard character, from the first keyboard character to the second keyboard character, based on the probability determined by the prediction engine; and

input the selected keyboard character into the system.

2. The system of claim 1, wherein the first user selection event corresponds to a first tap or a start of a drag-and-lift gesture and the second user selection event corresponds to a second tap or the lift of the drag-and-lift gesture.

3. The system of claim 1, wherein the prediction engine is configured to determine a distance and direction between respective first and second locations of the first and second user selection events.

4. The system of claim 1, wherein the prediction engine comprises a 4D probability distribution model for each input of the multiple input options.

5. The system of claim 4, wherein the prediction engine determines the probability that the user had intended to select an input option of the multiple input options by querying the model associated with that input option with the respective first and second locations of the first and second user selection events or with the first location, and a distance and direction between the first and second locations.

6. The system of claim 4, wherein the model is a 4D Gaussian.

7. The system of claim 1, wherein the first user selection event corresponds to a start of a flick gesture and the second user selection event corresponds to an end of the flick gesture;

and wherein the prediction engine is configured to;

determine a flick direction from the start and end locations; and

determine the probability that the user intended to select the input option of the multiple input options based on the start location and the direction of the flick.

8. The system of claim 1, wherein the prediction engine generates the probability that a user intended to select a given input option for each of the multiple input options.

9. The system of claim 1, wherein the user interface comprises a pluraiiE of multi-input keyboard targets and the prediction engine is further configured to:

determine, based on the first user selection event, the probability that the user had intended to select the selected multi-input keyboard target.

10. The system of claim 1, wherein in response to the first user selection event, the user interface is configured to display the multiple input options separated in space.

11. The system of claim 1, wherein the prediction engine comprises a plurality of models, one for each input option of the multiple input options, wherein each model relates previous first and second user selection events corresponding to an input option of the multiple input options to that input option, wherein the prediction engine is configured to generate, in association with the plurality of models, the probability that the first and second user selection events relates to a particular input option of the multiple input options.

12. The system of claim 1, wherein the multi-input keyboard target is a multi-character key on a virtual keyboard that represents multiple character input options.

13. A system comprising a processor and memory, the memory storing computer-readable instructions that, when executed by the processor, cause the system to:

generate a user interface configured to render a multi-input keyboard target associated with multiple input options available for input by user selection, wherein the multiple input options include a first input option associated with a first keyboard character and a second input option associated with a second keyboard character;

identify the first input option of the multiple input options in response to receipt of a first user selection event directed at the multi-input keyboard target that triggers the first input option;

change the identified input option from the first input option to a second input option of the multiple input options in response to receipt of a second user selection event directed at the multi-input keyboard target, based on a location of the first user selection event within the user interface and a location of the second user selection event within the user interface, the locations being inside of an area of the user interface corresponding to the multi-input keyboard target;

perform a selection of the second keyboard character, using the identified input option;

execute a prediction engine, wherein the prediction engine is configured to;

determine a time elapsed between the first and second user selection events;

determine that the time elapsed is less than a predetermined threshold; and

determine, based on the elapsed time, a probability that the user had intended to select the first input option followed by the second input option relative to the probability that the user had intended to select the first input option consecutively;

change the selection of the second keyboard character, caused from the second user selection event, to a selection of the first keyboard character, based on the probability that the user had intended to select the first input option consecutively being greater than the probability that the user had intended to select the first input option followed by the second input option, wherein the predetermined threshold is set such that if the elapsed time is greater than the predetermined threshold, a probability of 1 is utilized for the probability that the user had intended to select the first input option consecutively; and

input the selected keyboard character into the system.

14. The system of claim 13, wherein the multi-input keyboard target is a multi-character key on a virtual keyboard that represents multiple character input options.

15. The system of claim 14, wherein the first user selection event corresponds to a first tap on the multi-character keyboard target which selects the first input and the second user selection event corresponds to a second tap on the multi-character keyboard target which changes the selected input to the second input.

16. The system of claim 13, wherein the probability between the first user selection event and the predetermined threshold is modelled by a sigmoid function.

17. The system of claim 16, wherein the sigmoid function takes the form

18. A method of determining, using a prediction engine, a probability that a user intended to select an input option of a user interface of a system that displays multiple input options available via a multi-input key, the method comprising:

receiving by the user interface a first user selection event to select the multi-input key;

receiving by the user interface a second user selection event to select an input option of the multiple input options displayed, wherein a location of the second user selection event is spatially separated from a location of the first user selection event and the multiple input options displayed are provided in response to the first user selection event;

performing a selection of a keyboard character, using the selected input option, wherein the selection of the keyboard character corresponds to a first keyboard character of the multiple input options;

determining, based on location and time characteristics from the user inputs in each of the first user selection event and the second user selection event, a probability that the user had intended to select an input option of the multiple input options other than a selected input option, wherein the input option that the user had intended to select corresponds to a second keyboard character;

changing of the selection of the keyboard character, from the first keyboard character to the second keyboard character, based on the probability determined by the prediction engine; and

inputting the selected keyboard character into the system.