System and methods for determining keyboard input in the presence of multiple contact points

• US 9,377,871 B2
• Filed: 08/01/2014
• Issued: 06/28/2016
• Est. Priority Date: 08/01/2014
• Status: Expired due to Fees

• Current Assignee: Cerence Operating Company (Cerence Inc.)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Waddell, Gordon, Mannby, Claes-Fredrik
• Primary Examiner(s): FARAGALLA, MICHAEL A
• Application Number:
  US14/450,095
• Publication Number:
  US 20160034046A1
• Time in Patent Office: 697 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 3/04895 Guidance during keyboard input operation, e.g. prompting
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs

Abstract

System and methods are provided for interpreting multiple contact events detected on a virtual keyboard of a computing device to determine an intended key press. The systems and methods utilize event data collected from each of the multiple contact events on a touchscreen in order to provide a general confidence level at which each contact event is deemed to be an intended key press. The confidence level is based on a probabilistic/algorithmic assessment of various criteria for each contact event detected. The criteria may be variable and rely on other events detected on the virtual keyboard even though each contact event is considered a physically independent action from one another. Once an intended key is determined, that key may be input to a word predication application also stored on the computing device. The word prediction application may use the intended key to predict at least one word for the user.

Claims

1. A method for interpreting contact events on a virtual keyboard, the method comprising:

detecting multiple contact events on a virtual keyboard generated on a touchscreen of a computing device, the multiple contact events a result of a user'"'"'s interaction with the touchscreen;

for each detected contact event;

determining that a duration of the contact event falls within a threshold time range associated with an intended key press;

determining that a surface area of the contact event falls within a threshold surface area range associated with an intended key press; and

interpreting the contact event as an intended key press when the duration of the contact event falls within the threshold time interval and the surface area of the contact event surface area falls within the threshold surface area range;

or

interpreting the contact event as an unintended key press when the duration of the contact event falls outside the threshold time interval and the surface area of the contact event surface area falls outside the threshold surface area range; and

inputting the intended key presses for each of the detected contact events into a prediction application that selectively displays one or more predicted words via the virtual keyboard.

2. The method of claim 1, wherein if more than one of the multiple contact events is identified as an intended key press:

determining a distance of each of the contact events identified as an intended key press, wherein the distance is a position of a contact event relative to a reference location on the virtual keyboard; and

comparing each of the contact events based on associated distances to determine whether any of the contact events should not be interpreted as a key press.

3. The method of claim 2, wherein a reference location corresponds to one of the home keys on a QWERTY keyboard.

4. The method of claim 2, wherein, if more than one of the multiple contact events is still identified as an intended key press, further analyzing the contact event based on a location of the contact event.

5. The method of claim 4, further comprising:

determining if the duration of the contact event falls within a first time interval,

wherein the first time interval includes a set of reference locations, and

wherein the location of contact event within the first time interval does not vary over the first time interval; and

determining if the duration of the contact event falls within a second time interval,

wherein the second time interval includes at least one transition contact event; and

wherein the second time interval is shorter than the first time interval.

6. The method of claim 1, further comprising calculating a probability for each contact event that the contact event is an intended key press.

7. The method of claim 6, wherein calculating a probability for a contact event comprises:

calculating a probability that the duration of the contact event implies an intended key press;

calculating a probability that the surface area of the contact event implies an intended key press; and

summing the calculated probabilities associated with the contact event to generate an overall probability that the detected contact event should be interpreted as an intended key press.

8. The method of claim 7, further comprising adjusting the calculated probability for a contact event based on a distance that each contact event is located from a reference location.

9. The method of claim 8, further comprising adjusting the calculated probability based on characteristics of other contact events.

10. A non-transitory computer-readable storage medium whose contents, when executed by a computing device, cause the computing device to perform operations for interpreting contact events on a virtual keyboard, the operations comprising:

detecting multiple contact events on a virtual keyboard generated on a touchscreen of a computing device, the multiple contact events a result of a user'"'"'s interaction with the touchscreen;

for each detected contact event;

determining that a duration of the contact event falls within a threshold time range associated with an intended key press;

determining that a surface area of the contact event falls within a threshold surface area range associated with an intended key press; and

interpreting the contact event as an intended key press when the duration of the contact event falls within the threshold time interval and the surface area of the contact event surface area falls within the threshold surface area range, or

interpreting the contact event as an unintended key press when the duration of the contact event falls outside the threshold time interval and the surface area of the contact event surface area falls outside the threshold surface area range; and

inputting the intended key presses for each of the detected contact events into a prediction application that selectively displays one or more predicted words via the virtual keyboard.

11. The computer-readable storage medium of claim 10, wherein if more than one of the multiple contact events is identified as an intended key press:

determining a distance of each of the contact events identified as an intended key press, wherein the distance is a position of a contact event relative to a reference location on the virtual keyboard; and

comparing each of the contact events based on associated distances to determine whether any of the contact events should not be interpreted as a key press.

12. The computer-readable storage medium of claim 11, wherein a reference location corresponds to one of the home keys on a QWERTY keyboard.

13. The computer-readable storage medium claim 11, wherein, if more than one of the multiple contact events is still identified as an intended key press, further analyzing the contact event based on a location of the contact event.

14. The computer-readable storage medium of claim 13, further comprising:

determining if the duration of the contact event falls within a first time interval,

wherein the first time interval includes a set of reference locations, and

wherein the location of contact event within the first time interval does not vary over the first time interval; and

determining if the duration of the contact event falls within a second time interval,

wherein the second time interval includes at least one transition contact event; and

wherein the second time interval is shorter than the first time interval.

15. The computer-readable storage medium of claim 10, further comprising calculating a probability for each contact event that the contact event is an intended key press.

16. The computer-readable storage medium of claim 15, wherein calculating a probability for a contact event comprises:

calculating a probability that the duration of the contact event implies an intended key press;

calculating a probability that the surface area of the contact event implies an intended key press; and

summing the calculated probabilities associated with the contact event to generate an overall probability that the detected contact event should be interpreted as an intended key press.

17. The computer-readable storage medium of claim 16, further comprising adjusting the calculated probability for a contact event based on a distance that each contact event is located from a reference location.

18. The computer-readable storage medium of claim 17, further comprising adjusting the calculated probability based on characteristics of other contact events.

19. A method, comprising:

detecting multiple simultaneous contact events input to a virtual keyboard displayed by a mobile device;

for each of the multiple simultaneous contact events, identifying a single contact event having a first characteristic of a duration of the contact event that satisfies a threshold time range associated with an intended key press of a key of the virtual keyboard, and a second characteristic of having a surface area of the contact event that satisfies a threshold surface area range associated with the intended key press of the key of the virtual keyboard; and

inputting text represented by the key of the virtual keyboard associated with the identified single contact event into a text string displayed by the virtual keyboard.