Performing actions on a computing device using a contextual keyboard

• US 9,244,611 B2
• Filed: 11/21/2011
• Issued: 01/26/2016
• Est. Priority Date: 11/20/2010
• Status: Expired due to Fees

• Current Assignee: Cerence Operating Company (Cerence Inc.)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Kushler, Clifford A., Manning, Jane, McSherry, Mike, Sheedy, Aaron
• Primary Examiner(s): Spar, Ilana
• Assistant Examiner(s): Karki, Paras D
• Application Number:
  US13/301,724
• Publication Number:
  US 20120127082A1
• Time in Patent Office: 1,527 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

A system and method for performing actions via a keyboard application of a computing device is described. In some examples, a contextual keyboard facilitates the entry of text and the performance of actions across applications supported by a device. For example, the contextual keyboard may receive a word macro and/or a gesture, and determine that the received word macro and/or gesture is a command to perform one or more actions. Example actions to be performed include searching for information, storing information, posting information, sending information, enhancing information, and so on.

Claims

1. A keyboard system for a mobile device, comprising:

a keyboard component configured to present a virtual keyboard to a user via a touch-screen of the mobile device;

a gesture reception component configured to receive stroke gesture input via the keyboard component;

a context component configured to identify contextual information that describes the context in which the stroke gesture input is received;

an interpretive component configured to determine, based on the received stroke gesture input and the identified contextual information, without additional user input, whether the received stroke gesture input is a command for the keyboard system or is text entry related to inputting a text string,

wherein the keyboard system is distinct from other applications of the mobile device;

a text entry component configured to receive stroke gesture input determined as text entry via the virtual keyboard and update a text string based on the received text entry; and

an action component configured to receive stroke gesture input determined as a command for the keyboard system;

intercept the received stroke gesture input before it is sent to another active application; and

perform one or more actions associated with the command received via the virtual keyboard,

wherein the action is not the function of a key on the keyboard.

2. The keyboard system of claim 1, wherein the action component is configured to perform one or more actions in response to the command and seed text displayed by the touch-screen of the mobile device.

3. The keyboard system of claim 2, wherein the received stroke gesture input is a shape gesture distinct from a path across certain keys on the keyboard.

4. The keyboard system of claim 2, wherein the action component is configured to invoke an application and to provide the detected seed text to the invoked application.

5. The keyboard system of claim 1, wherein the action component is configured to perform one or more actions based on a portion of highlighted text displayed by the touch-screen of the mobile device.

6. The keyboard system of claim 1, wherein the text entry component is configured to receive text entry via the virtual keyboard when a user of the mobile device taps keys of the virtual keyboard or traces a path across keys of the virtual keyboard.

16. The keyboard system of claim 1, wherein the contextual information associated with the received stroke gesture input comprises one or more of text to the left of the text entry point, selected text, or substantially all displayed text.

17. The keyboard system of claim 1, wherein the contextual information associated with the received stroke gesture input comprises one or more of an application in use, content associated with an application in use, previously performed actions, a location of the mobile device, user metadata, or user preferences.

18. The keyboard system of claim 1, wherein the context component is configured to identify seed text associated with the received stroke gesture input, such that the interpretive component determines, based on the identified seed text, whether the received stroke gesture input is a command for the keyboard system.

19. The keyboard system of claim 18, wherein the interpretive component is further configured to determine what action to perform based on the identified seed text.

20. The keyboard system of claim 1, wherein the one or more actions include obtaining a map relating to displayed text.

7. A non-transitory computer-readable medium whose contents, when executed by a processor of a computing device, cause the computing device to perform a method to provide a contextual keyboard, comprising:

receiving stroke gesture input via the displayed keyboard,

wherein the displayed keyboard is configured to receive text input;

identifying contextual information that describes the context in which the stroke gesture input is received;

determining, based on the received stroke gesture input and the identified contextual information, without additional user input, whether the received stroke gesture input comprises a command associated with an action to be performed by the contextual keyboard,

wherein the contextual keyboard is distinct from other applications of the mobile device; and

wherein the action is not the function of a key on the keyboard;

when the received stroke gesture input is determined not to comprise a command for the keyboard application, then;

accepting the received stroke gesture input as input for facilitating the entry of text; and

when the received stroke gesture input is determined to comprise a command for the contextual keyboard, then;

intercepting the received stroke gesture input before it is sent to another active application, such that the stroke gesture input is not accepted as text input;

detecting a displayed seed object upon which an action may be performed; and

performing the associated action based on the determined command and the detected seed object.

8. The non-transitory computer-readable medium of claim 7, wherein the stroke gesture input received via the displayed keyboard is a word macro or a shape gesture distinct from a path across certain keys on the keyboard.

9. The non-transitory computer-readable medium of claim 7, wherein performing the associated action based on the determined command and the detected seed object includes storing information displayed by the computing device in a database of the computing device.

10. The non-transitory computer-readable medium of claim 7, wherein performing the associated action based on the determined command and the detected seed object includes posting information associated with the detected seed object to a webpage.

11. The non-transitory computer-readable medium of claim 7, wherein performing the associated action based on the determined command and the detected seed object includes inserting a link to content based on information associated with the detected seed object.

12. The non-transitory computer-readable medium of claim 7, wherein performing the associated action based on the determined command and the detected seed object includes searching for information associated with the detected seed object.

13. A method performed by a keyboard application of a mobile device, the method comprising:

receiving stroke gesture input via a virtual keyboard of the keyboard application, wherein the virtual keyboard is configured to receive text input;

identifying contextual information that describes the context in which the stroke gesture input is received;

determining, based on the received stroke gesture input and the identified contextual information, without additional user input, whether the received stroke gesture input comprises a command associated with an action to be performed by the keyboard application,

wherein the keyboard application is distinct from other applications of the mobile device; and

wherein the action is not the function of a key on the keyboard;

when the received stroke gesture input is determined not to comprise a command for the keyboard application, then;

accepting the received stroke gesture input as input for facilitating the entry of text; and

when the received stroke gesture input is determined to comprise a command for the keyboard application, then;

intercepting the received stroke gesture input before it is sent to another active application, such that the stroke gesture input is not accepted as text input; and

performing the associated action based on the determined command.

14. The method of claim 13, wherein performing the associated action based on the determined command includes initiating an action to be performed by a separate application running on the mobile device.

15. The method of claim 13, wherein the received input is a text string or a gesture, and wherein determining the received input is associated with an action to be performed by the keyboard application includes determining the text string or gesture is associated with one or more actions to be performed by the keyboard application.