System and method for issuing commands based on pen motions on a graphical keyboard

• US 20060253793A1
• Filed: 05/04/2005
• Published: 11/09/2006
• Est. Priority Date: 05/04/2005
• Status: Active Grant

• Current Assignee: Cerence Operating Company (Cerence Inc.)
• Original Assignee: International Business Machines Corporation
• Inventors: Zhai, Shumin, Kristensson, Per-Ola
• Granted Patent: US 7,487,461 B2
• Time in Patent Office: Days
• US Class Current: 715/773
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A command pattern recognition system based on a virtual keyboard layout combines pattern recognition with a virtual, graphical, or on-screen keyboard to provide a command control method with relative ease of use. The system allows the user conveniently issue commands on pen-based computing or communication devices. The system supports a very large set of commands, including practically all commands needed for any application. By utilizing shortcut definitions it can work with any existing software without any modification. In addition, the system utilizes various techniques to achieve reliable recognition of a very large gesture vocabulary. Further, the system provides feedback and display methods to help the user effectively use and learn command gestures for commands.

Claims

1. A method of issuing a command using a graphical keyboard, comprising:

inputting a movement on the graphical keyboard;

automatically recognizing the movement as a command, based on the movement in relation to a layout of the graphical keyboard; and

executing the command.

2. The method of claim 1, wherein recognizing the movement as a command comprises identifying the pattern of the movement trajectory in relation to a layout of the graphical keyboard.

3. The method of claim 1, wherein the movement starting within the vicinity of a designated key, triggers the recognition of the movement as a command. a

4. The method of claim 2, wherein recognizing the movement as a command comprises matching the movement trajectory with stored command templates.

5. The method of claim 1, wherein the graphical keyboard includes any one or more of:

a virtual keyboard operated by any one of;

a stylus or a digital pen;

a virtual keyboard presenting a plurality of keys comprising alphabet-numeric characters;

a virtual keyboard presenting a plurality of keys comprising a plurality of commands labeled with any one of;

a symbol or a word, and wherein a movement trajectory is recognized as a set of commands;

a virtual keyboard on a touch sensing surface sensitive to finger movement;

a virtual keyboard projected onto a surface; and

a virtual keyboard rendered in three dimensional space in a three dimensional display.

6. The method of claim 1, wherein the movement comprises any one of:

a pen stroke, a stylus stroke, or a finger trace.

7. The method of claim 2, wherein recognizing the movement comprises using a combination of channels that selectively process different aspects of the movement trajectory.

8. The method of claim 7, wherein the channels comprise any one or more of:

a shape channel;

a location information; and

a context model channel.

9. The method of claim 4, further comprising defining the command templates.

10. The method of claim 9, wherein defining the command templates comprises using trained models;

and

wherein recognizing the movement comprises classifying gestures according to the trained models.

11. The method of claim 2, wherein recognizing the movement comprises analyzing geometric properties of the movement trajectory.

12. The method of claim 1, further comprising sensing the movement by means of a digital pen.

13. The method of claim 1, further comprising sensing the movement by means of a finger motion on a touch sensor.

14. The method of claim 1, further comprising sensing the movement by tracking a three-dimensional motion of any one of:

a user'"'"'s hand, finger, pointer, or stylus.

15. The method of claim 1, further comprising sensing the movement by means of a tracked eye-gaze movement.

16. A computer program product having a plurality of instruction codes stored on a computer-readable medium, for issuing a command using a graphical keyboard, the computer program product comprising:

a first set of instruction codes for inputting movement on the graphical keyboard;

a second set of instruction codes for automatically recognizing the stroke trace as a command, based on the movement in relation to a layout of the graphical keyboard; and

a third set of instruction codes for executing the command.

17. The computer program product of claim 16, wherein the second set of instruction codes recognizes the movement as a command comprises by identifying the pattern of the movement trajectory in relation to a layout of the graphical keyboard.

18. The computer program product of claim 17, wherein the movement starting within the vicinity of a designated key triggers the recognition of the movement as a command.

19. An apparatus for issuing a command using a graphical keyboard, the apparatus comprising:

a sensing interface for recording movement on the graphical keyboard;

a plurality of channels for automatically recognizing the movement as a command, based on a movement trajectory in relation to a layout of the graphical keyboard; and

an integrator connected to the plurality of channels, for analyzing the movement trajectory and for outputting the command to be executed.

20. The apparatus of claim 19, wherein the plurality of channels identify a pattern of the movement trajectory in relation to a layout of the graphical keyboard.