Input device enhanced interface

• US 9,983,788 B2
• Filed: 09/22/2014
• Issued: 05/29/2018
• Est. Priority Date: 03/17/2011
• Status: Active Grant

• Current Assignee: Intellitact LLC
• Original Assignee: Intellitact LLC
• Inventors: Laubach, Kevin
• Primary Examiner(s): LEE, TING ZHOU
• Application Number:
  US14/493,192
• Publication Number:
  US 20150012871A1
• Time in Patent Office: 1,345 Days
• CPC Class Codes: G06F 3/018 Input/output arrangements for oriental characters
• CPC Class Codes: G06F 3/0238 Programmable keyboards key guide holders G06F3/0224
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 3/0489 using dedicated keyboard keys or combinations thereof
• CPC Class Codes: G06F 3/04892 Arrangements for controlling cursor position based on codes indicative of cursor displacements from one discrete location to another, e.g. using cursor control keys associated to different directions or using the tab key arrangements for controlling cursor position based on coordinate signals G06F3/038
• CPC Class Codes: G06F 3/04895 Guidance during keyboard input operation, e.g. prompting
• CPC Class Codes: G06F 3/04897 Special input arrangements or commands for improving display capability

Abstract

Some embodiments provide an interactive viewer for an input device interface. Specifically, some embodiments provide a Keyboard Enhanced Interface (KEI) system that includes a KEI viewer. The KEI viewer includes multiple graphical keys that correspond to keys of a key based input device. The KEI viewer also includes multiple symbols that identify different output actions and logic for presenting a set of symbols over a set of graphical keys to identify a set of keys of the key based input device that are usable to directly invoke a set of output actions based on an active operational mode of the KEI system. A symbol identifies a particular output action that is directly invocable with a press of a key of the key based input device that corresponds to a graphical key over which said symbol is presented.

Claims

1. A system comprising:

a display presenting a plurality of graphical keys that are configured to represent a keyboard input device;

a memory system storing at least a computer program with computer-executable instructions;

a processor that is in communication with the memory system to execute the computer program, the computer-executable instructions programming the processor in;

determining one of a plurality of operational modes that is active based on two or more of (i) detecting and evaluating taps and press and holds of one or more alphanumeric key combinations, (ii) detecting one of a plurality of applications which is frontmost, (iii) detecting whether key focus is on a user interface (UI) element that accepts text input, and (iv) detecting presence or absence of selected text in a text field with key focus;

presenting various symbols over various graphical keys of the plurality of graphical keys based on which of the plurality of operational modes is active, wherein each particular symbol of the various symbols being presented visually indicates an output action that can be invoked using input that corresponds to a graphical key over which the particular symbol is presented, wherein in response to determining activation of a first operational mode of the plurality of operational modes, said output action comprises one of text selection and manipulation of selected text, wherein in response to determining activation of a second operational mode of the plurality of operational modes, said output action is different than a character insertion operation and comprises an executable operation targeting a background application, process, or operating system, and wherein in response to determining activation of a third operational mode of the plurality of operational modes, said output action is different than a character insertion operation and comprises an executable operation targeting a frontmost application or a user interface (UI) element with focus.

2. The system of claim 1, wherein the computer-executable instructions further program the processor in changing the display from presenting a first set of symbols over a first subset of the plurality of graphical keys to presenting a second set of symbols over a second subset of the plurality of graphical keys in response to identifying a change from the first operational mode to the second operational mode based on said determining.

3. The system of claim 2, wherein the second set of symbols visually indicates different output actions than those visually indicated by the first set of symbols, and wherein individual keys corresponding to the second subset of graphical keys over which the second set of symbols is displayed are usable to perform each of the visually indicated output actions.

4. The system of claim 1, wherein the computer-executable instructions further program the processor in changing the display from presenting a first set of symbols over the plurality of graphical keys to presenting a second set of symbols over the plurality of graphical keys in response to identifying a change from a first application being frontmost to a second application being frontmost.

5. The system of claim 1, wherein the computer-executable instructions further program the processor in changing the display from presenting a first set of symbols over the plurality of graphical keys to presenting a second set of symbols over the plurality of graphical keys in response to identifying selected text in the text field with key focus, wherein the second set of symbols visually indicates output actions that may manipulate said selected text.

6. The system of claim 1, wherein a first symbol of the various symbols being presented visually indicates status of a first user interface (UI) element over a graphical key of the plurality of graphical keys and a second symbol of the various symbols being presented visually indicates status of a second UI element that can be manipulated using an individual key that corresponds to one of the plurality of graphical keys over which the second symbol is presented.

7. The system of claim 1, wherein the computer-executable instructions further program the processor in changing the display from presenting a first set of symbols over the plurality of graphical keys to presenting a second set of symbols over the plurality of graphical keys in response to a press and hold of an individual key.

8. The system of claim 1, wherein the computer-executable instructions further program the processor in changing the display from presenting a first set of symbols over the plurality of graphical keys when a first individual key is pressed and held down to presenting a second set of symbols over the plurality of graphical keys when a different second individual key is pressed and held down.

9. A method for adapting a plurality of keyboard keys to perform a plurality of output actions based on user interface device events and system state information, the method comprising:

presenting on a display, a plurality of graphical keys mirroring at least a set of the plurality of keyboard keys with a first set of symbols presented over the plurality of graphical keys, wherein each symbol of the first set of symbols visually identifies an output action from a first set of output actions, and wherein each output action of the first set of output actions is a text insertion operation;

presenting different sets of symbols over the plurality of graphical keys in response to pressing and holding different alphanumeric keys longer than a specific duration, wherein each set of symbols is associated with one of a plurality of operational modes and visually identifies a different set of output actions, wherein each output action of the different set of output actions is a different operation than a text insertion operation with each output action of the different set of output actions executing one of an application call, a system call, or a script;

mapping a second set of output actions from the different sets of output actions to taps of the plurality of keyboard keys in response to pressing and holding a first alphanumeric key longer than the specific duration and retaining said holding of the first alphanumeric while tapping any of the plurality keys; and

remapping the text insertion operations of the first set of output actions to taps of the plurality of keyboard keys in response to releasing said holding of the first alphanumeric key, wherein said restoring comprises remapping on the display, the first set of symbols in place of the second set of symbols.

10. The method of claim 9, wherein presenting the different set of symbols comprises changing said display to a second set of symbols identifying the second set of output actions in response to the pressing and holding of the first alphanumeric keyboard key.

11. The method of claim 10, wherein presenting the different set of symbols further comprises changing said display by presenting a third set of symbols over the plurality of graphical keys while a different second alphanumeric key is pressed and held down and the first alphanumeric key is released, wherein each symbol of the third set of symbols visually identifies an output action from a third set of output actions.

12. The method of claim 9 further comprising performing a desired output action from the second set of output actions in response to tapping an individual alphanumeric keyboard key while the first alphanumeric key is held down.

13. A method for adapting a plurality of alphanumeric keyboard keys to perform different output actions based on user interface device events and system state information, the method comprising:

mapping invocation of a set of character insertion output actions to a first set of the plurality of alphanumeric keyboard keys, wherein the set of character insertion output actions are normal text insertion operations;

detecting a first time at which a first alphanumeric keyboard key of the plurality of alphanumeric keyboard keys is pressed down;

suppressing repeated insertion of a character from the set of character insertion output actions mapped to said first alphanumeric keyboard key while the first alphanumeric keyboard key is held down subsequent to being pressed down;

detecting a second time at which the first alphanumeric keyboard key is released;

inserting said character in response to the difference between the first time and the second time being less than a specific duration;

mapping invocation of a first set of application or operating system specific output actions to a second set of the plurality of alphanumeric keyboard keys in response to a duration elapsed from said detecting of the first time at which the first alphanumeric keyboard key is pressed down exceeding said specific duration, and before said detecting of the second time at which the first alphanumeric keyboard key is released, wherein the first set of application or operating system specific output actions differ from said text insertion operations;

performing a particular output action of said first set of application or operating system specific output actions in response to tapping a second individual alphanumeric keyboard key after the specific duration passing from said detecting of the first time at which the first alphanumeric keyboard key is pressed down and before said detecting of the second time at which the first alphanumeric keyboard key is released, wherein said mapping maps invocation of the particular output action of the first set of application or operating system specific output actions to the second individual alphanumeric keyboard key.

14. The method of claim 13, further comprising remapping invocation of said set of character insertion output actions to the first set of keyboard keys in response to said detecting the second time at which the first alphanumeric keyboard key is released.

15. The method of claim 13 further comprising mapping invocation of a set of text manipulation output actions to a third set of the plurality of keyboard keys in response to selection of text within a text field, wherein the set of text manipulation output actions are not text insertion operations, and wherein the set of text manipulation output actions remains mapped to the third set of keyboard keys while any text within the text field remains selected.

16. The method of claim 15 further comprising remapping invocation of said set of character insertion output actions to the first set of keyboard keys when text is deselected in the text field.

17. The method of claim 13, wherein mapping invocation of the set of character insertion output actions is based on a first user interface (UI) element accepting focus, wherein the first UI element comprises a text field that accepts keyboard input, and the method further comprising mapping invocation of said first set of application or operating system specific output actions in response to focus shifting to a second UI element that does not accept keyboard input.

18. The method of claim 13 further comprising mapping invocation of a different second set of application or operating system specific output actions to a third set of the plurality of keyboard keys in response to a duration elapsed from said detecting of the first time at which the first alphanumeric keyboard key is pressed down exceeding said specific duration, and before said detecting of the second time at which the first alphanumeric keyboard key is released, and detecting an added press and hold of a third alphanumeric keyboard key.