Input device enhanced interface

• US 8,843,844 B2
• Filed: 03/17/2012
• Issued: 09/23/2014
• Est. Priority Date: 03/17/2011
• Status: Active Grant

• Current Assignee: Intellitact LLC
• Original Assignee: Intellitact LLC
• Inventors: Laubach, Kevin
• Primary Examiner(s): LEE, TING ZHOU
• Application Number:
  US13/423,204
• Publication Number:
  US 20120235921A1
• Time in Patent Office: 920 Days
• Field of Search: 715/773, 715/847, 345/168, 345/173
• US Class Current: 715/773
• CPC Class Codes: G06F 3/018 Input/output arrangements for oriental characters
• CPC Class Codes: G06F 3/0238 Programmable keyboards key guide holders G06F3/0224
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 3/0489 using dedicated keyboard keys or combinations thereof
• CPC Class Codes: G06F 3/04892 Arrangements for controlling cursor position based on codes indicative of cursor displacements from one discrete location to another, e.g. using cursor control keys associated to different directions or using the tab key arrangements for controlling cursor position based on coordinate signals G06F3/038
• CPC Class Codes: G06F 3/04895 Guidance during keyboard input operation, e.g. prompting
• CPC Class Codes: G06F 3/04897 Special input arrangements or commands for improving display capability

Abstract

Some embodiments provide an interactive viewer for an input device interface. Specifically, some embodiments provide a Keyboard Enhanced Interface (KEI) system that includes a KEI viewer. The KEI viewer includes multiple graphical keys that correspond to keys of a key based input device. The KEI viewer also includes multiple symbols that identify different output actions and logic for presenting a set of symbols over a set of graphical keys to identify a set of keys of the key based input device that are usable to directly invoke a set of output actions based on an active operational mode of the KEI system. A symbol identifies a particular output action that is directly invocable with a press of a key of the key based input device that corresponds to a graphical key over which said symbol is presented.

Claims

1. A method for providing a user interface (UI) that enhances computer interactions when using a keyboard, the computer having at least a processor, a display, and a keyboard comprising a plurality of physical keys each with a switching mechanism that is operated by a user physically manipulating the key between a resting up position and a depressed down position, the method comprising:

by operation of at least the computer processor;

presenting on the display, a plurality of graphical keys that correspond to the plurality of physical keys of the keyboard;

defining a plurality of symbols for a plurality of output actions that are invocable by manipulating one or more physical keys of the keyboard; and

presenting a set of the plurality of symbols over a set of the plurality of graphical keys based on an active operational mode of a plurality of operational modes of the UI, wherein the set of graphical keys over which the set of symbols are presented identify a set of physical keys of the keyboard that correspond to the set of graphical keys and that are usable to directly invoke a set of output actions identified by the set of symbols while said operational mode remains active, wherein the set of output actions identified by the set of symbols are different than default output actions indicated by the set of physical keys of the keyboard.

2. The method of claim 1 further comprising presenting at least one widget over at least one graphical key of the set of graphical keys, wherein said widget displays and directly controls functionality of a UI element of software running on the computer.

3. The method of claim 1, wherein the set of symbols is a first set of symbols and the set of graphical keys is a first set of graphical keys, wherein the method further comprises presenting a second set of the plurality of symbols over a second set of the plurality of graphical keys to identify a set of physical keys of the keyboard that correspond to the second set of graphical keys and that are usable to directly invoke a different set of output actions based on a different active operational mode of the UI, and wherein the second set of symbols differ from the first set of symbols.

4. The method of claim 1, wherein the plurality of operational modes comprises an operational mode that automatically becomes active when focus of a frontmost application is not on a text field, and wherein the set of symbols identifies a set of output actions of the frontmost application that can be directly invoked by manipulating one of the plurality of physical keys of the keyboard that correspond to the set of graphical keys over which the set of symbols are presented.

5. The method of claim 4, wherein at least one symbol in the set of symbols displays status information from a UI element of the frontmost application.

6. The method of claim 4, wherein at least one symbol in the set of symbols further identifies a global output action that is distinct from the frontmost application.

7. The method of claim 4, wherein at least one symbol in the set of symbols identifies a particular output action that has not been implemented by the frontmost application and the particular output action performs functionality that causes change within the frontmost application when invoked.

8. The method of claim 1, wherein the plurality of operational modes comprises (i) a first operational mode that automatically becomes active when focus is on a first UI element of a frontmost application and (ii) a second operational mode that automatically becomes active when focus is on a second UI element of the frontmost application, and wherein the set of symbols change to a first set of the plurality of symbols that are presented over a first set of the plurality of graphical keys to identify a first set of output actions associated with the first UI element when the first operational mode is active and the set of symbols change to a second set of the plurality of symbols that are presented over a second set of the plurality of the graphical keys to identify a second set of output actions associated with the second UI element when the second operational mode is active.

9. The method of claim 1, wherein the plurality of operational modes comprises an operational mode that becomes active by a press and hold of a particular physical key of the keyboard, and wherein the set of symbols identifies a set of output actions that becomes accessible using a set of physical keys of the keyboard while the particular physical key is held down.

10. The method of claim 1, wherein the plurality of operational modes comprises an operational mode that becomes active when focus of a frontmost application is on a text field and no characters are selected within said text field, and wherein the set of symbols identifies characters for insertion into said text field using the plurality of physical keys of the keyboard.

11. The method of claim 1, wherein the plurality of operational modes comprises an operational mode that becomes active when focus of a frontmost application is on a text field and at least one character is selected within said text field, and wherein the set of symbols identifies a set of output actions that comprises output actions for interacting with said at least one selected character.

12. A method of accessing output actions for an application that is not frontmost through a key driven user interface (UI), wherein the key driven UI comprises a plurality of graphical keys that correspond to a plurality of physical keys of a keyboard, each physical key with a switching mechanism that is operated by a user physically manipulating the physical key between a resting up position and a depressed down position, the method comprising:

detecting a hold of a particular physical key of the keyboard while a first application is frontmost with key focus;

presenting on a display, the plurality of graphical keys that correspond to the plurality of physical keys of the keyboard;

retrieving from a data store, (i) a mapping of a set of the plurality of physical keys of the keyboard that can be used to invoke output actions of a second application while the particular physical key is held and (ii) a set of symbols identifying the output actions of the second application, wherein the second application is not frontmost and does not have key focus; and

displaying while the first application remains frontmost with key focus, the set of symbols corresponding to the output actions of the second application over a set of the plurality of graphical keys corresponding to the set of physical keys of the keyboard based on the particular physical key hold, wherein each symbol in the set of symbols identifies an output action of the second application that is accessible using a physical key other than the particular physical key of keyboard that corresponds to a graphical key over which the symbol is displayed.

13. The method of claim 12 further comprising invoking an output action of the second application while the first application remains frontmost by tapping a physical key of the keyboard that corresponds to a graphical key over which a symbol of the first set of symbols is displayed while continuing to hold the particular physical key.

14. The method of claim 12, wherein the particular physical key is a first key and the set of symbols is a first set of symbols, the method further comprising detecting a hold of a second key of the keyboard while the first application is frontmost and displaying, while the first application remains frontmost with key focus, a second set of symbols over a different set of the plurality of graphical keys based on the second physical key hold, wherein each symbol in the second set of symbols identifies an output action of a third application that is accessible using a physical key of the keyboard that corresponds to a graphical key over which a symbol of the second set of symbols is displayed.

15. The method of claim 12, wherein the set of symbols is a first set of symbols and the set of the plurality of graphical keys is a first set of keys, the method further comprising displaying a second set of symbols over a second set of the plurality of graphical keys when the particular physical key is released, wherein each symbol in the second set of symbols identifies an output action of the first application that is accessible using a physical key of the keyboard that corresponds to a graphical key over which a symbol of the second set of symbols is displayed.

16. The method of claim 15 further comprising displaying a third set of symbols over a third set of the plurality of graphical keys when the particular physical key is released and a third application is frontmost with key focus, wherein each particular symbol of the third set of symbols identifies an output action of the third application that is accessible using a physical key of the keyboard that corresponds to a graphical key over which the particular symbol of the third set of symbols is displayed.

17. A method for enabling access to different sets of output actions in a key driven user interface (UI) for a computer system comprising a keyboard with a plurality of physical keys each with a switching mechanism that is operated by a user physically manipulating the key between a resting up position and a depressed down position, the method comprising:

monitoring system state information of the computer system;

determining which of a plurality of operational modes to activate based on said system state information;

enabling access to a first set of output actions when a first operational mode is active;

enabling access to a second set of output actions when a second operational mode is active;

presenting, on a display, a plurality of graphical keys that correspond to the plurality physical keys of the keyboard and a different set of symbols over a different set of the plurality of graphical keys based on the operational mode that is active, wherein the set of symbols identify the first set of output action when the first operational mode is active and the second set of output actions when the second operational mode is active, and wherein the set of graphical keys over which the set of symbols are presented identify a set of physical keys of the keyboard that correspond to the set of graphical keys and that are usable to directly invoke the set of output actions identified by the set of symbols while the operational mode is active;

generating within the key driven UI, a first message in response to pressing of a particular keyboard physical key when the first operational mode is active and a second message in response to pressing of the particular keyboard physical key when the second operational mode is active, wherein the first message comprises a first identifier identifying the first operational mode and a second identifier identifying the particular keyboard physical key, and wherein the second message comprises a second identifier identifying the second operational mode and a second identifier identifying the particular keyboard physical key; and

executing at least one of a first output action in response to the first message matching to a first message-to-output action mapping of the key driven UI and a different second output action in response to the second message matching to a second message-to-output action mapping of the key driven UI.

18. The method of claim 17, wherein monitoring system state information comprises monitoring which of a plurality of applications is frontmost, and wherein determining which of the plurality of operational modes to activate comprises activating the first operational mode when a first application is frontmost and activating the second operational mode when a second application is frontmost.

19. The method of claim 17, wherein monitoring system state information comprises monitoring which of a plurality of windows of a frontmost application has focus, and wherein determining which of the plurality of operational modes to activate comprises activating the first operational mode when a first window of the frontmost application has focus and activating the second operational mode when a second window of the frontmost application has focus.

20. The method of claim 17, wherein monitoring system state information comprises monitoring which of a plurality of UI elements of a frontmost application has focus, and wherein determining which of the plurality of operational modes to activate comprises activating the first operational mode when a first UI element of the frontmost application has focus and activating the second operational mode when a second UI element of the frontmost application has focus.