Computer interaction based upon a currently active input device

• US 7,802,202 B2
• Filed: 03/17/2005
• Issued: 09/21/2010
• Est. Priority Date: 03/17/2005
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Fox, Eric B., Jarrett, Robert J., Keely, Leroy B., Rimas-Ribikauskas, Emily K.
• Primary Examiner(s): Chow; Dennis-Doon
• Assistant Examiner(s): Kumar; Anil N
• Application Number:
  US11/083,896
• Publication Number:
  US 20060209016A1
• Time in Patent Office: 2,014 Days
• Field of Search: None
• US Class Current: 715/831
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

Described is a computer-implemented system and method that dynamically detects which input device (e.g., pen or mouse) is currently in use, and based on the device, varies a program'"'"'s user interaction model to better optimize the user'"'"'s ability to interact with the program via that input device. A tablet input subsystem receives pen and touch data, and also obtains keyboard and mouse data. The subsystem analyzes the data and determines which input device is currently active. The active device is mapped to an interaction model, whereby different user interface appearances, behaviors and the like may be presented to the user to facilitate improved interaction. For example, a program may change the size of user interface elements to enable the user to more accurately scroll and make selections. Timing, tolerances and thresholds may change. Pen hovering can become active, and interaction events received at the same location can be handled differently.

Claims

1. In a computing device, a computer-implemented method for facilitating an improved user interaction experience by dynamically modifying one or more user interface elements of at least one computer program, the method comprising:

detecting, dynamically during runtime, which input device of the plurality of input devices of the computing device is a currently active input device;

determining that the detected currently active input device is associated with one or more user characteristics associated with the user that indicate one or more user interface element behavior modifications that are to be applied during use of any detected currently active input device associated with the user, the behavior modifications comprising functional changes to the manner in which each interface element reacts to inputs from the currently active device, wherein an input gesture received from a first input device has a different meaning than the same input gesture received from a second, different input device, such that the same input gesture results in different actions being performed depending on which input device is determined to be currently active;

mapping the currently active input device to a user interaction model for that device which is selected from a plurality of user interaction models each corresponding to a variety of different input devices, the selected user interaction model corresponding with the currently active input device, the user interaction model including one or more user interface elements that are disabled based on which device is selected;

modifying the user interface behavior of the at least one computer program to which the currently active input device is providing input by modifying at least one user interface element of the at least one computer program, wherein said modification is based on the selected user interaction model which corresponds to the detected currently active device, and wherein the modification includes changing the appearance of at least one interface element and changing the interface behavior of at least one interface element according to the appropriate corresponding user interaction model, the interface behavior change including altering at least a gesture input timing threshold for the interaction model based on which input device is determined to be currently active, such that gesture input timing is altered to improve gesture recognition for the currently active input device.

2. The method of claim 1 wherein detecting which input device is the currently active input device comprises identifying the input device based on received input data.

3. The method of claim 1 wherein modifying the user interface behavior of the at least one computer program comprises providing visible, audible and/or tactile feedback to a user.

4. The method of claim 1 wherein modifying the user interface behavior of the at least one computer program comprises changing a size of a user interface element to cover a different interactive area, wherein the element comprises at least one element of a set, the set containing, a scrollbar user interface element, a menu selection user interface element, a button user interface element, an icon user interface element, text, a link, a radio button user interface element, and a check box user interface element.

5. The method of claim 1 wherein modifying the user interface behavior of the at least one computer program comprises changing at least two data values of a set of data, the set including appearance, timing, tolerance and threshold data.

6. The method of claim 1 wherein modifying the user interface behavior of the at least one computer program comprises handling input received at a same location differently based upon the currently active device.

7. The method of claim 1 wherein detecting which input device of the plurality of input devices is the currently active input device comprises, selecting an input device from the plurality, the plurality including at least two of, a touch digitizer, a pen digitizer, a mouse, a keyboard, a microphone, a camera and a television-style remote control device.

8. A computer-readable storage medium having computer-executable instructions stored thereon which, when executed, perform steps comprising the steps of the method of claim 1.

9. The computer-readable storage medium of claim 8 wherein modifying the at least one user interface element comprises providing visible, audible and/or tactile feedback to a user.

10. The computer-readable storage medium of claim 8 wherein modifying the at least one user interface element comprises changing a size of a user interface element to cover a different interactive area.

11. The method of claim 1, further comprising:

recognizing a different input device as an active input device after the different input device has been activated for a predetermined minimum time; and

ignoring the different input device if the different input device has not been activated for the predetermined minimum time.

12. The method of claim 1, further comprising determining that the detected currently active input device is one of a touch digitizer or pen digitizer;

and further, wherein modifying at least one user interface element of the selected user interaction model comprises;

determining that one of a pen device or a user'"'"'s finger is hovering over the at least one user interface element; and

magnifying the user interface element over which the pen device or user'"'"'s finger is hovering, such that the magnified user interface element is selectable by the pen device or user'"'"'s finger.

13. The method of claim 1, wherein modifying at least one user interface element of the selected user interaction model comprises:

determining that a user'"'"'s speech input is to be used to interact with the user interaction model; and

magnifying the at least one user interface element such that the element is distinguishable from a greater distance away from the element'"'"'s display.

14. The method of claim 1, further comprising modifying a displayed mouse icon, such that the displayed mouse icon corresponds to the detected currently active input device.

15. The method of claim 1, further comprising:

determining that one or more interaction selections are to be displayed based on which device was detected as the currently active input device; and

based on the determination, displaying the one or more interaction selections.

16. The method of claim 1, further comprising allowing a user to change user interaction model behavior such that, in the same general data-receiving state, each input device, upon inputting an input, performs a different function.

17. The method of claim 16, wherein a gesture that a user makes with an input device has a plurality of meanings to the user interaction model, depending on which device is the currently active input device.

20. The method of claim 1, further comprising determining that the detected currently active input device is one of a touch digitizer or a pen digitizer;

and

further, wherein modifying the user interface behavior of the at least one program comprises;

determining that one of a user'"'"'s finger or a pen device was used to interact with the touch digitizer or the pen digitizer, the user interaction constituting a touch in the region representing a particular user interface element;

magnifying the touched user interface element;

leaving the user interface element magnified for a time period constituting a reasonable amount of time;

detecting a second user touch with the user'"'"'s finger or the pen device in the general location of the user interface element or in the magnified area during the time period, the user thereby indicating a user selection of the user interface element; and

causing the magnified region to disappear if the user touches a location outside the region representing the user interface element or if the time period expires before the user interacts with the touch digitizer or pen digitizer for a second time.

18. A method for modifying one or more user interface elements of a computer program on a computer system based on which input device is currently selected and one or more user preferences associated with the currently selected input device, the method comprising:

detecting, dynamically during runtime, which input device of a plurality of input devices of the computing device is a currently active input device;

determining that the detected currently active input device is associated with one or more user characteristics associated with the user including the user'"'"'s hand preference, the user characteristics indicating one or more user interface element behavior modifications that are to be applied during use of any detected currently active input device associated with the user, the behavior modifications comprising functional changes to the manner in which each interface element reacts to inputs from the currently active device, wherein an input gesture received from a first input device has a different meaning than the same input gesture received from a second, different input device, such that the same input gesture results in different actions being performed depending on which input device is determined to be currently active and wherein the behavior modifications include changing both the timing and tolerances for click-events and gestures;

determining, based on user characteristics indicating the user'"'"'s hand preference, that one or more user interface elements are to be shifted horizontally during use of the detected currently active input device, such that the user'"'"'s hand is prevented from blocking the user interface elements;

mapping received input from the currently active input device to a user interaction model for that device which is selected from among a plurality of user interaction models each corresponding to a variety of different input devices, the selected user interaction model corresponding with the currently active input device, the user interaction model including one or more user interface elements that are disabled based on which device is selected; and

modifying at least one user interface element of the selected user interaction model based on the detected currently active device, the modifying including horizontally shifting the one or more user interface elements such that blocking of the user interface elements by the user'"'"'s hand is prevented, the modifying further including altering both a gesture input timing threshold and a gesture input tolerance setting for the interaction model based on which input device is determined to be currently active, such that both the gesture input timing and the gesture input tolerance settings are altered to improve gesture recognition for the currently active input device.

19. The method of claim 18, wherein the one or more user interface elements are shifted horizontally left when the user'"'"'s hand preference is determined to be right-handedness.