Monitoring pointer trajectory and modifying display interface

• US 8,261,211 B2
• Filed: 10/01/2009
• Issued: 09/04/2012
• Est. Priority Date: 10/01/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Pahud, Michel, Murillo, Oscar E., Karlson, Amy K., Bederson, Benjamin B.
• Primary Examiner(s): LEE, TING ZHOU
• Application Number:
  US12/571,448
• Publication Number:
  US 20110083089A1
• Time in Patent Office: 1,069 Days
• Field of Search: 715/764, 715/856, 715/860, 345/173, 345/156, 345/157
• US Class Current: 715/856
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

Apparatus and methods for improving touch-screen interface usability and accuracy by determining the trajectory of a pointer as it approaches the touch-screen and modifying the touch-screen display accordingly. The system may predict an object on the display the user is likely to select next. The system may designate this object as a Designated Target Object, or DTO. The system may modify the appearance of the DTO by, for example, changing the size of the DTO, or by changing its shape, style, coloring, perspective, positioning, etc.

Claims

1. A method of comprising the steps:

sensing a pointer spaced apart from a display screen of a touch-enabled device as the pointer approaches the display screen;

determining a 3-dimensional trajectory of the pointer in space;

determining a pointer type;

designating a target object on the display screen based upon the determined pointer type and the determined 3-dimensional trajectory while the pointer is spaced apart from the display screen, wherein certain objects are not eligible to be designated as the target object based on the pointer type; and

modifying an appearance of the designated target object on the display screen.

2. The method of claim 1 wherein the step of modifying the appearance of the target object includes the step of changing at least one of the following:

the size of the target object;

the perspective of the target object;

the shape of the target object;

the coloring of the target object;

the style of the target object;

or

the position of the target object relative to other objects on the display.

3. The method of claim 1 wherein the modifying step further modifies the appearance of the designated target object based upon the pointer type.

4. The method of claim 1 further comprising the step of storing data regarding behavior patterns of a user, and wherein the designating step further designates the target object based upon the stored behavior patterns of the user.

5. The method of claim 1 further comprising the step of storing data regarding usage patterns of an application, and wherein the designating step further designates the target object based upon the stored usage patterns of the application.

6. The method of claim 1 wherein the step of determining a 3-dimensional trajectory of the pointer includes the step of determining a curvature of the 3-dimensional trajectory.

7. The method of claim 6, further including the step of changing the appearance based upon the curvature of the 3-dimensional trajectory such that hidden objects become visible.

8. The method of claim 1 wherein the determining, the 3-dimensional trajectory, designating, and modifying steps are repeated as the pointer moves with respect to the display.

9. The method of claim 8 wherein the designating step includes the step of changing the designated target object.

10. The method of claim 8 wherein the designating step includes the step of un-designating a previously designated target object when the determining step determines that the trajectory is leading away from the previously designated target object.

11. A system comprising:

a processor;

a memory;

an input trajectory element configured to determine a 3-dimensional trajectory of a pointer when the pointer is spaced apart from a display screen of a touch-enabled device, based upon detected 3-dimensional positioning of the pointer over time;

a pointer type determination element configured to determine a type for the pointer; and

a predictor/adapter configured to provide a display control signal modifying an appearance of the display screen of the touch-enabled device based upon the determined 3-dimensional trajectory of the pointer, wherein the predictor/adapter is further configured to designate a target object on the display screen based upon the determined 3-dimensional trajectory and the determined pointer type, wherein certain objects are not eligible to be designated as the designated target object based on the determined pointer type, and wherein the predictor/adapter is further configured to provide another control signal for changing an appearance of the designated target object on the display screen.

12. The system of claim 11, wherein the predictor/adapter is further configured to provide the another control signal for changing the appearance of the designated target object based also upon the determined pointer type.

13. A method comprising the steps of:

sensing a pointer when the pointer is spaced apart from a display screen of a touch-enabled device;

determining a position of the pointer in 3-dimensional space over time;

computing a 3-dimensional trajectory of the pointer based upon the determined position over time;

modifying an appearance of the display screen based upon the 3-dimensional trajectory;

determining a pointer type for the pointer;

designating a target object on the display screen based upon the computed 3-dimensional trajectory and the determined pointer type, wherein certain objects are not eligible to be designated as the target object based on the pointer type;

and

modifying an appearance of the designated target object on the display screen.

14. The method of claim 13, wherein the step of modifying the appearance of the designated target object further modifies the appearance of the designated target object based upon the determined pointer type.