MONITORING POINTER TRAJECTORY AND MODIFYING DISPLAY INTERFACE

• US 20110083089A1
• Filed: 10/01/2009
• Published: 04/07/2011
• Est. Priority Date: 10/01/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Murillo, Oscar E., Pahud, Michel, Karlson, Amy K., Bederson, Benjamin B.
• Granted Patent: US 8,261,211 B2
• Time in Patent Office: Days
• US Class Current: 715/765
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

Apparatus and methods for improving touch-screen interface usability and accuracy by determining the trajectory of a pointer as it approaches the touch-screen and modifying the touch-screen display accordingly. The system may predict an object on the display the user is likely to select next. The system may designate this object as a Designated Target Object, or DTO. The system may modify the appearance of the DTO by, for example, changing the size of the DTO, or by changing its shape, style, coloring, perspective, positioning, etc.

Claims

1. A method of modifying a display on a touch-enabled device comprising the steps of:

sensing a pointer spaced apart from the display as the pointer approaches the display;

determining a 3-dimensional trajectory of the pointer in space;

designating a target object on the display based upon the determined trajectory while the pointer is spaced apart from the display; and

modifying an appearance of the designated target object.

2. The method of claim 1 wherein the step of modifying the appearance of the target object includes the step of changing at least one of the following:

the size of the target object;

the perspective of the target object;

the shape of the target object;

the coloring of the target object;

the style of the target object;

or

the position of the target object relative to other objects on the display.

3. The method of claim 1 further comprising the step of determining a pointer type, and wherein the designating step further comprises designating the target object based upon the pointer type.

4. The method of claim 3 wherein the designating step ignores certain objects based upon pointer type.

5. The method of claim 3 wherein the modifying step further modifies the appearance of the target object based upon the pointer type.

6. The method of claim 1 further comprising the step of storing data regarding behavior patterns of a user, and wherein the designating step further designates the target object based upon the stored behavior patterns of the user.

7. The method of claim 1 further comprising the step of storing data regarding usage patterns of an application, and wherein the designating step further designates the target object based upon the stored usage patterns of the application.

8. The method of claim 1 wherein the step of determining a 3-dimensional trajectory of the pointer includes the step of determining a curvature of the trajectory,

9. The method of claim 8, further including the step of changing the display appearance based upon the curvature of the trajectory such that hidden objects become visible.

10. The method of claim 1 wherein the determining, designating, and modifying steps are repeated as the pointer moves with respect to the display.

11. The method of claim 10 wherein the designating step includes the step of changing the designated target object.

12. The method of claim 10 wherein the designating step includes the step of un-designating a previously designated target object when the determining step determines that the trajectory is leading away from the previously designated target object.

13. A system for modifying a display on a touch-enabled device configured to accept input from a pointer comprising:

an input trajectory element configured to determine a 3-dimensional trajectory of the pointer when the pointer is spaced apart from the touch-enabled device display, based upon a detected 3-dimensional position of the pointer over time; and

a predictor/adapter configured to provide a display control signal modifying an appearance of the display on the touch-enabled device based upon the determined trajectory of the pointer.

14. The system of claim 13, wherein the predictor/adaptor is configured to designate a target object on the touch-enabled device display based upon the determined trajectory, and wherein the predictor/adaptor is further configured to provide a control signal for changing an appearance of the designated target object on the display.

15. The system of claim 14, further comprising a pointer type determination element configured to determine a type for the pointer, and wherein the predictor/adaptor is further configured to designate the target object based also upon the detected type.

16. The system of claim 14, further comprising a pointer type determination element configured to determine a type for the pointer, and wherein the predictor/adaptor is further configured to provide a display control signal modifying the appearance of the designated target object based also upon the detected type.

17. A method of modifying a display on a touch-enabled device comprising the steps of:

sensing a pointer when it is spaced apart from the touch-enabled device display;

determining a position of the pointer in 3-d space over time;

computing a 3-dimensional trajectory of the pointer based upon the determined position over time; and

modifying an appearance of the touch-enabled device display based upon the 3-dimensional trajectory.

18. The method of claim 17, further comprising the steps of:

designating a target object on the display based upon the 3-dimensional trajectory; and

modifying the appearance of the target object.

19. The method of claim 18, further comprising the steps of determining a pointer type and modifying the appearance of the target object based upon the pointer type.

20. The method of claim 18, further comprising the step of determining a pointer type and wherein the step of modifying the appearance of the target object further modifies the appearance of the target object based upon the pointer type.