TOUCH EVENT ANTICIPATION IN A COMPUTING DEVICE

• US 20120169646A1
• Filed: 12/29/2010
• Published: 07/05/2012
• Est. Priority Date: 12/29/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Berkes, Otto, Geiger, Avi, Amdur, Meredith E.
• Granted Patent: US 9,354,804 B2
• Time in Patent Office: Days
• US Class Current: 345/174
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04108 Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/042 by opto-electronic means
• CPC Class Codes: G06F 3/044 by capacitive means
• CPC Class Codes: G06F 3/045 using resistive elements, e.g. a single continuous surface or two parallel surfaces put in contact
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

Systems and methods for anticipation of touch events in a computing device are provided. The computing device may include a multi-touch sensitive display including a sensor configured to sense a position and/or movement of a hand. A graphical user interface (GUI) may be displayed, having a state including a plurality of possible touch inputs. The computing device may further include a touch event prediction module configured to compute one or more anticipated touch inputs based on the sensed hand position and/or movement, and the state of the GUI with the plurality of possible user inputs. The computing device may further include a preprocessing module configured to preprocess data for each anticipated touch input, and upon the detection of an actual touch input received from the user that matches one of the anticipated touch inputs, displaying the preprocessed data for the actual touch input on the GUI.

Claims

1. A computing device, comprising:

a multi-touch sensitive display including a sensor configured to sense a position and/or movement of a user hand;

a graphical user interface displayed on the display, the graphical user interface having a state including a plurality of possible touch inputs;

a touch event prediction module configured to compute one or more anticipated touch inputs based on the sensed user hand position and/or movement, and the state of the user interface with the plurality of possible user inputs; and

a preprocessing module configured to generate preprocessed data for each anticipated touch input, and upon the detection of an actual touch input received from the user that matches one of the anticipated touch inputs, cause the preprocessed data for the actual touch input to be displayed on the graphical user interface.

2. The computing device of claim 1, wherein the preprocessing module is configured to preload data to be displayed upon an anticipated touch event into a cache, and/or perform an advance computation for an anticipated touch event.

3. The computing device of claim 1, wherein the sensor is an in-pixel optical sensor, capacitive touch sensor, resistive touch sensor, pressure sensor, or infrared sensor.

4. The computing device of claim 1, wherein the touch event prediction module includes a statistics engine configured to compute an estimated probability that each of the anticipated user inputs is selected based on prior usage data, and to instruct the preprocessing module to allocate preprocessing resources to one or more anticipated user inputs with higher probability of selection and to forgo preprocessing of the other anticipated user inputs.

5. The computing device of claim 1, wherein the touch event prediction module and the preprocessing module are part of an application programming interface linking programs to operating system functionality on the computing device.

6. The computing device of claim 1, wherein the sensor is configured to detect, the position and/or movement of the hand when the hand is spaced apart a distance from a top surface of the display.

7. The computing device of claim 1, wherein the computing device is configured as a mobile telephone, a tablet computing device, a laptop computer, a desktop computer, a tabletop computer, or a kiosk computer.

8. The computing device of claim 1, wherein the sensor is configured to detect position and/or movement of one or more digits of the hand and/or a stylus held by the hand.

9. The computing device of claim 1, wherein the computing device is configured to display an indication to the user of the anticipated touch input, wherein the indication includes changing portions of the graphical user interface to become brighter, change color, fade and/or disappear to provide user feedback of the anticipated touch input.

10. A computing method, comprising:

displaying a graphical user interface including a state having a plurality of possible touch inputs on a touch sensitive display of a computing device;

sensing a position and/or movement of a user hand via a sensor of the computing device;

computing one or more anticipated touch inputs based on the sensed user hand position and/or movement, and the state of the user interface with the plurality of possible touch inputs;

preprocessing data for each anticipated touch input;

detecting an actual touch input that matches one of the anticipated touch inputs; and

displaying the preprocessed data the on the graphical user interface for the actual touch input.

11. The method of claim 10, wherein preprocessing data includes preloading data to be displayed upon performance of each anticipated touch event into a cache.

12. The method of claim 10, wherein preprocessing data includes performing an advance computation for each anticipated touch event.

13. The method of claim 10, further comprising:

computing an estimated probability that each of the anticipated user inputs will be selected based on prior usage data; and

performing preprocessing of data for anticipated user inputs with a higher probability, and forgoing preprocessing for anticipated user inputs with a lower probability.

14. The method of claim 10, wherein computing one or more anticipated touch inputs, and preprocessing data for each anticipated touch input are performed at least in part by an application programming interface linking programs to operating system functionality on the computing device.

15. The method of claim 10, wherein sensing is performed by a sensor selected from the group consisting of in-pixel optical sensor, capacitive touch sensor, resistive touch sensor, and infrared sensor.

16. The method of claim 10, wherein sensing a position and/or movement of a user hand further includes:

detecting the position and/or movement of the hand when the hand is spaced apart a distance from a top surface of the display.

17. The method of claim 10, wherein sensing a position and/or movement of a user hand further includes:

detecting a position and/or movement of one or more digits of the hand or a stylus held her the hand.

18. The method of claim 10, further comprising:

displaying an indication to the user of the anticipated touch input on the display, wherein the indication includes changing portions of the graphical user interface to become brighter, change color, fade and/or disappear to provide user feedback of the anticipated touch input.

19. The computing device of claim 10, wherein the computing device is configured as a mobile telephone, a tablet computing device, a laptop computer, a desktop computer, a tabletop computer, or a kiosk computer.

20. A computing method, comprising:

displaying a graphical user interface on a touch sensitive display of a computing device, the graphical user interface including a state having a plurality of possible touch inputs;

sensing a position and/or movement of a user hand spaced apart from a top surface of the display via a sensor of the computing device;

computing one or more anticipated touch inputs based on the sensed user hand position and/or movement, and the state of the user interface with the plurality of possible touch inputs; and

preprocessing data for each anticipated touch input, by preloading data to be displayed upon performance of each anticipated touch event into a cache and/or by performing an advance computation for each anticipated touch event;

detecting an actual touch input that matches one of the anticipated touch inputs; and

displaying the preprocessed data the on the graphical user interface for the actual touch input;

wherein the computing device is configured as a mobile telephone, a tablet computing device, a laptop computer, a desktop computer, a tabletop computer, or a kiosk computer.