MOTION AND CONTEXT SHARING FOR PEN-BASED COMPUTING INPUTS

• US 20130257777A1
• Filed: 05/28/2013
• Published: 10/03/2013
• Est. Priority Date: 02/11/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Benko, Hrvoje, Hinckley, Kenneth Paul, Chen, Xiang
• Granted Patent: US 9,201,520 B2
• Time in Patent Office: Days
• US Class Current: 345/173
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/0383 Signal control means within the pointing device

Abstract

A “Motion and Context Sharing Technique” uses a pen or stylus enhanced to incorporate multiple sensors, i.e., a “sensor pen,” and a power supply to enable various input techniques and gestures. Various combinations of pen stroke, pressure, motion, and other sensor pen inputs are used to enable various hybrid input techniques that incorporate simultaneous, concurrent, sequential, and/or interleaved, sensor pen inputs and touch inputs (i.e., finger, palm, hand, etc.) on displays or other touch sensitive surfaces. This enables a variety of motion-gesture inputs relating to the context of how the sensor pen is used or held, even when the pen is not in contact or within sensing range of the computing device digitizer. In other words, any particular touch inputs or combinations of touch inputs are correlated with any desired sensor pen inputs, with those correlated inputs then being used to initiate any desired action by the computing device.

Claims

1. A computer-implemented process, comprising using a computer to perform process actions for:

receiving sensor inputs from a plurality of sensors coupled to a stylus device;

detecting zero or more finger touches on any touch sensitive surface of a touch-sensitive computing device;

evaluating the sensor inputs in combination with any finger touches to identify an intended motion gesture from a plurality of available motion gestures; and

initiating a user interface action corresponding to the intended motion gesture in an application executing on the touch-sensitive computing device.

2. The computer-implemented process of claim 1 wherein one or more of the sensors are used to determine motion of the stylus device relative to motion of the touch-sensitive computing device, and wherein those motions are correlated for use in identifying the intended motion gesture.

3. The computer-implemented process of claim 1 further comprising initiating an alert on the touch-sensitive computing device when one or more sensors coupled to the touch-sensitive computing device indicate that the touch-sensitive computing device is moving away from the stylus device.

4. The computer-implemented process of claim 1 wherein one or more of the sensors are used to determine orientation of the stylus device relative to the touch-sensitive computing device, and wherein that relative orientation is used to identify the intended motion gesture.

5. The computer-implemented process of claim 1 wherein the stylus device is outside of a hover range of the touch-sensitive computing device while receiving the sensor inputs from the plurality of sensors coupled to the stylus device.

6. The computer-implemented process of claim 1 further comprising a user interface for specifying sensor inputs and touches corresponding to one or more of the plurality of available motion gestures.

7. The computer-implemented process of claim 1 wherein one or more of the sensors indicate a rotation of the stylus device around a long axis of the stylus device concurrently with a touch of an object displayed on a touch screen of the touch-sensitive computing device, and wherein the intended motion gesture corresponds to a user interface action for rotating the touched object.

8. The computer-implemented process of claim 1 wherein one or more of the sensors indicate that the stylus device is within hover range and in an approximately perpendicular orientation relative to a touch screen of the touch-sensitive computing, and wherein the intended motion gesture corresponds to a user interface action for activating a marking menu on the display device.

9. The computer-implemented process of claim 1 wherein one or more of the sensors indicate a rotation of the stylus device around a long axis of the stylus device when the stylus device is beyond hover range of a touch screen of the touch-sensitive computing device, and wherein the intended motion gesture corresponds to a user interface action for undoing a prior action of the application executing on the touch-sensitive computing device.

10. The computer-implemented process of claim 1 wherein one or more of the sensors are used to determine when the stylus device is picked up from a surface, and wherein the intended motion gesture corresponds to a user interface action for presenting a context sensitive menu relating to a currently active object in the application executing on the touch-sensitive computing device.

11. The computer-implemented process of claim 1 wherein one or more of the sensors are used to determine motion of the stylus device and motion of a touch on the touch-sensitive computing device, and when the those motions are in generally the same direction, using the directional correlation of those motions to identify the intended motion gesture.

12. A method for initiating actions in an application executing on a touch-sensitive computing device, comprising:

receiving sensor inputs from a plurality of sensors coupled to a stylus device relative to a touch-sensitive computing device;

detecting zero or more finger touches on any touch sensitive surface of the touch-sensitive computing device;

evaluating the sensor inputs in combination with any finger touches to identify an intended motion gesture from a plurality of available motion gestures; and

initiating a user interface action corresponding to the intended motion gesture in an application executing on the touch-sensitive computing device.

13. The method of claim 12 wherein one or more of the sensors indicate tapping on a barrel of the stylus device when the stylus device is within hover range of the touch-sensitive computing device, and wherein the intended motion gesture corresponds to a user interface action for emulating a button press of an input device.

14. The method of claim 12 wherein one or more of the sensors indicate tapping on a barrel of the stylus device when the stylus device is beyond hover range of the touch-sensitive computing device, and wherein the intended motion gesture corresponds to a user interface action for redoing a prior action of the application executing on the touch-sensitive computing device.

15. The method of claim 12 wherein one or more of the sensors indicate a tilting of the stylus device relative to a display of the touch-sensitive computing device, and wherein a finger touch on the touch-sensitive computing device corresponds to two or more layers objects, and wherein the intended motion gesture corresponds to a user interface action for cycling between the layers.

16. The method of claim 12 wherein the application executing on the touch-sensitive computing device is a painting type application and wherein one or more of the sensors indicate rapping of the stylus device against a surface while there is a touch on a display of the touch-sensitive computing device, and wherein the intended motion gesture corresponds to a user interface action for digitally spattering a plurality of digital paint drops around the touch on the display.

17. A computer storage media having computer executable instructions stored therein, said instructions causing a computing device to execute a method comprising:

receiving sensor inputs from a plurality of sensors coupled to a stylus device relative to a touch-sensitive computing device;

detecting zero or more finger touches on any touch sensitive surface of the touch-sensitive computing device;

evaluating the sensor inputs in combination with any finger touches to identify an intended motion gesture from a plurality of available motion gestures; and

initiating a user interface action corresponding to the intended motion gesture in an application executing on the touch-sensitive computing device.

18. The computer storage media of claim 17 wherein one or more of the sensors are used to determine motion of the stylus device relative to motion of the touch-sensitive computing device, and wherein those motions are correlated for use in identifying the intended motion gesture.

19. The computer storage media of claim 17 further comprising initiating an alert on the touch-sensitive computing device when one or more sensors coupled to the touch-sensitive computing device indicate that the touch-sensitive computing device is moving away from the stylus device.

20. The computer storage media of claim 17 wherein one or more of the sensors are used to determine orientation of the stylus device relative to the touch-sensitive computing device, and wherein that relative orientation is used to identify the intended motion gesture.