Sensor correlation for pen and touch-sensitive computing device interaction

• US 10,168,827 B2
• Filed: 07/01/2017
• Issued: 01/01/2019
• Est. Priority Date: 06/12/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Hinckley, Ken, Benko, Hrvoje, Pahud, Michel, Wilson, Andrew D., Irani, Pourang Polad, Guimbretiere, Francois
• Primary Examiner(s): Pervan, Michael
• Application Number:
  US15/640,507
• Publication Number:
  US 20170300170A1
• Time in Patent Office: 549 Days
• Field of Search: 178 1801- 1907, 345173-183
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/0383 Signal control means within the pointing device
• CPC Class Codes: G06F 3/04162 for exchanging data with external devices, e.g. smart pens, via the digitiser sensing hardware
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

Pen and computing device sensor correlation technique embodiments correlate sensor signals received from various grips on a touch-sensitive pen and touches to a touch-sensitive computing device in order to determine the context of such grips and touches and to issue context-appropriate commands to the touch sensitive pen or the touch-sensitive computing device. A combination of concurrent sensor inputs received from both a touch-sensitive pen and a touch-sensitive computing device are correlated. How the touch-sensitive pen and the touch-sensitive computing device are touched or gripped are used to determine the context of their use and the user'"'"'s intent. A context-appropriate user interface action based can then be initiated. Also the context can be used to label metadata.

Claims

1. A computer-implemented process for initiating user interface actions, comprising using a computer for:

determining a user'"'"'s preferred hand and non-preferred hand from sensor inputs, wherein the user'"'"'s preferred hand is determined by;

sensing sensor signals representing a bump in a motion of a touch-sensitive pen when a contact is made with a touch-sensitive computing device; and

correlating sensor signals representing the bump in the touch-sensitive pen'"'"'s motion with signals representing a motion of the touch-sensitive computing device to determine the user'"'"'s preferred hand; and

initiating a user interface action based on the determination of the user'"'"'s preferred hand and non-preferred hand.

2. The computer-implemented process of claim 1, further comprising determining the user'"'"'s preferred hand by:

sensing the sensor signals representing the bump in a motion of the touch-sensitive pen when the contact is made with the touch-sensitive computing device; and

correlating the sensor signals representing the bump in the touch-sensitive pen'"'"'s motion with the sensor signals representing the motion of the touch-sensitive computing device to determine that the hand holding the touch-sensitive pen is the preferred hand.

3. The computer-implemented process of claim 1 further comprising:

correlating the sensor signals representing the touch-sensitive computing device motion with touch patterns on the touch-sensitive computing device; and

using the correlated motion and touch patterns on the touch-sensitive computing device to suppress accidental screen content rotation on a display of the touch-sensitive computing device.

4. The computer-implemented process of claim 1 wherein the user interface action is displaying a menu on a display of the touch-sensitive computing device.

5. The computer-implemented process of claim 4 wherein the menu displayed is dependent on whether the user'"'"'s preferred hand or non-preferred hand is touching the touch-sensitive computing device.

6. The computer-implemented process of claim 1, further comprising:

determining whether the user is making a gesture with the user'"'"'s preferred hand or non-preferred hand.

7. The computer-implemented process of claim 1, further comprising:

determining grip patterns on the touch-sensitive pen and grip patterns on the touch-sensitive computing device;

using the grip patterns, determining whether the user is making a gesture with a bare hand or whether the user is making a gesture with a hand holding the touch-sensitive pen.

8. The computer-implemented process of claim 7, further comprising:

determining that the touch-sensitive pen is held in a stowed position in the user'"'"'s preferred hand;

detecting a contact from the user'"'"'s preferred hand on a display of the touch-sensitive computing device; and

displaying context-specific tools in response to the contact.

9. The computer-implemented process of claim 7, further comprising:

determining that the touch-sensitive pen is held in the user'"'"'s preferred hand in a grip indicating that the touch-sensitive pen is not in a position to write;

detecting a contact from the user'"'"'s non-preferred hand on a display of the touch-sensitive computing device; and

displaying context-specific tools in response to the contact.

10. The computer-implemented process of claim 7, further comprising:

determining that the touch-sensitive pen is held in a grip indicating that the touch-sensitive pen is in a position to write in the user'"'"'s preferred hand;

detecting a contact from the user'"'"'s non-preferred hand on a display of the touch-sensitive computing device; and

displaying context-specific tools in response to the contact.

11. The computer-implemented process of claim 7, further comprising:

determining that the touch-sensitive pen is held in a grip indicating that the touch-sensitive pen is in a position to write in the user'"'"'s preferred hand;

detecting a multi-finger touch gesture from the user'"'"'s non-preferred hand on the display of the touch-sensitive computing device; and

displaying context-specific tools in response to the multi-finger touch gesture.

12. The computer-implemented process of claim 1, further comprising:

determining which hand a user is holding the touch-sensitive computing device with; and

displaying a menu in a location on the screen of the touch-sensitive computing device that can be activated by the user with the thumb of the hand holding the touch-sensitive computing device.

13. The computer-implemented process of claim 1, wherein one or more of the signals represent a finger print of a user which is used to identify the user.

14. The computer-implemented process of claim 1, further comprising using historical signal data to improve the correlation of the signals.

15. A computer-implemented process for initiating user interface actions, comprising using a computer for:

determining a user'"'"'s preferred hand and non-preferred hand from sensor inputs, wherein the user'"'"'s preferred hand is determined by;

sensing sensor signals representing a bump in a motion of a first touch-sensitive device when a contact is made with a second touch-sensitive device; and

correlating sensor signals representing the bump in the first touch-sensitive device'"'"'s motion with signals representing a motion of the second touch-sensitive device to determine the user'"'"'s preferred hand; and

using the determination of the user'"'"'s preferred hand and non-preferred hand in an application.

16. The computer-implemented process of claim 15 wherein the determination of the user preferred hand and non-preferred hand is used to initiate a context-appropriate command using the application.

17. The computer-implemented process of claim 16, wherein the context-appropriate action is enabling context-appropriate functionality on the first touch-sensitive computing device or the second touch-sensitive computing device.

18. The computer-implemented process of claim 16, wherein the determination of the user'"'"'s preferred hand or non-preferred hand is used to detect unintentional contact with either the first touch-sensitive computing device or the second touch-sensitive computing device.

19. The computer-implemented process of claim 16, wherein using the determination of the user'"'"'s preferred hand and non-preferred hand is used in determining a user'"'"'s grip on either the first touch-sensitive computing device or the second touch-sensitive computing device.