Detecting gestures involving intentional movement of a computing device

• US 8,994,646 B2
• Filed: 12/17/2010
• Issued: 03/31/2015
• Est. Priority Date: 12/17/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hinckley, Kenneth P., Pahud, Michel, Shen, Wenqi
• Primary Examiner(s): Mengistu, Amare
• Assistant Examiner(s): Figueroa-Gibson, Gloryvid
• Application Number:
  US12/970,939
• Publication Number:
  US 20120154293A1
• Time in Patent Office: 1,565 Days
• Field of Search: 345/649, 345/660, 345/672, 345156-184, 715/788, 715/815, 715/863, 715/864, 715733-740, 715/769
• US Class Current: 345/156
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04162 for exchanging data with external devices, e.g. smart pens, via the digitiser sensing hardware
• CPC Class Codes: G06F 3/0483 Interaction with page-structured environments, e.g. book metaphor
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: H04M 1/724 User interfaces specially adapted for cordless or mobile telephones
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion telephones including GPS signal receivers H04M2250/10
• CPC Class Codes: H04M 2250/22 including a touch pad, a touch sensor or a touch detector digitizers for touch screens G06F3/041

Abstract

A computing device is described herein which accommodates gestures that involve intentional movement of the computing device, either by establishing an orientation of the computing device and/or by dynamically moving the computing device, or both. The gestures may also be accompanied by contact with a display surface (or other part) of the computing device. For example, the user may establish contact with the display surface via a touch input mechanism and/or a pen input mechanism and then move the computing device in a prescribed manner.

Claims

1. A computing device, comprising:

at least one contact-type input mechanism for providing at least one contact input event that indicates contact with the computing device;

wherein at least one contact input event corresponds to a selection of content rendered on a display surface of the computing device;

at least one sensing-type input mechanism for providing input to an interpretation and behavior selection module (IBSM), the at least one sensing-type input mechanism providing sensor data corresponding to a target entity towards which the computing device is pointed, the IBSM automatically determining a previously unregistered location and a corresponding identity of the target entity from the sensor data corresponding to the target entity;

wherein the IBSM jointly evaluates the selected content in combination with the location and identity of the target entity to detect a compound gesture;

the IBSM enabling the computing device to execute behavior that is associated with the compound gesture.

2. The computing device of claim 1, wherein the computing device is a handheld computing device.

3. The computing device of claim 1, wherein said at least one contact-type input mechanism comprises a touch input mechanism for sensing contact of a hand with the display surface of the computing device.

4. The computing device of claim 1, wherein the computing device further includes at least one movement-type input mechanism for providing at least one movement input event that indicate one or more of orientation and motion of the computing device.

5. The computing device of claim 4, wherein the IBSM is configured to detect the compound gesture when intentional movement of the computing device is performed in conjunction with contact with content that is presented on the display surface of the computing device.

6. The computing device of claim 5, wherein the intentional movement of the computing device in the prescribed manner involves rotation of the computing device.

7. The computing device of claim 6, wherein an angle of the rotation, with respect to an initial angle of the computing device, defines a manner in which the behavior is executed.

8. The computing device of claim 6, wherein a rate at which the rotation is performed defines a manner in which the behavior is executed.

9. The computing device of claim 1, wherein the target entity is a separate computing device and wherein the behavior is transmitting a selected object to the target entity.

10. The computing device of claim 1, wherein the target entity is a person and wherein the behavior is transmitting a selected object to a computing device associated with the target entity.

11. The computing device of claim 1, wherein the contact input event comprises selecting a visual representation of at least one file on a touch-sensitive display of the computing device.

12. The computing device of claim 11, wherein the target entity is a physical trash bucket that is a tangible object located at some distance away from the computing device, and wherein the behavior that is executed comprises automatically deleting said at least one file from the computing device.

13. The computing device of claim 11, wherein the behavior that is executed comprises automatically syncing said at least one file with corresponding files on one or more computing devices associated with the target entity.

14. The computing device of claim 1, wherein the computing device presents feedback to indicate that the compound gesture has been recognized.

15. The computing device of claim 1, further comprising applying the IBSM to determine that two or more of the computing device of claim 1 are physically pointed towards the same target entity and wherein the behavior that is executed is to automatically initiate sharing of capabilities of the target entity between those two or more of the computing device of claim 1 that are physically pointed towards the same target entity.

16. A method for controlling a computing device in response to an intentional movement of the computing device, comprising:

receiving at least one multi-touch gesture input event from at least one touch input mechanism in response to establishing a contact point on a surface of the computing device in a prescribed manner;

receiving at least one movement input event from at least one movement-type input mechanism in response to movement of the computing device in a prescribed manner;

using a sensor input mechanism for receiving sensor data corresponding to a target entity towards which the computing device is pointed and automatically determining a previously unregistered location and a corresponding identity of the target entity from the sensor data corresponding to the target entity;

wherein at least one movement input event is concurrent with at least one multi-touch gesture input event while the computing device is pointed at the target entity;

jointly evaluating the concurrent movement and multi-touch gesture input events in combination with the location and identity of the target entity for detecting a compound gesture; and

wherein the compound gesture is used for instructing the computing device to execute prescribed behavior.

17. The method of claim 16, wherein the contact point is established on a display surface of the computing device relative to one or more files graphically presented on the display devices.

18. A computer readable storage device for storing computer readable instructions, the computer readable instructions providing an interpretation and behavior selection module (IBSM) when executed by one or more processing devices, the computer readable instructions comprising:

logic configured to receive at least one multi-touch gesture input event from one or more touch input mechanisms in response to contact with content presented on a display surface of a computing device in a prescribed manner;

logic configured to receive at least one movement input event that is concurrent with the at least one multi-touch gesture input event from one or more movement-type input mechanisms in response to an intentional rotation of the computing device about at least one axis;

logic configured to receive sensor data from a sensing-type input mechanism and to evaluate that sensor data to determine a previously unregistered location and a corresponding identity of a target entity towards which the computing device is pointed;

logic configured to jointly evaluate the concurrent movement and multi-touch gesture input events in combination with the location and identity of the target entity to detect a compound gesture; and

logic configured to execute a behavior associated with the compound gesture.

19. The computer readable storage device of claim 18, wherein a point of contact established by said at least one multi-touch gesture input event defines a location at which zooming occurs.

20. The computer readable storage device of claim 18, wherein an angle of the rotation, with respect to an initial angle of the computing device, defines a manner in which the behavior is executed.