Touch sensitive display apparatus using sensor input

• US 8,581,856 B2
• Filed: 05/27/2009
• Issued: 11/12/2013
• Est. Priority Date: 05/27/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Benko, Hrvoje, Tan, Desney S., Morris, Daniel, Saponas, Timothy Scott
• Primary Examiner(s): Pervan, Michael
• Application Number:
  US12/472,446
• Publication Number:
  US 20100302137A1
• Time in Patent Office: 1,630 Days
• Field of Search: 178 1801- 1907, 345173-178, 715/863
• US Class Current: 345/173
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen

Abstract

Described herein is a system that includes a receiver component that receives gesture data from a sensor unit that is coupled to a body of a gloveless user, wherein the gesture data is indicative of a bodily gesture of the user, wherein the bodily gesture comprises movement pertaining to at least one limb of the gloveless user. The system further includes a location determiner component that determines location of the bodily gesture with respect to a touch-sensitive display apparatus. The system also includes a display component that causes the touch-sensitive display apparatus to display an image based at least in part upon the received gesture data and the determined location of the bodily gesture with respect to the touch-sensitive display apparatus.

Claims

1. A method comprising the following computer-executable acts:

receiving gesture data from an electromyography sensor unit that is in contact with skin of a gloveless user, wherein the electromyography sensor unit is embodied as one of a wristband, a band for placement on a forearm, a device for coupling to a human shoulder, or a necklace, wherein the gesture data is indicative of a gesture performed by the gloveless user, the gesture comprising movement of a first finger on a first hand of the gloveless user relative to a second finger on the first hand, and wherein the gesture data comprises data that identifies the first finger, data that identifies the second finger, and data that identifies the first hand;

determining location of the gesture with respect to a touch-sensitive display apparatus; and

performing a computing process at the touch-sensitive display apparatus based at least in part upon the received gesture data and the determined location of the gesture with respect to the touch-sensitive display apparatus, wherein the computing process is performed based at least in part upon the data that identifies the first finger, the data that identifies the second finger, and the data that identifies the first hand.

2. The method of claim 1, wherein the determined location of the gesture is in a hoverspace of the touch-sensitive display apparatus.

3. The method of claim 1, wherein the gesture data further comprises data that is indicative of an identity of the gloveless user, and wherein the computing process is performed based at least in part upon the data that is indicative of the identity of the gloveless user.

4. The method of claim 1, wherein the gesture comprises touching the touch-sensitive display apparatus with the first finger, wherein the gesture data comprises data that is indicative of an amount of pressure applied to the touch-sensitive display apparatus by the first finger of the gloveless user, and wherein the computing process is performed based at least in part upon the data that is indicative of the amount of pressure applied to the touch-sensitive display apparatus by the first finger.

5. The method of claim 4, wherein the gesture comprises touching the touch-sensitive display apparatus with the second finger, wherein the gesture data comprises data that is indicative of an amount of pressure applied to the touch-sensitive display apparatus by the second finger of the gloveless user, and wherein the computing process is performed based at least in part upon the data that is indicative of the amount of pressure applied to the touch-sensitive display apparatus by the second finger.

6. The method of claim 1, further comprising calibrating the touch-sensitive display apparatus based at least in part upon the received gesture data.

7. The method of claim 1, wherein the gesture comprises touching the touch-sensitive display apparatus with the first finger, and wherein determining the location of the gesture with respect to the touch-sensitive display apparatus comprises determining where on the touch-sensitive display apparatus that the first finger is in physical contact with the touch-sensitive display apparatus.

8. The method of claim 1, wherein determining location of the gesture with respect to the touch-sensitive display apparatus comprises:

emitting non-visible light through a diffuse display surface of the touch-sensitive display apparatus;

receiving reflected non-visible light, wherein the reflected light is at least a portion of emitted light that is reflected from the first finger of the gloveless user; and

determining that the gesture is occurring in a hoverspace above the touch-sensitive display apparatus based at least in part upon the reflected non-visible light.

9. The method of claim 1, wherein the gesture data indicates that the first finger and the second finger of the gloveless user are simultaneously performing gestures with respect to the touch-sensitive display apparatus.

10. The method of claim 1, wherein the gesture data indicates that the gloveless user and a second user are simultaneously performing respective gestures with respect to the touch-sensitive display apparatus, wherein the gesture data comprises data that disambiguates between the gloveless user and the second user, and wherein performing the computing process is based at least in part upon disambiguation between the gloveless user and the second user.

11. The method of claim 1, wherein determining location of the gesture with respect to the touch-sensitive display apparatus comprises receiving image data from a camera that is external from the touch-sensitive display apparatus, wherein the image data comprises an image of at least a portion of the gloveless user with respect to the touch-sensitive display apparatus.

12. A system that comprises the following computer-executable components:

a receiver component that receives gesture data from an electromyography sensor unit that is in contact with skin of a gloveless user, wherein the electromyography sensor unit is in a form of one of a wristband or a forearm band, wherein the gesture data is indicative of a bodily gesture of the gloveless user, wherein the bodily gesture comprises positioning of a first finger and a second finger relative to one another, and wherein the gesture data comprises data that identifies the first finger and data that identifies the second finger;

a location determiner component that determines location of the bodily gesture with respect to a touch-sensitive display apparatus; and

a display component that causes the touch-sensitive display apparatus to display an image based at least in part upon the received gesture data and the determined location of the bodily gesture with respect to the touch-sensitive display apparatus, wherein the display component disambiguates between the first finger and second finger based at least in part upon the data that identifies the first finger and the data that identifies the second finger and causes the touch-sensitive display apparatus to display the image based at least in part upon disambiguating between the first finger and the second finger.

13. The system of claim 12, wherein the touch-sensitive display apparatus comprises the receiver component, the location determiner component, and the display component.

14. The system of claim 12, wherein the location determiner component determines that a first bodily gesture occurs in a hover space of the touch-sensitive display apparatus and that a second bodily gesture occurs in contact with the touch-sensitive display apparatus, and wherein the display component causes the touch-sensitive display apparatus to display the image based at least in part upon the first bodily gesture occurring in the hover space and the second bodily gesture occurring in contact with the touch-sensitive display apparatus.

15. The system of claim 12, wherein the location determiner component is configured to determine the location of the bodily gesture in a hoverspace above the touch-sensitive display apparatus.

16. The system of claim 12, wherein the gesture data indicates that the gloveless user and a second user are simultaneously performing respective gestures with respect to the touch-sensitive display apparatus, wherein the gesture data comprises data that disambiguates between the gloveless user and the second user, and wherein the display component causes the touch-sensitive display apparatus to display the image based at least in part upon the data that disambiguates between the gloveless user and the second user.

17. The system of claim 12, wherein the location determiner component is configured to receive image data from a camera external from the touch-sensitive display apparatus, wherein the image data comprises an image of at least a portion of the gloveless user with respect to the touch-sensitive display apparatus.

18. The system of claim 12, wherein the gesture data further comprises data that is indicative of an identity of the gloveless user, and wherein the display component causes the touch-sensitive display apparatus to display the image based at least in part upon the data that is indicative of the identity of the gloveless user.

19. The system of claim 12, further comprising a calibration component configured to calibrate the touch-sensitive display apparatus based at least in part upon the received gesture data.

20. A computer readable data storage device comprising instructions that, when executed by a processor, cause the processor to perform acts comprising:

receiving gesture data from an electromyography sensor unit that is in contact with skin of a gloveless user, wherein the electromyography sensor unit is in a form of one of a wristband or a forearm band, wherein the gesture data is indicative of a gesture of the gloveless user, the gesture being positioning of a first digit of the gloveless user relative to a second digit of the gloveless user, and wherein the gesture data comprises data that identifies the first digit and data that identifies the second digit;

determining a location of the gesture with respect to a touch-sensitive display apparatus; and

graphically displaying image data to the gloveless user based at least in part upon the received gesture data and the determined location of the gesture with respect to the touch-sensitive display apparatus such that graphically displaying the image data to the gloveless user is based at least in part upon the data that identifies the first digit and the data that identifies the second digit.