Sensing mechanical energy to appropriate the body for data input

• US 8,421,634 B2
• Filed: 12/04/2009
• Issued: 04/16/2013
• Est. Priority Date: 12/04/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Tan, Desney S., Morris, Dan, Harrison, Christopher
• Primary Examiner(s): HUNNINGS, TRAVIS R
• Application Number:
  US12/630,892
• Publication Number:
  US 20110133934A1
• Time in Patent Office: 1,229 Days
• Field of Search: 340/573.1
• US Class Current: 340/573.1
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/1673 Arrangements for projecting a virtual keyboard
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

Described is using the human body as an input mechanism to a computing device. A sensor set is coupled to part of a human body. The sensor set detects mechanical (e.g., bio-acoustic) energy transmitted through the body as a result of an action/performed by the body, such as a user finger tap or flick. The sensor output data (e.g., signals) are processed to determine what action was taken. For example, the gesture may be a finger tap, and the output data may indicate which finger was tapped, what surface the finger was tapped on, or where on the body the finger was tapped.

Claims

1. A system, comprising:

a sensor set coupled to part of a human body, the sensor set configured to detect mechanical energy transmitted through the body resulting from an action performed by the body and provide output data corresponding to the action; and

a computing device configured to use information corresponding to the output data as data input.

2. The system of claim 1 wherein the sensor set comprises a plurality of sensors, and wherein each sensor is associated with a resonant frequency.

3. The system of claim 1 wherein the sensor set comprises a plurality of tuned vibratory sensors.

4. The system of claim 1 wherein the sensor set is incorporated into an armband coupled to an arm.

5. The system of claim 1 wherein the sensor set is configured to detect mechanical energy corresponding to taps of at least one finger on the body, or on a surface other than the body.

6. The system of claim 1 wherein the sensor set is configured to output signals to a data capture mechanism, and further comprising, a classifier configured to determine from features of the signals the action performed by the body.

7. The system of claim 6 wherein the classifier comprises a support vector machine.

8. The system of claim 6 wherein the classifier is configured to determine at least one of a location of a tap on the body, a magnitude of a tap on the body, a type of a tap on the body, or which finger corresponds to the action performed by the body.

9. The system of claim 6 wherein the classifier is configured to classify or differentiate at least one of taps on the body detectable by the sensor made by a finger or other object, gestures created by at least one part of the body to which the sensor is coupled, or active energy emitting objects that are able to be sensed as a result of coming into contact with the body.

10. The system of claim 1 further comprising at least one other sensor that provides information to the computing device.

11. The system of claim 1 wherein the sensor set outputs signals to a data capture mechanism, and further comprising an embedded processor that processes the signals into the output data.

12. The system of claim 11 further comprising at least one other sensor configured to provide information to the embedded processor.

13. The system of claim 1 further comprising a projector configured to project an image, and wherein the sensor detects mechanical energy resulting from an action performed by the body that interacts with the image.

14. The system of claim 13 wherein the projector is physically coupled to the body, or wherein the projector projects the image onto a surface of the body, or wherein both the projector is physically coupled to the body and the projector projects the image onto a surface of the body.

15. The system of claim 1 wherein the sensor set further detects a vibration of an object that is not part of the body.

16. In a computing environment, a method comprising:

processing, by at least one processor, signals received from a mechanical energy sensor set coupled to a human user into features representative of an action taken by the user;

using the features to classify the action into output data; and

providing the output data for input into a computing device.

17. The method of claim 16 wherein processing the signals comprises:

sampling the signals to obtain captured data; and

segmenting the captured data into independent input instances.

18. The method of claim 17 further comprising:

determining whether an independent input instance corresponds to a gesture.

19. One or more computer-readable media storage devices having computer-executable instructions, which in response to execution by a computer, cause the computer to perform steps comprising:

processing a set of signals received from a mechanical energy sensor set to determine whether the set of signals corresponds to a gesture;

responsive to a determination that the set of signals does not correspond to the gesture, discarding the set of signals;

responsive to a determination that the set of signals does correspond to the gesture extracting features from the set of signals, providing the features to a classifier that determines which gesture was performed from among a set of possible gestures, and providing output data indicating the gesture that was performed.

20. The one or more computer-readable media storage devices of claim 19 having further computer-executable instructions comprising:

obtaining the set of signals by segmenting a plurality of sampled signals.