Sensing Mechanical Energy to Appropriate the Body for Data Input

• US 20110133934A1
• Filed: 12/04/2009
• Published: 06/09/2011
• Est. Priority Date: 12/04/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Morris, Dan, Tan, Desney S., Harrison, Christopher
• Granted Patent: US 8,421,634 B2
• Time in Patent Office: Days
• US Class Current: 340/573.1
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/1673 Arrangements for projecting a virtual keyboard
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

Described is using the human body as an input mechanism to a computing device. A sensor set is coupled to part of a human body. The sensor set detects mechanical (e.g., bio-acoustic) energy transmitted through the body as a result of an action/performed by the body, such as a user finger tap or flick. The sensor output data (e.g., signals) are processed to determine what action was taken. For example, the gesture may be a finger tap, and the output data may indicate which finger was tapped, what surface the finger was tapped on, or where on the body the finger was tapped.

Claims

1. A system, comprising a sensor set coupled to part of a human body, the sensor set configured to detect mechanical energy transmitted through the body resulting from an action performed by the body and provide output data corresponding to the action, and a computing device that uses information corresponding to the output data as data input.

2. The system of claim 1 wherein the sensor set comprises a plurality of sensors, and wherein each sensor is associated with a resonant frequency.

3. The system of claim 1 wherein the sensor set comprises a plurality of tuned vibratory sensors

4. The system of claim 1 wherein the sensor set is incorporated into an armband coupled to an arm.

5. The system of claim 1 wherein the sensor set detects mechanical energy corresponding to taps of at least one finger on the body, or on a surface other than the body.

6. The system of claim 1 wherein the sensor set outputs signals to a data capture mechanism, and further comprising, a classifier that determines from features of the signals the action performed by the body.

7. The system of claim 6 wherein the classifier comprises a support vector machine.

8. The system of claim 6 wherein the classifier determines:

a location of a tap on the body, a magnitude of a tap on the body, a type of a tap on the body, or which finger corresponds to the action performed by the body, or any combination of a location of a tap on the body, a magnitude of a tap on the body, a type of tap of a tap on the body, or which finger corresponds to the action performed by the body.

9. The system of claim 6 wherein the classifier classifies or differentiates:

taps on the body detectable by the sensor made by a finger or other object, gestures created by at least one part of the body to which the sensor is coupled, or active energy emitting objects that are able to be sensed as a result of coming into contact with the body, or any combination taps on the body detectable by the sensor made by a finger or other object, gestures created by at least one part of the body to which the sensor is coupled, or active energy emitting objects that are able to be sensed as a result of coming into contact with the body.

10. The system of claim 1 further comprising at least one other sensor that provides information to the computing device.

11. The system of claim 1 wherein the sensor set outputs signals to a data capture mechanism, and further comprising an embedded processor that processes the signals into the output data.

12. The system of claim 11 further comprising at least one other sensor that provides information to the embedded processor.

13. The system of claim 1 further comprising a projector that projects an image, and wherein the sensor detects mechanical energy resulting from an action performed by the body that interacts with the image.

14. The system of claim 13 wherein the projector is physically coupled to the body, or wherein the projector projects the image onto a surface of the body, or wherein both the projector is physically coupled to the body and the projector projects the image onto a surface of the body.

15. The system of claim 1 wherein the sensor set further detects a vibration of an object that is not part of the body.

16. In a computing environment, a method performed on at least one processor, comprising, processing signals received from a mechanical energy sensor set coupled to a human user into features representative of an action taken by the user, using the features to classify the action into output data, and providing the output data for input into a computing device.

17. The method of claim 16 wherein processing the signals comprises sampling the signals to obtain captured data, and segmenting the captured data into independent input instances.

18. The method of claim 17 further comprising, determining whether an independent input instance corresponds to a gesture.

19. One or more computer-readable media having computer-executable instructions, which when executed perform steps, comprising:

processing a set of signals received from a mechanical energy sensor set to determine whether the set of signals corresponds to a gesture, and if not, discarding the set of signals, or if so;

extracting features from the set of signals; and

providing the features to a classifier that determines which gesture was performed from among a set of possible gestures; and

providing output data indicating of the gesture that was performed.

20. The one or more computer-readable media of claim 19 having further computer-executable instructions comprising, obtaining the set of signals by segmenting a plurality of sampled signals.