WEARABLE MUSCLE INTERFACE SYSTEMS, DEVICES AND METHODS THAT INTERACT WITH CONTENT DISPLAYED ON AN ELECTRONIC DISPLAY

• US 20140198035A1
• Filed: 01/14/2014
• Published: 07/17/2014
• Est. Priority Date: 01/14/2013
• Status: Active Grant

• Current Assignee: META Platforms Technologies LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Original Assignee: Thalmic Labs Inc.
• Inventors: Bailey, Matthew, Grant, Aaron, Lake, Stephen
• Granted Patent: US 10,528,135 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: H04W 4/80 Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication

Abstract

Systems, devices and methods that enable a user to access and interact with content displayed on a portable electronic display in an inconspicuous, hands-free manner are described. There is disclosed a completely wearable system comprising a wearable muscle interface device and a wearable head-mounted display, as well as methods for using the wearable system to effect interactions between the user and content displayed on the wearable head-mounted display. The wearable muscle interface device includes muscle activity sensors worn on an arm of the user to detect muscle activity generated when the user performs a physical gesture. The wearable system is adapted to recognize a plurality of gestures made by the user and, in response to each recognized gesture, to effect one or more interaction(s) with content displayed on the wearable head-mounted display.

Claims

1. A wearable muscle interface device that in use interacts with content displayed on a wearable head-mounted display, the wearable muscle interface device comprising:

a plurality of muscle activity sensors to be worn on an arm of a user, the muscle activity sensors responsive to signals generated by muscles in the arm of the user; and

a transmitter communicatively coupled to the plurality of muscle activity sensors, wherein in use the transmitter transmits at least one signal from the wearable muscle interface device directly to a receiver on the wearable head-mounted display based on the signals detected by the muscle activity sensors;

wherein the at least one signal transmitted, in use, from the wearable muscle interface device directly to the receiver on the wearable head-mounted display effects at least one interaction with content displayed on the wearable head-mounted display.

2. The wearable muscle interface device of claim 1, further comprising:

a processor that in use interprets the signals detected by the muscle activity sensors as a gesture, wherein the processor is communicatively coupled in between the transmitter and the plurality of muscle activity sensors, and wherein the at least one signal that, in use, is transmitted from the wearable muscle interface device is based on the gesture interpreted by the processor of the wearable muscle interface device.

3. The wearable muscle interface device of claim 1 wherein the wearable head-mounted display includes a processor communicatively coupled to the receiver of the wearable head-mounted display, and wherein the at least one signal that, in use, is transmitted from the wearable muscle interface device to the wearable head-mounted display is interpreted as a gesture by the processor of the wearable head-mounted display.

4. The wearable muscle interface device of claim 1, further comprising a haptic feedback module that in use provides haptic feedback to the user, the haptic feedback module including a vibratory motor.

5. The wearable muscle interface device of claim 1 wherein the plurality of muscle activity sensors includes at least one muscle activity sensor selected from the group consisting of:

an electromyographic (EMG) sensor and a mechanomyographic (MMG) sensor.

6. The wearable muscle interface device of claim 1, further comprising:

at least one accelerometer that in use detects signals generated by motion of the arm of the user, the at least one accelerometer communicatively coupled to the transmitter, and wherein in use the at least one signal transmitted from the transmitter of the wearable muscle interface device directly to the receiver on the wearable head-mounted display is based on both the signals detected by the muscle activity sensors and the signals detected by the at least one accelerometer.

7. The wearable muscle interface device of claim 1 wherein the transmitter includes a wireless transmitter.

8. A wearable system that in use provides hands-free access to and control of a portable electronic display, the wearable system comprising:

i) a wearable muscle interface device comprising;

a plurality of muscle activity sensors to be worn on an arm of a user, the muscle activity sensors responsive to signals generated by muscles in the arm of the user; and

a transmitter communicatively coupled to the plurality of muscle activity sensors, wherein in use the transmitter transmits at least one signal from the wearable muscle interface device based on the signals detected by the muscle activity sensors;

and

ii) a wearable head-mounted display comprising;

at least one display screen to be worn on a head of the user, the at least one display screen arranged to be positioned in front of at least one eye of the user when worn on the head of the user;

a receiver communicatively coupled to the at least one display screen, wherein in use the receiver directly receives the at least one signal transmitted from the transmitter of the wearable muscle interface device; and

a processor communicatively coupled to the receiver and to the at least one display screen, wherein in use the at least one signal received directly from the transmitter of the wearable muscle interface device by the receiver of the wearable head-mounted display effects control of at least one function of the wearable head-mounted display.

9. The wearable system of claim 8 wherein the transmitter of the wearable muscle interface device includes a wireless transmitter and the receiver of the wearable head-mounted display includes a wireless receiver.

10. The wearable system of claim 8 wherein the wearable muscle interface device further comprises:

a processor that in use interprets the signals detected by the muscle activity sensors as a gesture, wherein the processor of the wearable muscle interface device is communicatively coupled in between the transmitter and the plurality of muscle activity sensors, and wherein the at least one signal that, in use, is transmitted from the wearable muscle interface device is based on the gesture interpreted by the processor of the wearable muscle interface device.

11. The wearable system of claim 8 wherein the plurality of muscle activity sensors includes at least one muscle activity sensor selected from the group consisting of:

an electromyographic (EMG) sensor and a mechanomyographic (MMG) sensor.

12. The wearable system of claim 8 wherein the wearable muscle interface device further comprises:

at least one accelerometer that in use detects signals generated by motion of the arm of the user, the at least one accelerometer communicatively coupled to the transmitter, and wherein in use the at least one signal transmitted by the transmitter of the wearable muscle interface device is based on both the signals detected by the muscle activity sensors and the signals detected by the at least one accelerometer.

13. A method of using a wearable system to achieve hands-free access to and control of a portable electronic display, wherein the wearable system includes a wearable muscle interface device and a wearable head-mounted display, the method comprising:

detecting muscle activity corresponding to a physical gesture performed by a user of the wearable system by at least one muscle activity sensor of the wearable muscle interface device;

transmitting at least one signal from the wearable muscle interface device by a transmitter of the wearable muscle interface device based at least in part on the muscle activity detected by at least one muscle activity sensor of the wearable muscle interface device;

receiving the at least one signal directly from the wearable muscle interface device by a receiver of the wearable head-mounted display;

processing the at least one signal by a processor of the wearable head-mounted display; and

effecting at least one interaction between the user and the wearable head-mounted display by the processor of the wearable head-mounted display based on the processing of the at least one signal by the processor of the wearable head-mounted display.

14. The method of claim 13, further comprising:

in response to detecting muscle activity corresponding to a physical gesture performed by a user of the wearable system by at least one muscle activity sensor of the wearable muscle interface device, processing the detected muscle activity by a processor of the wearable muscle interface device, and wherein transmitting at least one signal from the wearable muscle interface device by a transmitter of the wearable muscle interface device based at least in part on the muscle activity detected by at least one muscle activity sensor of the wearable muscle interface device includes transmitting at least one signal from the wearable muscle interface device by the transmitter of the wearable muscle interface device based at least in part on processing the detected muscle activity by the processor of the wearable muscle interface device.

15. The method of claim 13, further comprising:

detecting motion of the wearable muscle interface device corresponding to the physical gesture performed by the user of the wearable system by at least one accelerometer of the wearable muscle interface device, and wherein transmitting at least one signal from the wearable muscle interface device by a transmitter of the wearable muscle interface device based at least in part on the muscle activity detected by at least one muscle activity sensor of the wearable muscle interface device includes transmitting at least one signal from the wearable muscle interface device by the transmitter of the wearable muscle interface device based on both the muscle activity detected by at least one muscle activity sensor of the wearable muscle interface device and the motion detected by at least one accelerometer of the wearable muscle interface device.

16. The method of claim 13 wherein:

transmitting at least one signal from the wearable muscle interface device by a transmitter of the wearable muscle interface device includes wirelessly transmitting at least one signal from the wearable muscle interface device by a wireless transmitter of the wearable muscle interface device; and

receiving the at least one signal directly from the wearable muscle interface device by a receiver of the wearable head-mounted display includes wirelessly receiving the at least one signal directly from the wearable muscle interface device by a wireless receiver of the wearable head-mounted display.