METHODS AND DEVICES FOR COMBINING MUSCLE ACTIVITY SENSOR SIGNALS AND INERTIAL SENSOR SIGNALS FOR GESTURE-BASED CONTROL

• US 20140240103A1
• Filed: 02/21/2014
• Published: 08/28/2014
• Est. Priority Date: 02/22/2013
• Status: Abandoned Application

• Current Assignee: META Platforms Technologies LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Original Assignee: Thalmic Labs Inc.
• Inventors: Lake, Stephen, Bailey, Matthew, Grant, Aaron
• Application Number:
  US14/186,889
• Publication Number:
  US 20140240103A1
• Time in Patent Office: Days
• US Class Current: 340/12.5
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G08C 17/02 using a radio link
• CPC Class Codes: G08C 2201/32 Remote control based on movements, attitude of remote control device

Abstract

There is disclosed a wearable electronic device for use with controllable connected devices. The wearable electronic device includes a band worn on, for example, the forearm of a user, and the band carries at least one muscle activity sensor, at least one inertial sensor, and a processor communicatively coupled to the sensors. The on-board processor is operable to identify, a plurality of gestures made by a user, based on muscle activity detected by the muscle activity sensor(s) and motion detected by the inertial sensor(s). In response to identifying a gesture, the wearable electronic device wirelessly transmits one or more signal(s) in order to interact with a controllable connected device.

Claims

1. A wearable electronic device comprising:

a band that, in use, is worn by a user;

at least one muscle activity sensor carried by the band, the at least one muscle activity sensor to, in use, detect muscle activity in response to the user performing a physical gesture and provide at least one signal in response to the detected muscle activity;

at least one inertial sensor carried by the band, the at least one inertial sensor to, in use, detect motion in response to the user performing a physical gesture and provide at least one signal in response to the detected motion; and

a processor carried by the band, the processor communicatively coupled to the at least one muscle activity sensor and to the at least one inertial sensor and the processor to, in use, identify a physical gesture performed by the user based on either or both of;

at least one signal provided by the at least one muscle activity sensor; and

at least one signal provided by the at least one inertial sensor.

2. The wearable electronic device of claim 1, further comprising a non-transitory computer-readable storage medium carried by the band and communicatively coupled to the processor, wherein the non-transitory computer-readable storage medium stores processor executable instructions that, when executed by the processor, cause the processor to identify a physical gesture performed by the user based on either or both of:

at least one signal provided by the at least one muscle activity sensor; and

at least one signal provided by the at least one inertial sensor.

3. The wearable electronic device of claim 2 wherein the non-transitory computer-readable storage medium stores processor executable instructions that, when executed by the processor, cause the processor to perform a machine intelligence method to identify a physical gesture performed by the user based on either or both of:

at least one signal provided by the at least one muscle activity sensor; and

at least one signal provided by the at least one inertial sensor.

4. The wearable electronic device of claim 3 wherein the processor executable instructions stored in the non-transitory computer-readable storage medium include processor executable instructions that, when executed by the processor, cause the processor to implement at least one of:

a classifier, a Hidden Markov Model, and/or a Long Short Term Neural Net.

5. The wearable electronic device of claim 1, further comprising a wireless transmitter carried by the band, the wireless transmitter communicatively coupled to the processor to, in use, wirelessly transmit at least one signal in response to the processor identifying a physical gesture performed by the user.

6. The wearable electronic device of claim 1 wherein the at least one muscle activity sensor includes at least one muscle activity sensor selected from the group consisting of:

an electromyography (EMG) sensor and a mechanomyography (MMG) sensor.

7. The wearable electronic device of claim 1 wherein the at least one muscle activity sensor includes a plurality of electromyography (EMG) sensors.

8. The wearable electronic device of claim 1, further comprising a haptic feedback module carried by the band, the haptic feedback module to, in use, provide haptic feedback to the user.

9. The wearable electronic device of claim 8 wherein the haptic feedback module includes a vibratory motor.

10. The wearable electronic device of claim 1 wherein the band is expandable.

11. The wearable electronic device of claim 1, further comprising at least one additional component carried by the band, the at least one additional component selected from the group consisting of:

an amplification circuit communicatively coupled to the at least one muscle activity sensor, a filtering circuit communicatively coupled to the at least one muscle activity sensor, an analog-to-digital conversion circuit communicatively coupled to the at least one muscle activity sensor, and a battery electrically coupled to the processor.

12. The wearable electronic device of claim 1 wherein the at least one inertial sensor includes at least one inertial sensor selected from the group consisting of:

an accelerometer, a gyroscope, and an inertial measurement unit (IMU).

13. A method of operating a wearable electronic device, wherein the wearable electronic device includes a band worn by a user, at least one muscle activity sensor carried by the band, at least one inertial sensor carried by the band, and a processor carried by the band, the processor communicatively coupled to the at least one muscle activity sensor and to the at least one inertial sensor, the method comprising:

detecting, by at least one muscle activity sensor carried by the band of the wearable electronic device, muscle activity in response to the user performing a physical gesture;

providing at least one signal from at least one muscle activity sensor to the processor in response to the detected muscle activity;

detecting, by at least one inertial sensor carried by the band of the wearable electronic device, motion in response to the user performing a physical gesture;

providing at least one signal from at least one inertial sensor to the processor in response to the detected motion;

processing, by the processor carried by the band of the wearable electronic device, the at least one signal provided by at least one muscle activity sensor and the at least one signal provided by at least one inertial sensor; and

identifying, by the processor carried by the band of the wearable electronic device, a physical gesture performed by the user based on at least one of;

the at least one signal provided by at least one muscle activity sensor; and

the at least one signal provided by at least one inertial sensor.

14. The method of claim 13 wherein identifying, by the processor carried by the band of the wearable electronic device, a physical gesture performed by the user includes identifying, by the processor carried by the band of the wearable electronic device, a physical gesture performed by the user based on both the at least one signal provided by at least one muscle activity sensor and the at least one signal provided by at least one inertial sensor.

15. The method of claim 13 wherein identifying, by the processor carried by the band of the wearable electronic device, a physical gesture performed by the user includes:

identifying, by the processor carried by the band of the wearable electronic device, a first physical gesture performed by the user based on the at least one signal provided by at least one muscle activity sensor; and

identifying, by the processor carried by the band of the wearable electronic device, a second physical gesture performed by the user based on the at least one signal provided by at least one inertial sensor.

16. The method of claim 13 wherein processing, by the processor carried by the band of the wearable electronic device, the at least one signal provided by at least one muscle activity sensor and the at least one signal provided by at least one inertial sensor includes utilizing one or more of a classifier, a Hidden Markov Model, a Long-Short Term Neural Net, or another machine intelligence method to process the at least one signal provided by at least one muscle activity sensor and the at least one signal provided by at least one inertial sensor.

17. The method of claim 13 wherein the wearable electronic device further includes a wireless transmitter carried by the band and communicatively coupled to the processor, the method further comprising:

wirelessly transmitting at least one signal in response to the processor identifying a physical gesture performed by the user.

18. The method of claim 13 wherein the wearable electronic device further includes an amplification circuit carried by the band and communicatively coupled to at least one muscle activity sensor, the method further comprising:

amplifying, by the amplification circuit, the at least one signal from at least one muscle activity sensor before providing the at least one signal from at least one muscle activity sensor to the processor.

19. The method of claim 13 wherein the wearable electronic device further includes a filtering circuit carried by the band and communicatively coupled to at least one muscle activity sensor, the method further comprising:

filtering, by the filtering circuit, the at least one signal from at least one muscle activity sensor before providing the at least one signal from at least one muscle activity sensor to the processor.

20. The method of claim 13 wherein the wearable electronic device further includes an analog-to-digital conversion circuit carried by the band and communicatively coupled to at least one muscle activity sensor, the method further comprising:

converting, by the analog-to-digital conversion circuit, the at least one signal from at least one muscle activity sensor from an analog signal to a digital signal before providing the at least one signal from at least one muscle activity sensor to the processor.

21. The method of claim 13 wherein the at least one muscle activity sensor includes a plurality of electromyography (EMG) sensors, and wherein:

detecting, by at least one muscle activity sensor carried by the band of the wearable electronic device, muscle activity in response to the user performing a physical gesture includes detecting, by at least one EMG sensor in the plurality of EMG sensors carried by the band of the wearable electronic device, muscle activity in response to the user performing a physical gesture; and

providing at least one signal from at least one muscle activity sensor to the processor in response to the detected muscle activity includes providing at least one signal from at least one EMG sensor in the plurality of EMG sensors to the processor in response to the detected muscle activity.

22. The method of claim 13 wherein the wearable electronic device further includes a haptic feedback module carried by the band, the haptic feedback module including a vibratory motor that is communicatively coupled to the processor, the method further comprising:

providing haptic feedback to the user by the vibratory motor in response to the processor identifying a physical gesture performed by the user.