SYSTEMS, DEVICES, AND METHODS FOR WEARABLE ELECTRONIC DEVICES SUCH AS STATE MACHINES

• US 20150277575A1
• Filed: 03/26/2015
• Published: 10/01/2015
• Est. Priority Date: 03/27/2014
• Status: Active Grant

• Current Assignee: META Platforms Technologies LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Original Assignee: Thalmic Labs Inc.
• Inventors: Ataee, Pedram, Lake, Stephen
• Granted Patent: US 10,199,008 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G09G 5/006 Details of the interface to the display terminal specific for a display terminal using a CRT G09G1/167; using a flat panel G09G3/2096; circuits for interfacing with colour displays G09G5/04

Abstract

Systems, devices, and methods that implement state machine models in wearable electronic devices are described. A wearable electronic device stores processor-executable gesture identification instructions that, when executed by an on-board processor, enable the wearable electronic device to identify one or more gesture(s) performed by a user. The wearable electronic device also stores processor-executable state determination instructions that, when executed by the processor, cause the wearable electronic device to enter into and transition between various operational states depending on signals detected by on-board sensors. The state machine models described herein enable the wearable electronic devices to identify and automatically recover from operational errors, malfunctions, or crashes with minimal intervention from the user.

Claims

1. A wearable electronic device comprising:

at least one sensor responsive to at least one input effected by a user of the wearable electronic device, wherein in response to the at least one input the at least one sensor provides sensor signals;

a processor communicatively coupled to the at least one sensor; and

a non-transitory processor-readable storage medium communicatively coupled to the processor, wherein the non-transitory processor-readable storage medium stores processor-executable sensor signal processing instructions and processor-executable state determination instructions, and wherein when executed by the processor, the state determination instructions cause the wearable electronic device to;

in response to a determination that the sensor signal processing instructions are not calibrated, enter a standby state wherein in the standby state the wearable electronic device recognizes a first indication from the user;

in response to a recognition of the first indication from the user while the wearable electronic device is in the standby state, enter a calibration state wherein in the calibration state the wearable electronic device recognizes a second indication from the user and calibrates the sensor signal processing instructions in response to the second indication from the user; and

in response to calibrating the sensor signal processing instructions while the wearable electronic device is in the calibration state, enter an active state wherein in the active state the wearable electronic device executes the calibrated sensor signal processing instructions, and wherein when executed by the processor the calibrated sensor signal processing instructions cause the wearable electronic device to process at least one input effected by the user based at least in part on the calibration of the sensor signal processing instructions from the calibration state.

2. The wearable electronic device of claim 1 wherein the at least one sensor is responsive to at least one gesture performed by the user and provides sensor signals in response to the at least one gesture, and wherein the processor-executable sensor signal processing instructions include processor-executable gesture identification instructions that, when executed by the processor while the wearable electronic device is in the active state, cause the wearable electronic device to identify at least one gesture performed by the user based at least in part on the calibration of the gesture identification instructions from the calibration state.

3. The wearable electronic device of claim 2 wherein the first indication from the user includes a rest gesture performed by the user, the rest gesture indicative that the user is ready to calibrate the gesture identification instructions, and wherein the non-transitory processor-readable storage medium further stores processor-executable instructions that, when executed by the processor, cause the wearable electronic device to recognize the rest gesture as the first indication from the user in the standby state.

4. The wearable electronic device of claim 2 wherein the first indication from the user includes a recognition by the wearable electronic device that the user has donned the wearable electronic device, and wherein the non-transitory processor-readable storage medium further stores processor-executable instructions that, when executed by the processor, cause the wearable electronic device to recognize when the user dons the wearable electronic device as the first indication from the user in the standby state.

5. The wearable electronic device of claim 2 wherein the non-transitory processor-readable storage medium further stores processor-executable instructions that, when executed by the processor, cause the wearable electronic device to:

recognize at least one reference gesture performed by the user as the second indication from the user in the calibration state; and

calibrate the gesture identification instructions based on the at least one reference gesture.

6. The wearable electronic device of claim 2 wherein the at least one sensor includes at least one muscle activity sensor selected from the group consisting of:

an electromyography (“

EMG”

) sensor and a mechanomyography (“

MMG”

) sensor.

7. The wearable electronic device of claim 1 wherein the at least one sensor includes at least one inertial sensor selected from the group consisting of:

an inertial measurement unit, an accelerometer, and a gyroscope.

8. The wearable electronic device of claim 1, further comprising:

at least one light-emitting diode (“

LED”

) communicatively coupled to the processor, wherein the non-transitory processor-readable storage medium further stores processor-executable instructions that, when executed by the processor, cause the wearable electronic device to activate at least one of;

a first color of the at least one LED in response to entering the standby state;

a second color of the at least one LED in response to entering the calibration state; and

/or

a third color of the at least one LED in response to entering the active state.

9. The wearable electronic device of claim 1, further comprising:

a haptic feedback device communicatively coupled to the processor, wherein the non-transitory processor-readable storage medium further stores processor-executable instructions that, when executed by the processor, cause the wearable electronic device to activate the haptic feedback module in response to at least one of;

entering the standby state;

entering the calibration state; and

/or

entering the active state.

10. A method of operating a wearable electronic device as a state machine, the wearable electronic device including at least one sensor responsive to at least one input effected by a user of the wearable electronic device, wherein in response to the at least one input the at least one sensor provides sensor signals, a processor communicatively coupled to the at least one sensor, and a non-transitory processor-readable storage medium communicatively coupled to the processor, wherein the non-transitory processor-readable storage medium stores at least processor-executable sensor signal processing instructions, the method comprising:

entering the wearable electronic device into a standby state in response to a determination that the sensor signal processing instructions are not calibrated;

while the wearable electronic device is in the standby state;

detecting a first indication from the user by the wearable electronic device, the first indication from the user indicative that the user is ready to calibrate the sensor signal processing instructions; and

recognizing the first indication from the user by the processor;

entering the wearable electronic device into a calibration state in response to recognizing the first indication from the user while the wearable electronic device is in the standby state;

while the wearable electronic device is in the calibration state;

detecting second indication from the user by the wearable electronic device;

recognizing the second indication from the user by the processor; and

calibrating the sensor signal processing instructions by the processor in response to recognizing the second indication from the user; and

entering the wearable electronic device into an active state in response to calibrating the sensor signal processing instructions while the wearable electronic device is in the calibration state; and

while the wearable electronic device is in the active state;

detecting at least one input by the at least one sensor; and

executing the calibrated sensor signal processing instructions by the processor, wherein executing the calibrated sensor signal processing instructions causes the processor to process the at least one input based on the calibration of the sensor signal processing instructions from the calibration state.

11. The method of claim 10 wherein the at least one sensor is responsive to at least one gesture performed by the user and provides sensor signals in response to the at least one gesture, and wherein the processor-executable sensor signal processing instructions include processor-executable gesture identification instructions, and wherein:

entering the wearable electronic device into a standby state in response to a determination that the sensor signal processing instructions are not calibrated includes entering the wearable electronic device into a standby state in response to a determination that the gesture identification instructions are not calibrated;

while the wearable electronic device is in the standby state, detecting a first indication from the user by the wearable electronic device, the first indication from the user indicative that the user is ready to calibrate the sensor signal processing instructions includes detecting a first indication from the user by the wearable electronic device, the first indication from the user indicative that the user is ready to calibrate the gesture identification instructions;

while the wearable electronic device is in the calibration state, calibrating the sensor signal processing gesture identification instructions by the processor in response to recognizing the second indication from the user includes calibrating the gesture identification instructions by the processor in response to recognizing the second indication from the user;

entering the wearable electronic device into an active state in response to calibrating the sensor signal processing instructions while the wearable electronic device is in the calibration state includes entering the wearable electronic device into an active state in response to calibrating the gesture identification instructions while the wearable electronic device is in the calibration state;

while the wearable electronic device is in the active state, detecting at least one input by the at least one sensor includes detecting at least one gesture by the at least one sensor; and

while the wearable electronic device is in the active state, executing the calibrated sensor signal processing gesture identification instructions by the processor includes executing the calibrated gesture identification instructions, wherein executing the calibrated gesture identification instructions causes the processor to identify the at least one gesture based on the calibration of the gesture identification instructions from the calibration state.

12. The method of claim 11 wherein, while the wearable electronic device is in the standby state:

detecting a first indication from the user by the wearable electronic device includes detecting, by the at least one sensor, a rest gesture performed by the user; and

recognizing the first indication from the user by the processor includes recognizing the rest gesture by the processor.

13. The method of claim 11 wherein, while the wearable electronic device is in the standby state:

detecting a first indication from the user by the wearable electronic device includes detecting, by the wearable electronic device, when the user dons the wearable electronic device; and

recognizing the first indication from the user by the processor includes recognizing, by the processor, when the user dons the wearable electronic device.

14. The method of claim 11 wherein, while the wearable electronic device is in the calibration state:

detecting a second indication from the user by the wearable electronic device includes detecting, by the at least one sensor, a reference gesture performed by the user;

recognizing the second indication from the user by the processor includes recognizing the reference gesture by the processor; and

calibrating the gesture identification instructions by the processor in response to recognizing the second indication from the user includes calibrating the gesture identification instructions by the processor based on the reference gesture.

15. The method of claim 11 wherein the at least one sensor includes at least one muscle activity sensor selected from the group consisting of:

an electromyography (“

EMG”

) sensor and a mechanomyography (“

MMG”

) sensor; and

wherein;

while the wearable electronic device is in the active state, detecting at least one gesture by the at least one sensor includes detecting muscle activity by the at least one muscle activity sensor when the user performs the at least one gesture.

16. The method of claim 11 wherein the at least one sensor includes at least one inertial sensor selected from the group consisting of:

an inertial measurement unit, an accelerometer, and a gyroscope; and

wherein;

while the wearable electronic device is in the active state, detecting at least one gesture by the at least one sensor includes detecting motion of the wearable electronic device by the at least one inertial sensor when the user performs the at least one gesture.

17. The method of claim 10 wherein the wearable electronic device further includes at least one light-emitting diode (“

LED”

) communicatively coupled to the processor, and wherein the method further comprises at least one of;

activating a first color of the at least one LED by the processor in response to entering the standby state;

activating a second color of the at least one LED by the processor in response to entering the calibration state; and

/or

activating a third color of the at least one LED by the processor in response to entering the active state.

18. The method of claim 10 wherein the wearable electronic device further includes a haptic feedback device communicatively coupled to the processor, and wherein the method further comprises at least one of:

activating the haptic feedback device by the processor in response to entering the standby state;

activating the haptic feedback device by the processor in response to entering the calibration state; and

/or

activating the haptic feedback device by the processor in response to entering the active state.

19. The method of claim 10 wherein the non-transitory processor-readable storage medium further stores processor-executable state determination instructions that, when executed by the processor, cause the wearable electronic device to enter into and transition between the standby state, the calibration state, and the active state.