Method and apparatus for a gesture controlled interface for wearable devices

• US 9,720,515 B2
• Filed: 07/06/2016
• Issued: 08/01/2017
• Est. Priority Date: 01/02/2015
• Status: Active Grant

• Current Assignee: Wearable Devices Ltd.
• Original Assignee: Wearable Devices Ltd.
• Inventors: Wagner, Guy, Langer, Leeor, Dahan, Asher
• Primary Examiner(s): OSORIO, RICARDO
• Application Number:
  US15/202,750
• Publication Number:
  US 20160313801A1
• Time in Patent Office: 391 Days
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G09B 21/02 Devices for Braille writing typewriters for Braille B41J3/32
• CPC Class Codes: G09G 2370/16 Use of wireless transmission of display information
• CPC Class Codes: G09G 5/003 Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto specific for a CRT G09G1/165; for a flat panel G09G3/2092

Abstract

A gesture-controlled interface apparatus includes one or a plurality of bio-potential sensors and a processor. The one or a plurality of bio-potential sensors are wearable on a body of a user, for detecting one or a plurality of bio-electrical signals from the body of the user, wherein the one or a plurality of bio-potential sensors include at least one surface nerve conduction (SNC) sensor for detecting at least one surface nerve conduction signal. The processor is configured to compare the detected at least one surface nerve conduction signal with data of a plurality of reference signals corresponding to a plurality of known gestures, each of the reference signals distinctly associated with one of the known gestures, to identify a known gesture from the plurality of known gestures that corresponds to said at least one surface nerve conduction signal, and to communicate the identified known gesture to a computerized device.

Claims

1. A gesture-controlled interface apparatus comprising:

a strap configured to be worn on a wrist of a user, comprising one or a plurality of bio-potential sensors, for detecting one or a plurality of bio-electrical signals from the wrist of the user, wherein the one or a plurality of bio-potential sensors include at least one surface nerve conduction (SNC) sensor for detecting at least one surface nerve conduction signal from nerves in the wrist,

wherein each of the detected one or a plurality of bio-electrical signals has an amplitude proportional to pressure applied to squeezed fingers;

a processor configured to translate the one or a plurality of bio-electrical signals to a pressure control signal proportional to pressure applied to the squeezed fingers, and to apply the pressure control signal to a computerized device.

2. The apparatus according to claim 1, wherein the one or a plurality of bio-potential sensors are selected from the group consisting of surface electromyography (sEMG) sensors, capacitive electromyography (cEMG) sensors, and skin conductivity sensors.

3. The apparatus according to claim 1, wherein the squeezed fingers comprises at least two fingers pressed together.

4. The apparatus according to claim 1, wherein the each of the detected one or a plurality of bio-electrical signals has a frequency proportional to pressure applied to squeezed fingers.

5. The apparatus according to claim 1, wherein the processor is configured to apply the pressure control signal to the computerized device so as to smoothly change a parameter of the computerized device.

6. The apparatus according to claim 1, wherein the processor is configured to apply proportional control algorithms to the one or a plurality of bio-electrical signal by using a transform in a proportional control data pipeline to introduce domain knowledge as a pre-processing step.

7. The apparatus according to claim 1, wherein the processor is configured to compare the detected at least one surface nerve conduction signal with data of a plurality of reference signals corresponding to a plurality of known gestures, each of the reference signals distinctly associated with one of the known gestures, so as to identify a known gesture of said plurality of known gestures.

8. The apparatus according to claim 7 further comprising at least one motion sensor configured to detect movement of the body, and wherein the processor is configured to identify the known gesture using the detected movement.

9. The apparatus according to claim 7, further comprising haptic actuators configured to activate haptic feedback on the body of the user when the identified known gesture is registered in the computerized device.

10. The apparatus according to claim 7, wherein the processor is configured to train the data for the body of the user by using the one or a plurality of bio-electrical signals to associate the at least one surface nerve conduction signal with each of the plurality of known gestures.

11. The apparatus according to claim 7, wherein the processor is configured to identify the known gesture by de-noising the detected at least one surface nerve conduction (SNC) signal, detecting an event in the at least one SNC signal, applying segmentation for determining one or more frames of the detected event, extracting statistical features within the one or more frames, and applying a classification algorithm based on the data to the extracted statistical features so as to determine the known gesture.

12. The apparatus according to claim 7, wherein the processor is configured to identify the known gesture by applying the one or a plurality of detected bio-electrical signals to a gesture recognition pipeline.

13. A method for communication between a gesture-controlled interface apparatus and a computerized device, the method comprising:

detecting one or a plurality of bio-electrical signals from one or a plurality of bio-potential sensors placed on a wrist of a user, wherein the one or a plurality of bio-potential sensors include at least one surface nerve conduction (SNC) sensor for detecting at least one surface nerve conduction signal from nerves in a wrist of the user, and each of the detected one or a plurality of bio-electrical signals has an amplitude proportional to pressure applied to squeezed fingers;

using a processor, translating the one or a plurality of bio-electrical signals to a pressure control signal proportional to pressure applied to the squeezed fingers; and

applying the pressure control signal to a computerized device.

14. The method according to claim 13, further comprising estimating the pressure applied to the squeezed fingers by applying the one or a plurality of detected bio-electrical signals to a proportional control pipeline including a convolutional neural network (CNN) and a long short term memory (LSTM) neural network.

15. The method according to claim 14, further comprising training the LSTM neural network by using an auxiliary signal.

16. The method according to claim 13, wherein the squeezed fingers comprises at least two fingers pressed together.

17. The method according to claim 13, wherein each of the detected one or a plurality of bio-electrical signals has a frequency proportional to pressure applied to squeezed fingers.

18. The method according to claim 13, further comprising applying the pressure control signal to the computerized device so as to smoothly change a parameter of the computerized device.

19. The method according to claim 13, further comprising applying proportional control algorithms to the one or a plurality of bio-electrical signal by using a transform in a proportional control data pipeline to introduce domain knowledge as a pre-processing step.

20. The method according to claim 13, further comprising comparing the detected at least one surface nerve conduction signal with data of a plurality of reference signals corresponding to a plurality of known gestures, each of the reference signals distinctly associated with one of the known gestures, so as to identify a known gesture of said plurality of known gestures.

21. The method according to claim 20, wherein identifying the known gesture comprises de-noising the detected at least one surface nerve conduction (SNC) signal, detecting an event in the at least one SNC signal, applying segmentation for determining one or more frames of the detected event, extracting statistical features within the one or more frames, and applying a classification algorithm based on the data to the extracted statistical features so as to determine the known gesture.

22. The method according to claim 20, wherein identifying the known gesture comprises applying the one or a plurality of detected bio-electrical signals to a gesture recognition pipeline.