Methods and systems for monitoring and influencing gesture-based behaviors

• US 10,474,244 B2
• Filed: 11/08/2018
• Issued: 11/12/2019
• Est. Priority Date: 12/16/2014
• Status: Active Grant

• Current Assignee: Somatix, Inc.
• Original Assignee: Somatix, Inc.
• Inventors: Ofir, Eran, Schatzberg, Uri
• Primary Examiner(s): Lee, Paul D
• Application Number:
  US16/184,851
• Publication Number:
  US 20190079593A1
• Time in Patent Office: 369 Days
• Field of Search: 702127
• CPC Class Codes: A24F 47/00 Smokers' requisites not otherwise provided for
• CPC Class Codes: A61B 2562/0219 Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
• CPC Class Codes: A61B 2562/0228 Microwave sensors
• CPC Class Codes: A61B 5/00 Measuring for diagnostic purposes radiation diagnosis A61B6/00; diagnosis by ultrasonic, sonic or infrasonic waves A61B8/00; Identification of persons
• CPC Class Codes: A61B 5/11 Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb for measuring pulse A61B5/02 ;A61B5/1038 takes precedence; motion detection to correct for motion artifacts in physiological signals A61B5/721
• CPC Class Codes: A61B 5/1116 Determining posture transitions
• CPC Class Codes: A61B 5/1123 Discriminating type of movement, e.g. walking or running A61B5/1116, A61B5/112 take precedence
• CPC Class Codes: A61B 5/681 Wristwatch-type devices
• CPC Class Codes: A61B 5/6824 Arm or wrist
• CPC Class Codes: G06F 3/00 Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

Methods and systems are provided herein for analyzing, monitoring, and/or influencing a user'"'"'s behavioral gesture in real-time. A gesture recognition method may be provided. The method may comprise: obtaining sensor data collected using at least one sensor located on a wearable device, wherein said wearable device is configured to be worn by a user; and analyzing the sensor data to determine a probability of the user performing a predefined gesture, wherein the probability is determined based in part on a magnitude of a motion vector in the sensor data, and without comparing the motion vector to one or more physical motion profiles.

Claims

1. A gesture recognition method comprising:

obtaining sensor data collected using at least two different types of sensors located on a wearable device, wherein said wearable device is configured to be worn by a user, the at least two different types of sensors comprise an accelerometer and a gyroscope, and wherein the sensor data comprises an acceleration vector obtained from the accelerometer and an angular velocity vector obtained from the gyroscope; and

analyzing the sensor data by evaluating a magnitude of the acceleration vector and a magnitude of the angular velocity vector and determining a correlation between the magnitudes of the acceleration vector and angular velocity vector within different temporal periods to determine a likelihood of the user performing a predefined gesture, wherein the likelihood is determined without comparing the acceleration vector and the angular velocity vector to one or more physical motion profiles.

2. The method of claim 1, wherein the predefined gesture is selected from a group of different gestures associated with different activities including the user smoking or about to smoke a cigarette, eating, drinking, or taking medication.

3. The method of claim 1, further comprising:

(1) calculating a pitch angle, roll angle, or yaw angle of the wearable device based on at least the acceleration vector or the angular velocity vector, and (2) determining the likelihood of the user performing the predefined gesture based on the calculated angle(s).

4. The method of claim 3, wherein the sensor data comprises a plurality of features encoded within said sensor data, and wherein the plurality of features are associated with various aspects of the predefined gesture.

5. The method of claim 4, wherein analyzing the sensor data further comprises calculating a multi-dimensional distribution function, wherein said multi-dimensional distribution function is a probability function of the plurality of features.

6. The method of claim 4, wherein the plurality of features comprise two or more of the following features:

(1) a time duration of a submotion during the gesture;

(2) the magnitude of the acceleration vector;

(3) the magnitude of the angular velocity vector;

(4) the roll angle; and

(5) the pitch angle.

7. The method of claim 4, further comprising:

inputting the plurality of features into a machine learning algorithm to determine the likelihood of the user performing the predefined gesture.

8. The method of claim 7, wherein the machine learning algorithm comprises supervised-learning, semi-supervised learning or unsupervised learning techniques.

9. The method of claim 1, wherein the at least two different types of sensors further comprise one or more of the following:

a magnetometer, a heart rate monitor, a global positioning system (GPS) receiver, a barometer, an external temperature sensor, a microphone, a skin temperature sensor, a capacitive sensor, a sensor configured to detect a galvanic skin response, an imaging device, a proximity sensor, an altitude sensor, an attitude sensor, a humidity sensor, a vibration sensor, an infrared sensor, or an audio sensor.

10. The method of claim 1, wherein the wearable device comprises one or more of the following:

smartwatches, wristbands, gloves, smart rings, or armbands.

11. The method of claim 1, wherein the analyzing of the sensor data is carried out with aid of one or more processors onboard the wearable device.

12. The method of claim 1, wherein the analyzing of the sensor data is carried out with aid of one or more processors onboard a user device or a remote server in communication with the wearable device, wherein the user device and the remote server are provided separate from the wearable device.

13. The method of claim 12, further comprising:

storing the sensor data in a memory on the wearable device when the wearable device is not in operable communication with the user device and/or the server.

14. A system for implementing gesture recognition, comprising:

a memory for storing sensor data collected using at least two different types of sensors located on a wearable device, wherein said wearable device is configured to be worn by a user, the at least two different types of sensors comprise an accelerometer and a gyroscope, and wherein the sensor data comprises an acceleration vector obtained from the accelerometer and an angular velocity vector obtained from the gyroscope; and

one or more processors configured to analyze the sensor data by evaluating a magnitude of the acceleration vector and a magnitude of the angular velocity vector and determining a correlation between the magnitudes of the acceleration vector and angular velocity vector within different temporal periods to determine a likelihood of the user performing a predefined gesture, wherein the likelihood is determined without comparing the acceleration vector and the angular velocity vector to one or more physical motion profiles.

15. The system of claim 14, wherein the predefined gesture is selected from a group of different gestures associated with different activities including the user smoking or about to smoke a cigarette, eating, drinking, or taking medication.

16. The system of claim 14, wherein the one or more processors are configured to (1) calculate a pitch angle, roll angle, or yaw angle of the wearable device based on at least the acceleration vector or the angular velocity vector, and (2) determine the likelihood of the user performing the predefined gesture based on the calculated angle(s).

17. The system of claim 16, wherein the sensor data comprises a plurality of features encoded within said sensor data, and wherein the plurality of features are associated with various aspects of the predefined gesture.

18. The system of claim 17, wherein the one or more processors are configured to analyze the sensor data by calculating a multi-dimensional distribution function, wherein said multi-dimensional distribution function is a probability function of the plurality of features.

19. The system of claim 17, wherein the plurality of features comprise two or more of the following features:

(1) a time duration of a submotion during the gesture;

(2) the magnitude of the acceleration vector;

(3) the magnitude of the angular velocity vector;

(4) the roll angle; and

(5) the pitch angle.

20. The system of claim 17, wherein the one or more processors are configured to execute a machine learning algorithm, and wherein the plurality of features is input into the machine learning algorithm to extract said features from the sensor data to determine the likelihood of the user performing the predefined gesture.

21. The system of claim 20, wherein the machine learning algorithm comprises supervised-learning, semi-supervised learning or unsupervised learning techniques.

22. The system of claim 14, wherein the at least two different types of sensors further comprise one or more of the following:

a magnetometer, a heart rate monitor, a global positioning system (GPS) receiver, a barometer, an external temperature sensor, a microphone, a skin temperature sensor, a capacitive sensor, a sensor configured to detect a galvanic skin response, an imaging device, a proximity sensor, an altitude sensor, an attitude sensor, a humidity sensor, a vibration sensor, an infrared sensor, or an audio sensor.

23. The system of claim 14, wherein the wearable device comprises one or more of the following:

smartwatches, wristbands, gloves, smart rings, or armbands.

24. The system of claim 14, wherein the one or more processors are located onboard the wearable device.

25. The system of claim 14, wherein the one or more processors are located onboard a user device or a remote server in communication with the wearable device, wherein the user device and the remote server are provided separate from the wearable device.

26. The system of claim 25, wherein the memory is located on the wearable device and configured to store the sensor data when the wearable device is not in operable communication with the user device or the server.