Methods and systems for monitoring and influencing gesture-based behaviors

• US 10,126,829 B2
• Filed: 05/23/2017
• Issued: 11/13/2018
• Est. Priority Date: 12/16/2014
• Status: Active Grant

• Current Assignee: Somatix, Inc.
• Original Assignee: Somatix, Inc.
• Inventors: Ofir, Eran, Schatzberg, Uri
• Primary Examiner(s): Lee, Paul D
• Application Number:
  US15/603,246
• Publication Number:
  US 20170262064A1
• Time in Patent Office: 539 Days
• Field of Search: 702127
• CPC Class Codes: A24F 47/00 Smokers' requisites not otherwise provided for
• CPC Class Codes: A61B 2562/0219 Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
• CPC Class Codes: A61B 2562/0228 Microwave sensors
• CPC Class Codes: A61B 5/00 Measuring for diagnostic purposes radiation diagnosis A61B6/00; diagnosis by ultrasonic, sonic or infrasonic waves A61B8/00; Identification of persons
• CPC Class Codes: A61B 5/11 Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb for measuring pulse A61B5/02 ;A61B5/1038 takes precedence; motion detection to correct for motion artifacts in physiological signals A61B5/721
• CPC Class Codes: A61B 5/1116 Determining posture transitions
• CPC Class Codes: A61B 5/1123 Discriminating type of movement, e.g. walking or running A61B5/1116, A61B5/112 take precedence
• CPC Class Codes: A61B 5/681 Wristwatch-type devices
• CPC Class Codes: A61B 5/6824 Arm or wrist
• CPC Class Codes: G06F 3/00 Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

Methods and systems are provided herein for analyzing, monitoring, and/or influencing a user'"'"'s behavioral gesture in real-time. A gesture recognition method may be provided. The method may comprise: obtaining sensor data collected using at least one sensor located on a wearable device, wherein said wearable device is configured to be worn by a user; and analyzing the sensor data to determine a probability of the user performing a predefined gesture, wherein the probability is determined based in part on a magnitude of a motion vector in the sensor data, and without comparing the motion vector to one or more physical motion profiles.

Claims

1. A gesture recognition method comprising:

selectively activating a first sensor out of a sensor group located on a wearable device, wherein said wearable device is configured to be worn by a user, and the sensor group comprises a first sensor and a second sensor;

obtaining first sensor data collected using the first sensor;

analyzing the first sensor data to determine a likelihood of the user performing a predefined gesture based on a magnitude of a first motion vector in the first sensor data;

determining whether the likelihood that the user is performing the predefined gesture is below or above a threshold value;

selectively activating the second sensor to collect additional sensor data comprising a second motion vector having a magnitude when the likelihood that the user is performing the predefined gesture is above the threshold value; and

comparing the magnitudes of the first and second motion vectors in the collected sensor data without requiring comparing of the first and second motion vectors to a plurality of physical motion profiles, to determine whether the user is performing the predefined gesture.

2. The method of claim 1, wherein the analyzing of the sensor data is carried out with aid of one or more processors onboard the wearable device.

3. The method of claim 1, wherein the analyzing of the sensor data is carried out with aid of one or more processors onboard a user device or a remote server in communication with the wearable device, wherein the user device and the remote server are provided separate from the wearable device.

4. The method of claim 1, wherein the predefined gesture is selected from a group of different gestures associated with different activities including the user smoking or about to smoke a cigarette, eating, drinking, or taking medication.

5. The method of claim 1, where the sensor group comprises an accelerometer and a gyroscope, wherein the magnitudes of the two or more motion vectors comprise:

(1) a magnitude of an acceleration vector obtained from the accelerometer, and (2) a magnitude of an angular velocity vector obtained from the gyroscope.

6. The method of claim 5, further comprising:

determining the likelihood of the user performing the predefined gesture by comparing the magnitudes of the acceleration vector and the angular velocity vector.

7. The method of claim 5, further comprising:

calculating a pitch angle, roll angle, and/or yaw angle of the wearable device based on at least the acceleration vector or the angular velocity vector, and determining the likelihood of the user performing the predefined gesture based on the calculated angle(s).

8. The method of claim 1, wherein the sensor group comprises one or more of the following:

a magnetometer, a heart rate monitor, a global positioning system (GPS) receiver, a barometer, an external temperature sensor, a microphone, a skin temperature sensor, a capacitive sensor, or a sensor configured to detect a galvanic skin response.

9. The method of claim 1, wherein the analyzing of the sensor data comprises calculating a multi-dimensional distribution function, wherein said multi-dimensional distribution function comprises a plurality of features associated with various aspects of the predefined gesture, and wherein the plurality of features are extracted from the sensor data and de-correlated using Singular Value Decomposition (SVD).

10. The method of claim 1, further comprising:

applying a data compression step to the sensor data, wherein the data compression step is configured to at least reduce a bandwidth required to transmit the sensor data or reduce a power consumption of the wearable device during the transmission of the sensor data.

11. The method of claim 10, wherein the sensor group comprises a plurality of sensors configured to collect the sensor data at a plurality of different frequencies, wherein the data compression step is used to extract a portion of the sensor data when at least one of the plurality of sensors is collecting the sensor data above a certain frequency, and wherein the extracted portion of the sensor data is analyzed to determine the likelihood of the user performing the predefined gesture.

12. The method of claim 1, wherein the second sensor is inactive when the likelihood that the user is performing the predefined gesture is below the threshold value, and wherein the second sensor is selectively activated (1) when the wearable device is powered on and the likelihood that the user is performing the predefined gesture is above the threshold value, or (2) upon determining that the user is performing the predefined gesture.

13. The method of claim 1, wherein the wearable device is configured to operate in a plurality of modes comprising a low power mode and an accuracy mode, and wherein an accuracy of detecting whether the user is performing the predefined gesture is improved when the wearable device is in the accuracy mode compared to the low power mode.

14. A gesture recognition method comprising:

obtaining sensor data collected using at least one sensor located on a wearable device, wherein said wearable device is configured to be worn by a user;

analyzing the sensor data by comparing magnitudes of two or more motion vectors in the sensor data without requiring comparing of the motion vectors to a plurality of physical motion profiles, to determine a likelihood of the user performing a predefined gesture; and

determining whether the likelihood that the user is performing the predefined gesture is below or above a threshold value,

wherein the at least one sensor (1) starts collecting additional sensor data at a first predetermined frequency when the likelihood that the user is performing the predefined gesture is below the threshold value, or (2) starts collecting additional sensor data at a second predetermined frequency different from the first predetermined frequency when the likelihood that the user is performing the predefined gesture is above the threshold value.

15. A system for implementing gesture recognition, comprising:

a memory for storing sensor data collected using a sensor group located on a wearable device, wherein said wearable device is configured to be worn by a user, and the sensor group comprises a first sensor and a second sensor; and

one or more processors configured to;

selectively activate the first sensor;

obtain first sensor data collected using the first sensor;

analyze the first sensor data to determine a likelihood of the user performing a predefined gesture based on a magnitude of a first motion vector in the first sensor data;

determine whether the likelihood that the user is performing the predefined gesture is below or above a threshold value;

selectively activate the second sensor to collect additional sensor data comprising a second motion vector having a magnitude when the likelihood that the user is performing the predefined gesture is above the threshold value; and

compare the magnitudes of the first and second motion vectors in the collected sensor data without requiring comparing of the first and second motion vectors to a plurality of physical motion profiles, to determine whether the user is performing the predefined gesture.

16. The system of claim 15, wherein the one or more processors are located onboard the wearable device.

17. The system of claim 15, wherein the one or more processors are located onboard a user device or a remote server in communication with the wearable device, wherein the user device and the remote server are provided separate from the wearable device.

18. The system of claim 15, wherein the predefined gesture is selected from a group of different gestures associated with different activities including the user smoking or about to smoke a cigarette, eating, drinking, or taking medication.

19. The system of claim 15, where the sensor group comprises an accelerometer and a gyroscope, wherein the magnitudes of the two or more motion vectors comprise:

(1) a magnitude of an acceleration vector obtained from the accelerometer, and (2) a magnitude of an angular velocity vector obtained from the gyroscope.

20. The system of claim 19, where the one or more processors are configured to determine the likelihood of the user performing the predefined gesture by comparing the magnitudes of the acceleration vector and the angular velocity vector.

21. The system of claim 19, where the one or more processors are configured to calculate a pitch angle, roll angle, and/or yaw angle of the wearable device based on at least the acceleration vector or the angular velocity vector, and determine the likelihood of the user performing the predefined gesture based on the calculated angle(s).

22. The system of claim 15, wherein the sensor group comprises one or more of the following:

a magnetometer, a heart rate monitor, a global positioning system (GPS) receiver, a barometer, an external temperature sensor, a microphone, a skin temperature sensor, a capacitive sensor, or a sensor configured to detect a galvanic skin response.

23. The system of claim 15, wherein the one or more processors are configured to analyze the sensor data by calculating a multi-dimensional distribution function, wherein said multi-dimensional distribution function comprises a plurality of features associated with various aspects of the predefined gesture, and wherein the plurality of features are extracted from the sensor data and de-correlated using Singular Value Decomposition (SVD).

24. The system of claim 15, wherein the one or more processors are configured to apply a data compression step to the sensor data, wherein the data compression step is configured to at least reduce a bandwidth required to transmit the sensor data or reduce a power consumption of the wearable device during the transmission of the sensor data.

25. The system of claim 17, wherein the sensor group comprises a plurality of sensors configured to collect the sensor data at a plurality of different frequencies, wherein the data compression step is used to extract a portion of the sensor data when at least one of the plurality of sensors is collecting the sensor data above a certain frequency, and wherein the one or more processors are configured to analyze the extracted portion of the sensor data to determine the likelihood of the user performing the predefined gesture.

26. The system of claim 15, wherein the second sensor is inactive when the likelihood that the user is performing the predefined gesture is below the threshold value, wherein the one or more processors are configured to selectively activate the second sensor (1) when the wearable device is powered on and the likelihood that the user is performing the predefined gesture is above the threshold value, or (2) upon determining that the user is performing the predefined gesture.

27. The system of claim 15, wherein the wearable device is configured to operate in a plurality of modes comprising a low power mode and an accuracy mode, and wherein an accuracy of detecting whether the user is performing the predefined gesture is improved when the wearable device is in the accuracy mode compared to the low power mode.

28. A system for implementing gesture recognition, comprising:

a memory for storing sensor data collected using at least one sensor located on a wearable device, wherein said wearable device is configured to be worn by a user; and

one or more processors configured to;

analyze the sensor data by comparing magnitudes of two or more motion vectors in the sensor data, without comparing of the motion vectors to a plurality of physical motion profiles, to determine a likelihood of the user performing a predefined gesture; and

determine whether the likelihood that the user is performing the predefined gesture is below or above a threshold value,

wherein the at least one (1) starts collecting additional sensor data at a first predetermined frequency when the likelihood that the user is performing the predefined gesture is below the threshold value, or (2) starts collecting additional sensor data at a second predetermined frequency different from the first predetermined frequency when the likelihood that the user is performing the predefined gesture is above the threshold value.