User identification via motion and heartbeat waveform data

• US 9,851,808 B2
• Filed: 05/23/2017
• Issued: 12/26/2017
• Est. Priority Date: 08/07/2015
• Status: Active Grant

• Current Assignee: Fitbit, Inc. (Alphabet Inc.)
• Original Assignee: Fitbit, Inc. (Alphabet Inc.)
• Inventors: Yuen, Shelten Gee Jao, Park, James, Ghoreyshi, Atiyeh, Wu, Anjian
• Primary Examiner(s): Schwartz, Darren B
• Application Number:
  US15/603,305
• Publication Number:
  US 20170255273A1
• Time in Patent Office: 217 Days
• CPC Class Codes: A61B 2560/0223 of calibration, e.g. protocols for calibrating sensors
• CPC Class Codes: A61B 2562/0219 Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
• CPC Class Codes: A61B 5/0002 Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network A61B5/07 takes precedence; transmission systems for measured values G08C; transmission H04B; transmission of digital information H04L; wireless communication networks H04W
• CPC Class Codes: A61B 5/0022 Monitoring a patient using a global network, e.g. telephone networks, internet
• CPC Class Codes: A61B 5/0024 for multiple sensor units attached to the patient, e.g. using a body or personal area network
• CPC Class Codes: A61B 5/02416 using photoplethysmograph signals, e.g. generated by infrared radiation A61B5/14552 takes precedence
• CPC Class Codes: A61B 5/02438 with portable devices, e.g. worn by the patient
• CPC Class Codes: A61B 5/1032 Determining colour for diagnostic purposes measuring colour in general G01J3/46
• CPC Class Codes: A61B 5/11 Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb for measuring pulse A61B5/02 ;A61B5/1038 takes precedence; motion detection to correct for motion artifacts in physiological signals A61B5/721
• CPC Class Codes: A61B 5/1112 Global tracking of patients, e.g. by using GPS
• CPC Class Codes: A61B 5/1118 Determining activity level
• CPC Class Codes: A61B 5/112 Gait analysis
• CPC Class Codes: A61B 5/1123 Discriminating type of movement, e.g. walking or running A61B5/1116, A61B5/112 take precedence
• CPC Class Codes: A61B 5/117 Identification of persons methods or arrangements for recognising patterns, e.g. fingerprints, G06F18/00, G06V40/00; identification of persons by analysing their voice or speech G10L17/00
• CPC Class Codes: A61B 5/1172 using fingerprinting
• CPC Class Codes: A61B 5/318 Heart-related electrical modalities, e.g. electrocardiography [ECG]
• CPC Class Codes: A61B 5/332 Portable devices specially adapted therefor
• CPC Class Codes: A61B 5/441 Skin evaluation, e.g. for skin disorder diagnosis
• CPC Class Codes: A61B 5/6802 Sensor mounted on worn items
• CPC Class Codes: A61B 5/681 Wristwatch-type devices
• CPC Class Codes: A61B 5/6844 : Monitoring or controlling distance between sensor and tissue
• CPC Class Codes: A61B 5/725 : using specific filters therefor, e.g. Kalman or adaptive filters A61B5/316 take precedence
• CPC Class Codes: A61B 5/7257 : using Fourier transforms
• CPC Class Codes: A61B 5/7264 : Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems neural networks per se G06N3/00; expert systems per se G06N5/00
• CPC Class Codes: A61B 5/7267 : involving training the classification device
• CPC Class Codes: A61B 5/7271 : Specific aspects of physiological measurement analysis A61B5/316, A61B5/349 and A61B5/374 take precedence
• CPC Class Codes: A61B 5/742 : using visual displays A61B5/339 takes precedence
• CPC Class Codes: A61B 5/7475 : User input or interface means, e.g. keyboard, pointing device, joystick
• CPC Class Codes: A63B 24/0062 : Monitoring athletic performances, e.g. for determining the work of a user on an exercise apparatus, the completed jogging or cycling distance
• CPC Class Codes: A63B 69/36 : for golf
• CPC Class Codes: G01C 22/006 : Pedometers
• CPC Class Codes: G01P 1/02 : Housings
• CPC Class Codes: G01P 13/00 : Indicating or recording presence, absence, or direction, of movement electric switches H01H; counting moving objects G06M7/00
• CPC Class Codes: G01S 19/00 : Satellite radio beacon positioning systems; Determining position, velocity or attitude using signals transmitted by such systems
• CPC Class Codes: G01S 19/19 : Sporting applications
• CPC Class Codes: G06F 1/163 : Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 21/32 : using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 2218/10 : by analysing the shape of a waveform, e.g. extracting parameters relating to peaks
• CPC Class Codes: G06F 3/017 : Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06Q 50/01 : Social networking
• CPC Class Codes: G06V 40/10 : Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/15 : Biometric patterns based on physiological signals, e.g. heartbeat, blood flow
• CPC Class Codes: G06V 40/20 : Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/23 : Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G09B 5/02 : with visual presentation of the material to be studied, e.g. using film strip
• CPC Class Codes: G16H 50/70 : for mining of medical data, e.g. analysing previous cases of other patients

Abstract

The disclosure relates to methods, devices, and systems to identify a user of a wearable fitness monitor using data obtained using the wearable fitness monitor. Data obtained from motion sensors of the wearable fitness monitor and data obtained from heartbeat waveform sensors of the wearable fitness monitor may be used to identify the user.

Claims

1. A method, implemented on a wearable fitness monitor comprising one or more motion sensors, one or more heartbeat waveform sensors, and one or more processors, the method comprising:

(a) obtaining, by the one or more processors, a motion signature obtained using data from the one or more motion sensors of a wearable fitness monitor configured to be worn by a person, wherein the motion signature characterizes a movement experienced by the wearable fitness monitor;

(b) obtaining, by the one or more processors, a heartbeat waveform signature obtained using data from the one or more heartbeat waveform sensors, wherein the heartbeat waveform signature characterizes a detected heartbeat waveform of a wearer of the wearable fitness monitor;

(c) obtaining, by the one or more processors, a first test feature vector from the motion signature and a second test feature vector from the heartbeat waveform signature;

(d) applying, by the one or more processors, a machine learning classifier to the first test feature vector and the second test feature vector, wherein the machine learning classifier was trained using motion data and heartbeat waveform data obtained from a reference user, and wherein the machine learning classifier is configured to receive feature vectors as inputs and provide classifications of the feature vectors as outputs, wherein the classifications indicate whether the feature vectors belong to the reference user;

(e) obtaining, using the machine learning classifier and by the one or more processors, a classification indicating whether the first test feature vector and the second test feature vector belong to the reference user; and

(f) determining, based on the classification obtained in (e) and by the one or more processors, whether an identity of the wearer of the wearable fitness monitor is the user.

2. The method of claim 1, wherein the one or more motion sensors comprise an accelerometer, a gyroscope, a magnetometer, an altimeter, a GPS receiver, or any combination thereof.

3. The method of claim 1, wherein the heartbeat waveform signature comprises a heartbeat waveform or one or more features derived therefrom.

4. The method of claim 3, wherein the one or more features derived from the heartbeat waveform comprise a slope of the heartbeat waveform, an area of the heartbeat waveform, a curvature of the heartbeat waveform, a polynomial fit of the heartbeat waveform, or any combination thereof.

5. The method of claim 3, wherein the one or more features derived from the heartbeat waveform comprise a frequency domain characteristic of the heartbeat waveform.

6. The method of claim 5, wherein the frequency domain characteristic of the heartbeat waveform is obtained from a Fourier transform or a cosine transform of the heartbeat waveform.

7. The method of claim 1, wherein the one or more heartbeat waveform sensors comprise a electrocardiography (ECG) sensor, a photoplethysmography (PPG) sensor, or any combination thereof.

8. The method of claim 7, wherein the one or more heartbeat waveform sensors comprise the PPG sensor and the ECG sensor.

9. The method of claim 7, wherein the one or more heartbeat waveform sensors comprise a PPG sensor, the method further comprising:

obtaining a skin characteristic of the user using the PPG sensor.

10. The method of claim 1, wherein the machine learning classifier comprises a linear discriminant analysis classifier, a neural network model, a support vector machine, a logistic regression analysis classifier, a naive Bayes, a random forest, a decision tree, and any combination thereof.

11. The method of claim 1, the motion signature comprises a profile of a step by the user.

12. The method of claim 1, wherein the machine learning classifier comprises a linear discriminant analysis classifier.

13. The method of claim 1, wherein the movement experienced by the wearable fitness monitor is caused by a periodic motion selected from the group consisting of running, walking, cycling, swimming, weight lifting, climbing, rowing, a gymnastic exercise, dancing, an aerobic exercise, a yoga routine, golfing, swinging a club, swinging a racquet, striking a ball or another object, swimming, diving, surfing, skating, skiing, skate boarding, exercising on a machine, driving a vehicle, and riding an animal.

14. The method of claim 1, further comprising, responsive to determining that the identity of the wearer of the fitness monitor is not the user, preventing the wearable fitness monitor from allowing a transaction.

15. The method of claim 1, further comprising, responsive to determining that the identity of the wearer of the fitness monitor is not the user, requiring the user to authenticate himself or herself.

16. The method of claim 1, further comprising, responsive to determining that the identity of the wearer of the fitness monitor is not the user, discrediting a fitness metric obtained for the user via the wearable fitness monitor.

17. The method of claim 1, further comprising, responsive to determining that the identity of the wearer of the fitness monitor is the user, crediting a fitness metric obtained for the user via the wearable fitness monitor.

18. The method of claim 1, further comprising, responsive to determining that the identity of the wearer of the fitness monitor is the user, allowing the wearable fitness monitor to facilitate a transaction.

19. A system comprising:

(A) a wearable fitness monitor configured to be worn by a person and comprising;

one or more first motion sensors,

one or more heartbeat waveform sensors, and

a communication interface configured for communicating data from the one or more first motion sensors to a device external to the wearable fitness monitor; and

(B) classification logic configured to;

(a) obtain a motion signature obtained using data from the one or more motion sensors, wherein the motion signature characterizes a movement experienced by the wearable fitness monitor,

(b) obtain a heartbeat waveform signature obtained using data from the one or more heartbeat waveform sensors, wherein the heartbeat waveform signature characterizes a detected heartbeat waveform of a wearer of the wearable fitness monitor,

(c) obtain a first test feature vector from the motion signature and a second test feature vector from the heartbeat waveform signature,

(d) apply a machine learning classifier to the first test feature vector and the second test feature vector, wherein the machine learning classifier was trained using motion data and heartbeat waveform data obtained from a reference user, and wherein the machine learning classifier is configured to receive feature vectors as inputs and provide classifications of the feature vectors as outputs, wherein the classifications indicate whether the feature vectors belong to the reference user;

(e) obtain, using the machine learning classifier, a classification indicating whether the first test feature vector and the second test feature vector belong to the reference user; and

(f) based on the classification obtained in (e), determine whether an identity of the wearer of the wearable fitness monitor is the user.

20. A computer program product comprising a non-transitory machine readable medium storing program code that, when executed by one or more processors, causes the one or more processors to implement a method for determining an identity of a wearer of a wearable fitness monitor comprising one or more motion sensors and one or more heartbeat waveform sensors, said program code comprising:

(a) code for obtaining a motion signature obtained using data from the one or more motion sensors, wherein the motion signature characterizes a movement experienced by the wearable fitness monitor,

(b) code for obtaining a heartbeat waveform signature obtained using data from the one or more heartbeat waveform sensors, wherein the heartbeat waveform signature characterizes a detected heartbeat waveform of the wearer of the wearable fitness monitor,

(c) code for obtaining a first test feature vector from the motion signature and a second test feature vector from the heartbeat waveform signature,

(d) code for applying a machine learning classifier to the first test feature vector and the second test feature vector, wherein the machine learning classifier was trained using motion data and heartbeat waveform data obtained from a reference user, and wherein the machine learning classifier is configured to receive feature vectors as inputs and provide classifications of the feature vectors as outputs, wherein the classifications indicate whether the feature vectors belong to the reference user,

(e) code for obtaining, using the machine learning classifier, a classification indicating whether the first test feature vector and the second test feature vector belong to the reference user, and

(f) code for determining, based on the classification obtained in (e), whether an identity of the wearer of the wearable fitness monitor is the user.