System for monitoring individuals as they age in place

• US 9,795,324 B2
• Filed: 12/05/2014
• Issued: 10/24/2017
• Est. Priority Date: 09/05/2014
• Status: Active Grant

• Current Assignee: Beijing Zitiao Network Technology Company Limited (ByteDance Ltd.)
• Original Assignee: Vision Service Plan (VSP Global, Inc.)
• Inventors: Sales, Jay William, Klosinski, Jr., Richard Chester, Workman, Matthew Allen, Murphy, Meghan Kathleen, Steen, Matthew David
• Primary Examiner(s): Padmanabhan, Kavita
• Assistant Examiner(s): BURKE, TIONNA M
• Application Number:
  US14/562,454
• Publication Number:
  US 20160071390A1
• Time in Patent Office: 1,054 Days
• Field of Search: 715236
• CPC Class Codes: A61B 2560/0475 Special features of memory means, e.g. removable memory cards
• CPC Class Codes: A61B 2562/0219 Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
• CPC Class Codes: A61B 2562/0223 Magnetic field sensors
• CPC Class Codes: A61B 2562/0257 Proximity sensors
• CPC Class Codes: A61B 2576/00 Medical imaging apparatus involving image processing or analysis A61B1/00009, A61B6/52 and A61B8/52 take precedence; image processing per se G06T; details of algorithms for analysing medical images G06T7/0012
• CPC Class Codes: A61B 3/112 for measuring diameter of pupils
• CPC Class Codes: A61B 5/0002 Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network A61B5/07 takes precedence; transmission systems for measured values G08C; transmission H04B; transmission of digital information H04L; wireless communication networks H04W
• CPC Class Codes: A61B 5/0022 Monitoring a patient using a global network, e.g. telephone networks, internet
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/0205 Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
• CPC Class Codes: A61B 5/024 Detecting, measuring or recording pulse rate or heart rate A61B5/0205, A61B5/021 take precedence
• CPC Class Codes: A61B 5/0531 Measuring skin impedance
• CPC Class Codes: A61B 5/0816 Measuring devices for examining respiratory frequency measuring frequency of electric signals G01R23/00
• CPC Class Codes: A61B 5/1032 Determining colour for diagnostic purposes measuring colour in general G01J3/46
• CPC Class Codes: A61B 5/1103 Detecting eye twinkling
• CPC Class Codes: A61B 5/1114 Tracking parts of the body
• CPC Class Codes: A61B 5/1116 Determining posture transitions
• CPC Class Codes: A61B 5/1118 Determining activity level
• CPC Class Codes: A61B 5/112 Gait analysis
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A61B 5/117 : Identification of persons methods or arrangements for recognising patterns, e.g. fingerprints, G06F18/00, G06V40/00; identification of persons by analysing their voice or speech G10L17/00
• CPC Class Codes: A61B 5/1176 : Recognition of faces
• CPC Class Codes: A61B 5/14552 : Details of sensors specially adapted therefor
• CPC Class Codes: A61B 5/165 : Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/318 : Heart-related electrical modalities, e.g. electrocardiography [ECG]
• CPC Class Codes: A61B 5/369 : Electroencephalography [EEG] devices for psychotechnics A61B5/16
• CPC Class Codes: A61B 5/4076 : Diagnosing or monitoring particular conditions of the nervous system A61B5/4821, A61B5/4824 take precedence
• CPC Class Codes: A61B 5/4266 : sweat secretion
• CPC Class Codes: A61B 5/443 : Evaluating skin constituents, e.g. elastin, melanin, water
• CPC Class Codes: A61B 5/486 : Bio-feedback A61B5/375 takes precedence
• CPC Class Codes: A61B 5/4884 : inducing physiological or psychological stress, e.g. applications for stress testing
• CPC Class Codes: A61B 5/6803 : Head-worn items, e.g. helmets, masks, headphones or goggles
• CPC Class Codes: A61B 5/7246 : using correlation, e.g. template matching or determination of similarity
• CPC Class Codes: A61B 5/7278 : Artificial waveform generation or derivation, e.g. synthesising signals from measured signals A61B5/327 takes precedence
• CPC Class Codes: A61B 5/7282 : Event detection, e.g. detecting unique waveforms indicative of a medical condition A61B5/4094, A61B5/4818, A61B5/349, A61B5/0823 take precedence
• CPC Class Codes: A61B 7/04 : Electric stethoscopes microphones, acoustic transducers therefor H04R
• CPC Class Codes: A61F 2/76 : Means for assembling, fitting or testing prostheses, e.g. for measuring or balancing, e.g. alignment means
• CPC Class Codes: A61F 2002/7695 : Means for testing non-implantable prostheses for testing implantable prostheses A61F2240/008; for implanting artificial joints A61F2/468
• CPC Class Codes: A63B 24/0062 : Monitoring athletic performances, e.g. for determining the work of a user on an exercise apparatus, the completed jogging or cycling distance
• CPC Class Codes: G02C 11/10 : Electronic devices other than hearing aids
• CPC Class Codes: G06F 18/22 : Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 21/35 : communicating wirelessly
• CPC Class Codes: G06F 2218/00 : Aspects of pattern recognition specially adapted for signal processing
• CPC Class Codes: G06V 20/10 : Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 40/18 : Eye characteristics, e.g. of the iris
• CPC Class Codes: G06V 40/19 : Sensors therefor
• CPC Class Codes: G06V 40/197 : Matching; Classification
• CPC Class Codes: G06V 40/25 : Recognition of walking or running movements, e.g. gait recognition
• CPC Class Codes: G07C 9/37 : using biometric data, e.g. fingerprints, iris scans or voice recognition
• CPC Class Codes: G08B 21/02 : Alarms for ensuring the safety of persons
• CPC Class Codes: G08B 21/0423 : detecting deviation from an expected pattern of behaviour or schedule
• CPC Class Codes: G08B 21/0461 : integrated or attached to an item closely associated with the person but not worn by the person, e.g. chair, walking stick, bed sensor
• CPC Class Codes: G08B 21/0476 : Cameras to detect unsafe condition, e.g. video cameras
• CPC Class Codes: G09B 19/0092 : Nutrition
• CPC Class Codes: G09B 5/00 : Electrically-operated educational appliances working with questions and answers G09B7/00; simulators G09B9/00; advertising or displaying in general G09F
• CPC Class Codes: G09B 5/06 : with both visual and audible presentation of the material to be studied
• CPC Class Codes: G16H 20/40 : relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
• CPC Class Codes: G16H 40/63 : for local operation
• CPC Class Codes: G16H 40/67 : for remote operation
• CPC Class Codes: G16H 50/20 : for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: H04L 63/0861 : using biometrical features, e.g. fingerprint, retina-scan cryptographic mechanisms or cryptographic arrangements for entity authentication using biological data H04L9/3231

Abstract

A computer-implemented method, and related system, for monitoring the wellbeing of an individual by providing eyewear that includes at least one sensor for monitoring the motion of the user. In various embodiments, the system receives data generated by the at least one sensor, uses the data to determine the user'"'"'s movements using the received data, and compares the user'"'"'s movements to previously established movement patterns of the user. If the system detects one or more inconsistencies between the user'"'"'s current movements as compared to the previously established movement patterns of the user, the system may notify the user or a third party of the detected one or more inconsistencies. The system may similarly monitor a user'"'"'s compliance with a medical regime and notify the user or a third party of the user'"'"'s compliance with the regime.

Claims

1. A computer-implemented method of monitoring the wellbeing of an individual, the method comprising the steps of:

a. providing a user with computerized eyewear comprising at least one sensor for monitoring the motion of the user;

b. receiving, by one or more processors, a pre-defined command from the user, for the at least one sensor to generate a first set of data identifying one or more user-defined movement patterns for the user;

c. in response to receiving the pre-defined command, collecting, by one or more processors, via the at least one sensor, the first set of data;

d. using, by one or more processors, the first set of data to determine one or more typical movement patterns of the user;

e. generating, by one or more processors, an established one or more user-defined movement patterns for the user based on the first set of data generated by the at least one sensor after the user with the computerized eyewear provides the pre-defined command for the at least one sensor to generate the first set of data identifying the one or more user-defined movement patterns for the user;

f. receiving, by one or more processors, a second set of data generated by the at least one sensor after the established one or more user-defined movement patterns have been generated;

g. at least partially in response to receiving the second set of data generated by the at least one sensor, determining, by one or more processors, the user'"'"'s movements using the received second set of data;

h. at least partially in response to determining the user'"'"'s movements, comparing, by one or more processors, the user'"'"'s movements based on the second set of data to the previously established one or more user-defined movement patterns for the user;

i. detecting, by one or more processors, one or more inconsistencies between the current user'"'"'s determined movements as compared to the previously established one or more user-defined movement patterns;

j. at least partially in response to detecting the one more inconsistencies, notifying, by one or more processors, at least one recipient of the one or more inconsistencies, where the at least one recipient is a recipient selected from a group consisting of;

the user and a third party.

2. The computer-implemented method of claim 1, wherein the at least one sensor comprises at least one sensor selected from a group consisting of:

a. a motion sensor;

b. an accelerometer;

c. a gyroscope;

d. a geomagnetic sensor;

e. a global positioning system sensor;

f. an impact sensor;

g. a microphone;

h. a forward facing camera;

i. a heart rate monitor;

j. a pulse oximeter;

k. a blood alcohol monitor;

l. a respiratory rate sensor; and

m. a transdermal sensor.

3. The computer-implemented method of claim 2, wherein the at least one sensor comprises at least one sensor selected from a group consisting of:

a motion sensor, an accelerometer, a global positioning sensor, a gyroscope, and a forward facing camera.

4. The computer-implemented method of claim 2, wherein the method further comprises the step of:

a. calculating, by a processor, a number of steps taken by the user in a particular day;

b. at least partially in response to calculating the number of steps, comparing, by a processor, the calculated number of steps taken by the user in the particular day to a predetermined average number of steps taken by the user in a day; and

c. at least partially in response to comparing the calculated number of steps to the predetermined average number of steps, notifying the user or a third party if the calculated number of steps in the particular day is less than a predetermined percentage of the predetermined average number of steps taken by the user in a day.

5. The computer-implemented method of claim 2, further comprising the steps of:

a. detecting, by a processor, whether the user moves during a predefined time period; and

b. at least partially in response to detecting whether the user moves during the predefined time period, notifying, by a processor, the at least one recipient selected from a group consisting of;

the user or a third party if the user does not move during the predefined time period.

6. The computer-implemented method of claim 2, further comprising the steps of:

a. detecting, by a processor, from the received data generated by the at least one sensor if the user experiences a sudden acceleration or sudden impact; and

b. at least partially in response to detecting that the user has experienced a sudden acceleration or sudden impact, notifying, by a processor, the user or a third party that the user experienced the sudden acceleration or sudden impact.

7. The computer-implemented method of claim 2, further comprising the steps of:

a. detecting, by a processor, from the received data generated by the at least one sensor;

(1) whether the user is breathing; and

(2) whether the user'"'"'s heart is beating; and

b. at least partially in response to determining that the user is not breathing or that the user'"'"'s heart is not beating, sending a notification to a third party.

8. The computer-implemented method of claim 2, further comprising the steps of:

a. receiving, by a processor, from the user or third party, a medicine regime associated with the user;

b. storing, by a processor, the medicine regime in memory;

c. receiving, by a processor, data generated by a forward facing camera associated with the computerized eyewear;

d. analyzing, by a processor, the received data to determine data selected from a group consisting of one or more;

i. types of medicine taken by the user;

ii. times the medicine is taken by the user; and

iii. doses of the medicine taken by the user;

e. at least partially in response to analyzing the received data, comparing, by a processor, the one or more of the types of medicine taken, the one or more times the medicine is taken, or the one or more doses of medicine taken to the stored medicine regime for the user;

f. at least partially in response to comparing the one or more of the type of medicine taken, the time the medicine is taken and the dose of medicine taken, identifying, by a processor, one or more inconsistencies between the stored medicine regime, and the one or more types of medicine taken, the one or more times the medicine is taken, or the one or more doses of medicine taken;

g. at least partially in response to identifying the one or more inconsistencies between the medicine regime and the one or more of the types of medicine taken, the one or more times the medicine is taken, or the one or more doses of medicine taken, sending an alert to the user or a third party of the one or more inconsistencies.

9. The computer-implemented method of claim 8, wherein:

a. the data generated comprises one or more images captured by the forward facing camera;

b. the step of analyzing the received data further comprises;

i. detecting, by a processor, one or more pills in the one or more images;

ii. comparing, by a processor, the one or more detected pills found in the one or more images to one or more known images of pills stored in a database;

iii. identifying, by a processor, the one or more pills by matching the one or more pills from the one or more images to the one or more known images of pills stored in the database; and

iv. detecting, by a processor, a time that the one or more images were taken.

19. The computer-implemented method of claim 1, wherein the pre-defined command is defined by the user.

10. A computer-implemented method of monitoring the wellbeing of an individual, the method comprising the steps of:

a. providing a user with computerized eyewear comprising at least one sensor for monitoring actions taken by a user, the at least one sensor including a forward-facing camera that is a component of the computerized eyewear and is configured to automatically capture image data;

b. receiving, by a processor, a medicine regime associated with the user;

c. receiving, by a processor, data generated by the at least one sensor including image data automatically captured by the forward-facing camera of the computerized eyewear;

d. analyzing, by a processor, the received data generated by the at least one sensor to determine data that identifies;

i. types of medicine taken by the user based on image data automatically captured by the forward-facing camera that is a component of the computerized eyewear;

ii. times the medicine is taken by the user; and

iii. doses of medicine taken by the user based on image data automatically captured by the forward-facing camera that is a component of the computerized eyewear;

e. at least partially in response to analyzing the received data generated by the at least one sensor, comparing, by a processor, the medicine regime for the user to the determined types of medicine taken by the user based on image data automatically captured by the forward-facing camera that is a component of the computerized eyewear, times the medicine is taken by the user, and doses of medicine taken by the user based on image data automatically captured by the forward-facing camera that is a component of the computerized eyewear;

f. detecting, by a processor, one or more inconsistencies between the medicine regime associated with the user and the determined types of medicine taken by the user based on image data automatically captured by the forward-facing camera that is a component of the computerized eyewear, times the medicine is taken by the user, and doses of medicine taken by the user based on image data automatically captured by the forward-facing camera that is a component of the computerized eyewear;

g. at least partially in response to detecting one or more inconsistencies between the medicine regime associated with the and the determined types of medicine taken by the user based on image data automatically captured by the forward-facing camera that is a component of the computerized eyewear, the medicine is taken by the user, and doses of medicine taken by the user based on image data automatically captured by the forward-facing camera that is a component of the computerized eyewear, notifying, by a processor, at least one recipient of the detected inconsistencies, where the at least one recipient is selected from a group consisting of;

the user and the third party.

11. The computer-implemented method of claim 10, wherein the at least one sensor further comprises one or more sensors selected from a group consisting of:

a scanner, a motion sensor, and a microphone.

12. The computer-implemented method of claim 10, wherein:

a. the data generated is one or more images automatically captured by a forward facing camera;

b. the step of analyzing the received data further comprises;

i. detecting, by a processor, one or more pills in the one or more images;

ii. comparing, by a processor, the one or more detected pills found in the one or more images to one or more known images of pills stored in a database;

iii. identifying, by a processor, the one or more pills by matching the one or more pills from the one or more images to the one or more known images of pills stored in the database; and

iv. detecting, by a processor, a time that the one or more images were taken.

13. The computer-implemented method of claim 10, further comprising the step of:

a. detecting, by a processor, a level of one or more medicines in the user'"'"'s bloodstream by at least one sensor of the computerized eyewear that monitors the level of one or more medicines in the user'"'"'s bloodstream;

b. comparing, by a processor, the level of the one or medicines in the user'"'"'s bloodstream to a predefined level for each of the one or more medicines stored in a database for the user; and

c. at least partially in response to comparing the level of the one or medicines in the user'"'"'s bloodstream, notifying the at least one recipient when the level of the one or more medicines is below the predefined level for each of the one or more medicines, where the at least one recipient is selected from a group consisting of;

the user and a third party.

14. The computer-implemented method of claim 10, further comprising the steps of:

a. determining, by a processor, the user'"'"'s one or more movements using the received data generated by the at least one sensor;

b. analyzing, by a processor, the received data generated by the at least one sensor to determine the one or more movements associated with the user;

c. at least partially in response to analyzing the received data, comparing, by a processor, the user'"'"'s one or more movements to previously established one or more movement patterns for the user;

d. detecting, by a processor, one or more inconsistencies between the user'"'"'s one or more movements as compared to the previously established one or more movement patterns; and

e. at least partially in response to detecting one or more inconsistencies between the current user'"'"'s one or more movements as compared to previously established one or more movement patterns, notifying, by a processor, at least one recipient selected from a group consisting of;

the user or a third party of the detected one or more inconsistencies.

15. A computer-implemented method of monitoring the wellbeing of an individual, the method comprising the steps of:

a. providing a user with computerized eyewear comprising at least one sensor for monitoring actions taken by a user, the at least one sensor including;

i) a sound-capturing unit that is a component of the computerized eyewear and that is configured to capture sound data,

ii) a forward-facing camera that is a component of the computerized eyewear and that is configured to capture image data,

iii) a location-determination unit that is a component of the computerized eyewear and that is configured to capture location data;

b. receiving, by one or more processors, a medicine regime associated with the user, wherein the medicine regime is provided by;

i) at least one of a group of users consisting of;

(1) the user and (2) the user'"'"'s physician, stating a name for each medicine in the medicine regime such that the sound-capturing unit of the computerized eyewear captures sound data for the name of each medicine in the medicine regime; and

ii) the forward-facing camera of the computerized eyewear capturing image data of each medicine of the medicine regime and the medicine bottle for each medicine of the medicine regime;

c. storing, by one or more processors, sound data and image data for the provided medicine regime in a database;

d. receiving, by one or more processors, data generated by the at least one sensor including image data captured by the forward-facing camera of the computerized eyewear;

e. analyzing, by one or more processors, the received data generated by the at least one sensor to determine at least image data that identifies;

i) a first type of medicine taken by the user at a first geographic location, wherein the determination of the first type of medicine taken by the user is based on image data captured by the forward-facing camera and the determination of the first geographic location is based on location data captured by the location-determination unit;

ii) a first time that the first type of medicine was taken by the user at the first geographic location; and

iii) a first dose of the first type of medicine taken by the user at the first geographic location based on image data captured by the forward-facing camera that is a component of the computerized eyewear;

f. analyzing, by one or more processors, the received data generated by the at least one sensor to determine at least image data that identifies;

i) a second type of medicine taken by the user at a second geographic location, wherein

the determination of the second type of medicine taken by the user is based on image data captured by the forward-facing camera and the determination of the second geographic location is based on location data captured by the location-determination unit, and

the first type of medicine is different from the second type of medicine, and the first geographic location is different from the second geographic location

ii) a second time that the second type of medicine was taken by the user at the second geographic location; and

iii) a second dose of the second type of medicine taken by the user at the second geographic location based on image data captured by the forward-facing camera that is a component of the computerized eyewear;

g. at least partially in response to analyzing the received data generated by the at least one sensor, comparing, by one or more processors, the medicine regime for the user to;

i) the determined first type of medicine taken by the user at the first geographic location, the first time that the first type of medicine was taken, and the first dose of the first type of medicine that was taken, and

ii) the determined second type of medicine taken by the user at the second geographic location, the second time that the second type of medicine was taken, and the second dose of the second type of medicine that was taken;

h. detecting, by one or more processors, one or more inconsistencies between the medicine regime associated with the user and

i) the determined first type of medicine taken by the user at the first geographic location, the first time that the first type of medicine was taken, and the first dose of the first type of medicine that was taken, or

ii) the determined second type of medicine taken by the user at the second geographic location, the second time that the second type of medicine was taken, and the second dose of the second type of medicine that was taken;

i. at least partially in response to detecting the one more inconsistencies, notifying, by one or more processors, at least one recipient of the one or more inconsistencies, where the at least one recipient is a recipient selected from a group consisting of;

the user and a third party.

16. The computer implemented method of claim 15, wherein the steps of determining the first type of medicine taken by the user at the first geographic location and determining the second type of medicine taken by the user at the second geographic location further comprise:

a. detecting, by one or more processors, one or more pills from the image data;

b. comparing, by one or more processors, the one or more detected pills found from the image data to one or more known images of pills stored in the database; and

c. identifying, by one or more processors, the one or more pills by matching the one or more pills from the image data to the one or more known images of pills stored in the database.

17. The computer-implemented method of claim 15, further comprising the steps of:

a. detecting, by one or more processors, a level of one or more medicines in the user'"'"'s bloodstream by at least one sensor of the computerized eyewear that monitors the level of one or more medicines in the user'"'"'s bloodstream;

b. comparing, by one or more processors, the level of the one or medicines in the user'"'"'s bloodstream to a predefined level for each of the one or more medicines stored in a database for the user; and

c. at least partially in response to comparing the level of the one or medicines in the user'"'"'s bloodstream, notifying the at least one recipient when the level of the one or more medicines is below the predefined level for each of the one or more medicines, where the at least one recipient is selected from a group consisting of;

the user and a third party.

18. The computer-implemented method of claim 15, further comprising the steps of:

a. determining, by one or more processors, the user'"'"'s one or more movements using the received data generated by the at least one sensor;

b. analyzing, by one or more processors, the received data generated by the at least one sensor to determine the one or more movements associated with the user;

c. at least partially in response to analyzing the received data, comparing, by one or more processors, the user'"'"'s one or more movements to previously established one or more movement patterns for the user;

d. detecting, by one or more processors, one or more inconsistencies between the user'"'"'s one or more movements as compared to the previously established one or more movement patterns; and

e. at least partially in response to detecting one or more inconsistencies between the current user'"'"'s one or more movements as compared to previously established one or more movement patterns, notifying, by one or more processors, at least one recipient selected from a group consisting of;

the user or a third party of the detected one or more inconsistencies.