WEARABLE MENTAL STATE MONITOR COMPUTER APPARATUS, SYSTEMS, AND RELATED METHODS

• US 20160066829A1
• Filed: 12/19/2014
• Published: 03/10/2016
• Est. Priority Date: 09/05/2014
• Status: Abandoned Application

• Current Assignee: Vision Service Plan (VSP Global, Inc.)
• Original Assignee: Vision Service Plan (VSP Global, Inc.)
• Inventors: Sales, Jay William, Klosinski, Richard Chester Jr., Workman, Matthew Allen, Murphy, Meghan Kathleen, Steen, Matthew David
• Application Number:
  US14/578,039
• Publication Number:
  US 20160066829A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A61B 2560/0475 Special features of memory means, e.g. removable memory cards
• CPC Class Codes: A61B 2562/0219 Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
• CPC Class Codes: A61B 2562/0223 Magnetic field sensors
• CPC Class Codes: A61B 2562/0257 Proximity sensors
• CPC Class Codes: A61B 2576/00 Medical imaging apparatus involving image processing or analysis A61B1/00009, A61B6/52 and A61B8/52 take precedence; image processing per se G06T; details of algorithms for analysing medical images G06T7/0012
• CPC Class Codes: A61B 3/112 for measuring diameter of pupils
• CPC Class Codes: A61B 5/0002 Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network A61B5/07 takes precedence; transmission systems for measured values G08C; transmission H04B; transmission of digital information H04L; wireless communication networks H04W
• CPC Class Codes: A61B 5/0022 Monitoring a patient using a global network, e.g. telephone networks, internet
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/0205 Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
• CPC Class Codes: A61B 5/024 Detecting, measuring or recording pulse rate or heart rate A61B5/0205, A61B5/021 take precedence
• CPC Class Codes: A61B 5/0531 Measuring skin impedance
• CPC Class Codes: A61B 5/0816 Measuring devices for examining respiratory frequency measuring frequency of electric signals G01R23/00
• CPC Class Codes: A61B 5/1032 Determining colour for diagnostic purposes measuring colour in general G01J3/46
• CPC Class Codes: A61B 5/1103 Detecting eye twinkling
• CPC Class Codes: A61B 5/1114 Tracking parts of the body
• CPC Class Codes: A61B 5/1116 Determining posture transitions
• CPC Class Codes: A61B 5/1118 Determining activity level
• CPC Class Codes: A61B 5/112 Gait analysis
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A61B 5/117 : Identification of persons methods or arrangements for recognising patterns, e.g. fingerprints, G06F18/00, G06V40/00; identification of persons by analysing their voice or speech G10L17/00
• CPC Class Codes: A61B 5/1176 : Recognition of faces
• CPC Class Codes: A61B 5/14552 : Details of sensors specially adapted therefor
• CPC Class Codes: A61B 5/165 : Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/318 : Heart-related electrical modalities, e.g. electrocardiography [ECG]
• CPC Class Codes: A61B 5/369 : Electroencephalography [EEG] devices for psychotechnics A61B5/16
• CPC Class Codes: A61B 5/4076 : Diagnosing or monitoring particular conditions of the nervous system A61B5/4821, A61B5/4824 take precedence
• CPC Class Codes: A61B 5/4266 : sweat secretion
• CPC Class Codes: A61B 5/443 : Evaluating skin constituents, e.g. elastin, melanin, water
• CPC Class Codes: A61B 5/486 : Bio-feedback A61B5/375 takes precedence
• CPC Class Codes: A61B 5/4884 : inducing physiological or psychological stress, e.g. applications for stress testing
• CPC Class Codes: A61B 5/6803 : Head-worn items, e.g. helmets, masks, headphones or goggles
• CPC Class Codes: A61B 5/7246 : using correlation, e.g. template matching or determination of similarity
• CPC Class Codes: A61B 5/7278 : Artificial waveform generation or derivation, e.g. synthesising signals from measured signals A61B5/327 takes precedence
• CPC Class Codes: A61B 5/7282 : Event detection, e.g. detecting unique waveforms indicative of a medical condition A61B5/4094, A61B5/4818, A61B5/349, A61B5/0823 take precedence
• CPC Class Codes: A61B 7/04 : Electric stethoscopes microphones, acoustic transducers therefor H04R
• CPC Class Codes: A61F 2/76 : Means for assembling, fitting or testing prostheses, e.g. for measuring or balancing, e.g. alignment means
• CPC Class Codes: A61F 2002/7695 : Means for testing non-implantable prostheses for testing implantable prostheses A61F2240/008; for implanting artificial joints A61F2/468
• CPC Class Codes: A63B 24/0062 : Monitoring athletic performances, e.g. for determining the work of a user on an exercise apparatus, the completed jogging or cycling distance
• CPC Class Codes: G02C 11/10 : Electronic devices other than hearing aids
• CPC Class Codes: G06F 18/22 : Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 21/35 : communicating wirelessly
• CPC Class Codes: G06F 2218/00 : Aspects of pattern recognition specially adapted for signal processing
• CPC Class Codes: G06V 20/10 : Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 40/18 : Eye characteristics, e.g. of the iris
• CPC Class Codes: G06V 40/19 : Sensors therefor
• CPC Class Codes: G06V 40/197 : Matching; Classification
• CPC Class Codes: G06V 40/25 : Recognition of walking or running movements, e.g. gait recognition
• CPC Class Codes: G07C 9/37 : using biometric data, e.g. fingerprints, iris scans or voice recognition
• CPC Class Codes: G08B 21/02 : Alarms for ensuring the safety of persons
• CPC Class Codes: G08B 21/0423 : detecting deviation from an expected pattern of behaviour or schedule
• CPC Class Codes: G08B 21/0461 : integrated or attached to an item closely associated with the person but not worn by the person, e.g. chair, walking stick, bed sensor
• CPC Class Codes: G08B 21/0476 : Cameras to detect unsafe condition, e.g. video cameras
• CPC Class Codes: G09B 19/0092 : Nutrition
• CPC Class Codes: G09B 5/00 : Electrically-operated educational appliances working with questions and answers G09B7/00; simulators G09B9/00; advertising or displaying in general G09F
• CPC Class Codes: G09B 5/06 : with both visual and audible presentation of the material to be studied
• CPC Class Codes: G16H 20/40 : relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
• CPC Class Codes: G16H 40/63 : for local operation
• CPC Class Codes: G16H 40/67 : for remote operation
• CPC Class Codes: G16H 50/20 : for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: H04L 63/0861 : using biometrical features, e.g. fingerprint, retina-scan cryptographic mechanisms or cryptographic arrangements for entity authentication using biological data H04L9/3231

Abstract

A computer-implemented method of assessing the metal state of an individual by: (1) providing the individual with a wearable device (e.g., eyewear) that includes one or more sensors for assessing the mental state of the individual, (2) using information from one or more of the sensors to assess the mental state of the individual; and (3) informing the individual or a third party of the individual'"'"'s mental state. In various embodiments, the method further involves using the wearable device to determine one or more environmental factors that are related to the individual'"'"'s mental state. For example, the method may involve determining (e.g., from one or more images taken using the wearable device) that the individual is frequently in a stressed emotional state when a particular person is present, when the individual is engaged in a particular activity, and/or when the wearer experiences a certain internal or external context.

Claims

1. A computer-implemented method of assessing the mental state of a wearer of eyewear comprising one or more sensors coupled to the eyewear, the one or more sensors being adapted to detect one or more characteristics of the wearer of the eyewear, wherein the one or more characteristics are associated with the wearer'"'"'s mental state, the method comprising:

a. receiving, by a processor, one or more signals from the one or more sensors, wherein each of the one or more signals relates to at least one characteristic associated with the wearer, the at least one characteristic being selected from a group consisting of;

i. pupil size,

ii. heart rate,

iii. perspiration rate,

iv. respiration rate,

v. physical movement, and

vi. brainwave activity;

b. analyzing, by a processor, the one or more received signals to determine the at least one characteristic;

c. facilitating, by a processor, determination of a mental state of the wearer based, at least in part, on the at least one characteristic; and

d. associating, by a processor, the mental state of the wearer with at least one of one or more stimuli selected from a group consisting of;

(1) an object;

(2) an activity;

(3) an internal context associated with the wearer; and

(4) an external context associated with the wearer.

2. The computer-implemented method of claim 1, further comprising the step of, providing a wireless transmitter coupled to the one or more sensors, the wireless transmitter being configured to transmit the one or more signals to a handheld computing device, wherein the step of receiving one or more signals from the one or more sensors further comprises, receiving the one or more signals from the handheld computing device via the Internet.

3. The computer-implemented method of claim 1, wherein the one or more sensors comprises at least one sensor selected from a group consisting of:

a. an eye facing camera;

b. a forward facing camera;

c. an electrocardiogram sensor;

d. a heart rate monitor;

e. a microphone;

f. a skin conductance response sensor;

g. an accelerometer; and

h. a gyroscope.

4. The computer-implemented method of claim 3, wherein the eyewear comprises an eye facing camera, a forward facing camera, and a heart rate monitor.

5. The computer-implemented method of claim 3, further comprising the step of:

a. determining, by a processor, that the wearer of the eyewear is under emotional stress;

b. in response to determining that the wearer of the eyewear is under emotional stress, capturing, by a processor, an image by the forward facing camera;

c. receiving, by a processor, the image;

d. identifying an object from within the received image; and

e. facilitating notification of the wearer of the association of the wearer'"'"'s emotionally stressed state and the object in the image.

6. The computer-implemented method of claim 5, wherein the object is a particular individual.

7. The computer-implemented method of claim 6, wherein the step of identifying the particular individual comprises using one or more face recognition techniques to determine that a human is present within the image.

8. The computer-implemented method of claim 6, wherein the step of identifying the particular individual comprises using one or more face recognition techniques to determine the identity of the particular individual.

9. The computer-implemented method of claim 5, wherein the step of determining that the wearer of the eyewear is under emotional stress further comprises:

a. receiving, by a processor, an image of the wearer'"'"'s pupil from the eye facing camera;

b. determining, by a processor, from the received image the size of the wearer'"'"'s pupil;

c. comparing, by a processor, the size of the wearer'"'"'s pupil to a predefined measurement of the wearer'"'"'s pupil; and

d. at least partially in response to determining that the wearer'"'"'s pupil size exceeds the predefined measurement of the wearer'"'"'s pupil determining, by a processor, that the wearer is in a state of emotional stress.

10. The computer-implemented method of claim 5, wherein the step of determining that the wearer of the eyewear is under emotional stress further comprises:

a. determining, by a processor, the respiration rate of the wearer from the received one or more signals;

b. comparing the determined respiration rate to a predefined respiration rate level; and

c. in response to determining that the respiration rate exceeds the predefined respiration rate level, determining, by a processor, that the wearer is under emotional stress.

11. The computer-implemented method of claim 3, further comprising the step of:

a. receiving, by a processor, an image captured by the eyewear;

b. after receiving the image, identifying, by a processor, an object in the received image; and

c. notifying, by a processor, the wearer of the association of the wearer'"'"'s mental state and the object in the received image.

12. The computer-implemented method of claim 11, wherein the object is a person.

13. The computer-implemented method of claim 3, further comprising the steps of:

a. notifying, by a processor, the wearer of the eyewear of their mental state; and

b. providing, by a processor, to the wearer one or more suggested actions to address their current mental state.

14. The computer-implemented method of claim 13, wherein:

a. the wearer'"'"'s mental state is one of emotional stress; and

b. the one or more suggested actions comprise one or more techniques for reducing stress.

15. A computer-implemented method of assessing the mental state of a wearer of eyewear comprising one or more sensors coupled to the eyewear, the one or more sensors being adapted to detect one or more characteristics of the wearer of the eyewear, wherein the one or more characteristics are associated with the wearer'"'"'s mental state, the method comprising:

a. receiving, by a processor, one or more first images from the eye facing camera, wherein at least one of the one or more first images relates to at least one characteristic associated with the wearer;

b. analyzing, by a processor, the at least one or more first images to determine the at least one characteristic; and

c. facilitating, by a processor, determination of a mental state of the wearer based on the at least one characteristic.

16. The computer-implemented method of claim 15, further comprising the steps of:

a. receiving, by a processor, one or more second images from the front facing camera;

b. determining, by a processor, the mental state of wearer based at least in part on one or more wearer characteristics selected from a group consisting of;

i. the wearer'"'"'s pupil size,

ii. the wearer'"'"'s respiration rate,

iii. the wearer'"'"'s perspiration rate,

iv. the paleness of the wearer'"'"'s skin, and

v. the wearer'"'"'s brain activity level;

c. associating, by a processor, the mental state of the wearer with a factor selected from a group consisting of;

i. an object in close proximity to the wearer; and

ii. an activity in which the wearer is participating in.

17. The computer-implemented method of claim 16, wherein the step of associating the mental state of the wearer with the factor further comprises the steps of:

a. detecting, by a processor, the object in the one or more second images; and

b. storing, in memory, the association of the wearer'"'"'s mental state with the object.

18. The computer-implemented method of claim 16, wherein the step of associating the mental state of the wearer with the factor further comprises the steps of:

a. using the one or more second images to determine an activity that the wearer was engaged in when the one or more images were taken; and

b. storing, in memory, the association of the wearer'"'"'s mental state with the activity.

19. The computer-implemented method of claim 16, wherein the eyewear further comprises a near-field communication sensor that is configured to detect the presence of the at least one of the factor, the method further comprising the steps of:

a. receiving, by a processor, a signal from the near-field communication sensor;

b. determining, by a processor, the identity of the factor from the received signal; and

c. storing, by a processor, the identity of the factor and the mental state of the wearer in memory.

20. The computer-implemented method of claim 16, wherein the step of determining the mental state of the wearer further comprises:

a. measuring, by a processor, a pupil size of the wearer;

b. measuring, by a processor, a respiration rate level of the wearer;

c. comparing, by a processor,

i. the pupil size of the wearer to a predefined pupil size; and

ii. comparing the respiration rate of the wearer to a predefined respiration rate; and

d. determining, by a processor, that the wearer is in a stressful state in response to determining that the measured pupil size exceeds the predefined pupil size and the measured respiration rate exceeds the predefined respiration rate.

21. A system for assessing the mental state of a wearer of a wearable device comprising:

a. at least one processor; and

b. memory operatively coupled to the at least one processor,

wherein the at least one processor is configured to;

i. receive one or more signals from a device selected from a group consisting of;

a helmet and eyewear, the device having one or more sensors embedded therein, wherein each of the one or more signals relates to at least one characteristic associated with the wearer, the at least one characteristic selected from a group consisting of;

1. pupil size,

2. heart rate,

3. perspiration rate,

4. respiration rate,

5. movement, and

6. brainwave activity;

ii. analyze the one or more received signals to determine the at least one characteristic;

iii. determine a mental state of the wearer based at least in part on the at least one characteristic;

iv. associate the mental state of the wearer with one or more stimuli selected from a group consisting of;

(1) an object;

(2) an activity;

(3) an internal context of the wearer; and

(4) an external context of the wearer.

22. The system of claim 22, wherein the at least one processor is further configured to monitor at least one of the wearer'"'"'s pupil size and brainwave activity over time to determine whether the wearer has sustained a concussion.