SYSTEMS AND METHODS FOR CONTROLLING A VEHICLE OR DEVICE IN RESPONSE TO A MEASURED HUMAN RESPONSE TO A PROVOCATIVE ENVIRONMENT

• US 20160167672A1
• Filed: 02/19/2016
• Published: 06/16/2016
• Est. Priority Date: 05/14/2010
• Status: Active Grant

• Current Assignee: Novutz LLC
• Original Assignee: Armour Holdings, Inc.
• Inventors: Krueger, Wesley W. O.
• Granted Patent: US 9,994,228 B2
• Time in Patent Office: Days
• CPC Class Codes: A61B 3/145 by video means
• CPC Class Codes: A61B 5/02055 Simultaneously evaluating both cardiovascular condition and temperature
• CPC Class Codes: A61B 5/021 Measuring pressure in heart or blood vessels A61B5/0205 takes precedence
• CPC Class Codes: A61B 5/0533 Measuring galvanic skin response
• CPC Class Codes: A61B 5/1114 Tracking parts of the body
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: A61B 5/318 Heart-related electrical modalities, e.g. electrocardiography [ECG]
• CPC Class Codes: A61B 5/369 Electroencephalography [EEG] devices for psychotechnics A61B5/16
• CPC Class Codes: A61B 5/4023 Evaluating sense of balance
• CPC Class Codes: A61B 5/4863 Measuring or inducing nystagmus
• CPC Class Codes: A61B 5/6803 Head-worn items, e.g. helmets, masks, headphones or goggles
• CPC Class Codes: A61B 5/7282 Event detection, e.g. detecting unique waveforms indicative of a medical condition A61B5/4094, A61B5/4818, A61B5/349, A61B5/0823 take precedence
• CPC Class Codes: A61B 5/7405 using sound
• CPC Class Codes: A61B 5/7425 Displaying combinations of multiple images regardless of image source, e.g. displaying a reference anatomical image with a live image geometric transforms for image registration G06T3/14; enhancement using image fusion G06T5/50; algorithms for registration of images G06T7/30
• CPC Class Codes: A61B 5/7445 Display arrangements, e.g. multiple display units
• CPC Class Codes: A61B 5/7455 characterised by tactile indication, e.g. vibration or electrical stimulation
• CPC Class Codes: A61B 5/746 Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms alarms per se G08B
• CPC Class Codes: A61M 2021/0022 by the tactile sense, e.g. vibrations
• CPC Class Codes: A61M 2021/0027 by the hearing sense
• CPC Class Codes: A61M 2021/0044 : by the sight sense
• CPC Class Codes: A61M 2021/0083 : especially for waking up
• CPC Class Codes: A61M 21/00 : Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis
• CPC Class Codes: A61M 2205/18 : with alarm
• CPC Class Codes: A61M 2205/3306 : Optical measuring means
• CPC Class Codes: A61M 2205/3317 : Electromagnetic, inductive or dielectric measuring means
• CPC Class Codes: A61M 2205/332 : Force measuring means
• CPC Class Codes: A61M 2205/3569 : sublocal, e.g. between console and disposable
• CPC Class Codes: A61M 2205/3592 : using telemetric means, e.g. radio or optical transmission
• CPC Class Codes: A61M 2205/507 : Head Mounted Displays [HMD]
• CPC Class Codes: A61M 2210/0612 : Eyes
• CPC Class Codes: A61M 2230/04 : Heartbeat characteristics, e.g. ECG, blood pressure modulation
• CPC Class Codes: A61M 2230/10 : Electroencephalographic signals
• CPC Class Codes: A61M 2230/20 : Blood composition characteristics
• CPC Class Codes: A61M 2230/201 : Glucose concentration
• CPC Class Codes: A61M 2230/205 : partial oxygen pressure (P-O2)
• CPC Class Codes: A61M 2230/207 : hematocrit
• CPC Class Codes: A61M 2230/208 : pH-value
• CPC Class Codes: A61M 2230/30 : Blood pressure A61M2230/04 takes precedence
• CPC Class Codes: A61M 2230/40 : Respiratory characteristics
• CPC Class Codes: A61M 2230/432 : partial CO2 pressure (P-CO2)
• CPC Class Codes: A61M 2230/435 : partial O2 pressure (P-O2)
• CPC Class Codes: A61M 2230/50 : Temperature
• CPC Class Codes: A61M 2230/60 : Muscle strain, i.e. measured on the user
• CPC Class Codes: A61M 2230/62 : Posture
• CPC Class Codes: A61M 2230/63 : Motion, e.g. physical activity
• CPC Class Codes: A61M 2230/65 : Impedance, e.g. conductivity, capacity
• CPC Class Codes: B60W 2040/0872 : Driver physiology
• CPC Class Codes: B60W 40/08 : related to drivers or passengers
• CPC Class Codes: G01C 23/00 : Combined instruments indicating more than one navigational value, e.g. for aircraft; Combined measuring devices for measuring two or more variables of movement, e.g. distance, speed or acceleration
• CPC Class Codes: G02B 2027/0138 : comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 : comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 : slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 : Head mounted
• CPC Class Codes: G06T 19/006 : Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 20/20 : in augmented reality scenes
• CPC Class Codes: G06V 40/19 : Sensors therefor
• CPC Class Codes: G16H 40/63 : for local operation
• CPC Class Codes: G16H 50/30 : for calculating health indices; for individual health risk assessment
• CPC Class Codes: H04N 13/344 : with head-mounted left-right displays
• CPC Class Codes: H04N 13/366 : using viewer tracking

Abstract

A sensor system and method for determining if an environment is provocative, measuring a human response to this environment, and controlling a vehicle or device is disclosed. The system and method comprise a human head-worn unit with an orientation sensing element, a head-attachable display comprising a first image not responsive to pitch and roll and a display element responsive to the orientation sensing element. The system and method also comprises at least one human response sensor such as a galvanic skin response sensor, a blood pressure sensor, or an electroencephalograph. The system and method further comprise an algorithm that computes a signal that is a measure of the condition of the human based on input from the orientation sensing element and the human response sensor or sensors. The system and method also comprise an electronic interface configured for transmitting the human physical condition signal.

Claims

1. A system comprising:

a unit attachable to a person'"'"'s head wherein the head-attachable unit further comprises;

an orientation sensing element responsive to pitch of the person'"'"'s head and roll of the person'"'"'s head wherein pitch represents a rotation about a first axis representing up and down movement of the person'"'"'s face while the rear of the person'"'"'s head moves in the opposite direction and roll represents rotation about a second axis perpendicular to the pitch axis in which the face rotates about the nose when looked at from the front;

a control circuit, wherein the control circuit produces an orientation sensor signal responsive to the orientation sensing element; and

a head-attachable display viewable by the person wherein the head-attachable display comprises;

a first image that is not responsive to pitch of the person'"'"'s head and not responsive to roll of the person'"'"'s head; and

a display element viewable by the person wherein the display element is responsive to the orientation sensor signal;

a first human response sensor wherein the first response sensor comprises a sensor selected from the group of a galvanic skin response sensor;

a blood pressure sensor, and an electroencephalograph;

a second human response sensor wherein the second response sensor comprises a video camera wherein the video camera is responsive to an eye parameter selected from the group of a palpebral movement, nystagmus, and a saccade;

a third human response sensor wherein the third response sensor is responsive to the person'"'"'s heart rate;

a physical response detection algorithm, wherein;

the algorithm is computed in a digital processor responsive to the orientation sensing element, the first human response sensor, the second human response sensor; and

the heart rate sensor;

the algorithm computes a human physical condition signal that is a measure of the condition of the person wherein the condition is selected from the group of dizziness, vertigo, motion sickness, motion intolerance, and spatial disorientation; and

an electronic interface configured for transmitting the human physical condition signal to an external system selected from the group of a vehicle and a device.

2. The system of claim 1 wherein:

the system comprises a fourth human response sensor wherein the fourth response sensor is responsive to a blood oxygen saturation of the person and the algorithm is responsive to the blood oxygen saturation; and

the display comprises a first display screen viewable by the person'"'"'s left eye and a second display screen viewable by the person'"'"'s right eye;

wherein;

the second image is viewable on the first display screen and the second display screen; and

the second image comprises a stereoscopic image.

3. The system of claim 1 wherein:

the head-attachable unit is a helmet;

the first image comprises a vertical line and a horizontal line;

the first human response sensor is a galvanic skin response sensor;

the system comprises a fourth human response sensor wherein the fourth response sensor is responsive to a blood oxygen saturation of the person and the algorithm is responsive to the blood oxygen saturation;

the system comprises a fifth human response sensor wherein the fifth response sensor is responsive to an expired respiratory blood oxygen level of the person and the algorithm is responsive to the expired respiratory oxygen level;

the system comprises a sixth human response sensor wherein the sixth response sensor is responsive to an expired respiratory carbon dioxide level of the person and the algorithm is responsive to the expired respiratory carbon dioxide level;

the system comprises a seventh human response sensor wherein the seventh response sensor is responsive to a respiratory rate of the person and the algorithm is responsive to the respiratory rate;

the system comprises an eighth human response sensor wherein the eighth response sensor is responsive to a blood glucose level of the person and the algorithm is responsive to the blood glucose level;

the system comprises a ninth human response sensor wherein the ninth response sensor is responsive to a blood sodium level of the person and the algorithm is responsive to the blood sodium level;

the system comprises a tenth human response sensor wherein the tenth response sensor is responsive to a blood hemoglobin level of the person and the algorithm is responsive to the blood hemoglobin level;

the system comprises an eleventh human response sensor wherein the eleventh response sensor is responsive to a blood alcohol level of the person and the algorithm is responsive to the blood alcohol level;

the system comprises a twelfth human response sensor wherein the twelfth response sensor is responsive to a blood lactate level of the person and the algorithm is responsive to the blood lactate level;

the system comprises a thirteenth human response sensor wherein the thirteenth response sensor is responsive to an electroencephalograph of the person and the algorithm is responsive to the electroencephalograph;

the system comprises a fourteenth human response sensor wherein the fourteenth response sensor is responsive to an electrocardiogram of the person and the algorithm is responsive to the electrocardiogram;

the system comprises a fifteenth human response sensor wherein the fifteenth response sensor is responsive to a body temperature of the person and the algorithm is responsive to the body temperature;

the system comprises a first environmental input sensor wherein the first environmental input sensor is responsive to an air pressure and the algorithm is responsive to the air pressure;

the system comprises a second environmental input sensor wherein the second environmental input sensor is responsive to an oxygen level and the algorithm is responsive to the oxygen level;

the electronic interface transmits the human physical condition to an aircraft; and

the system comprises an auditory alarm responsive to the algorithm.

4. A system comprising:

a unit attachable to a person'"'"'s head wherein the head-attachable unit comprises;

an orientation sensing element responsive to pitch of the head and roll of the head;

a control circuit, wherein the control circuit produces an orientation sensor signal responsive to the orientation sensing element; and

a head-attachable display viewable by the person wherein the head-attachable display comprises;

a first image that is not responsive to pitch of the head and not responsive to roll of the head; and

a display element viewable by the person wherein the display element is responsive to the orientation sensor signal;

a first human response sensor wherein the first response sensor comprises a sensor selected from the group of a galvanic skin response sensor;

a blood pressure sensor, and an electroencephalograph;

a physical response detection algorithm, wherein;

the algorithm computes a human physical condition signal in response to the orientation sensing element and the first response sensor; and

the human physical condition signal is a measure of the condition of the person wherein the condition is selected from the group of dizziness, vertigo, motion sickness, motion intolerance, and spatial disorientation; and

an electronic interface configured for transmitting the human physical condition signal.

5. The system of claim 4 wherein:

the first image comprises a first left image and a first right image;

the first left image is different from the first right image;

the first left image comprises a moving computer generated visual element viewable by the person'"'"'s left eye;

the first right image comprises a moving computer generated visual element viewable by the person'"'"'s right eye; and

the combination of the first left image and the first right image create a three-dimensional visual environment selected from the group of a virtual reality environment, an augmented reality environment, and a synthetic environment; and

the second image is responsive to the first human response sensor.

6. The system of claim 4 wherein the second image comprises a virtual gimbal.

7. The system of claim 4 wherein:

the system comprises a second human response sensor wherein the second response sensor comprises a video camera wherein the video camera is responsive to an ocular movement selected from the group of nystagmus and a saccade; and

the algorithm is responsive to the ocular movement.

8. The system of claim 7 wherein:

the second human response sensor is responsive to vection nystagmus detected as a temporal involuntary eye movement cause by self perceived motion sensation as a result of a physiological effect selected from the group of vertigo, motion sickness, motion intolerance, and spatial disorientation resulting from sensory mismatch between the person'"'"'s visual, vestibular, and proprioceptive organs.

9. The system of claim 7 wherein:

the second human response sensor is responsive to a saccade detected as a quick simultaneous movement of both eyes in the same direction indicative of a physiological effect selected from the group of sleepiness, vertigo, motion sickness, motion intolerance, and spatial disorientation.

10. The system of claim 4 wherein:

the system comprises a second human response sensor wherein the second response sensor comprises a video camera wherein the video camera is responsive to a palpebral movement; and

the algorithm is responsive to the palpebral movement.

11. The system of claim 4 wherein:

the first human response sensor comprises a galvanic skin response sensor; and

the algorithm is responsive to the galvanic skin response sensor.

12. The system of claim 4 wherein:

the first human response sensor comprises a blood pressure sensor; and

the algorithm is responsive to the blood pressure sensor.

13. The system of claim 4 wherein:

the first human response sensor comprises an electroencephalograph; and

the algorithm is responsive to the electroencephalograph.

14. The system of claim 4 wherein:

the system further comprises a heart rate sensor; and

the algorithm is responsive to the heart rate sensor.

15. The system of claim 4 wherein:

the display is opaque;

the display comprises a first display screen viewable by the person'"'"'s left eye and a second display screen viewable by the person'"'"'s right eye and a stereoscopic image;

the first human response sensor comprises a galvanic skin response sensor.

the system further comprises a second human response sensor wherein the second response sensor comprises a video camera wherein the video camera is responsive to an eye parameter selected from the group of a palpebral movement, nystagmus, and a saccade;

the system further comprises a heart rate sensor;

the system further comprises a respiratory rate sensor;

the system further comprises a blood alcohol sensor;

the system further comprises an electroencephalograph; and

the algorithm is responsive to the video camera, the heart rate sensor, the respiratory rate sensor, the blood alcohol sensor, and the electroencephalograph.

16. A method for calculating a healthiness value, the method comprising the steps of:

placing a unit on a person'"'"'s head wherein the unit comprises an orientation-sensing element and a display;

attaching a first human response sensor to the person wherein the first human response sensor is selected from the group of a galvanic skin response sensor, a blood pressure sensor, and an electroencephalograph;

measuring an orientation signal from the orientation sensing element wherein the orientation signal comprises a signal selected from the group of a head pitch and a head roll;

presenting a first image on the display wherein the first image is not responsive to head pitch or head roll;

presenting a second image on the display wherein the second image is responsive to the orientation signal;

measuring a human response from the first human response sensor; and

calculating the healthiness value in response to the orientation sensing element and the human response.

17. The method of claim 16 wherein:

the method comprises attaching a second human response sensor wherein the second human response sensor is responsive to a blood alcohol level of the person; and

measuring a human response further comprises a response to the blood alcohol level.

18. The method of claim 16 wherein:

the method comprises controlling an aircraft in response to the healthiness value.

19. The method of claim 16 wherein:

the method comprises establishing an eye-facing video camera on the unit;

the method comprises measuring an eye position using the eye-facing camera; and

measuring a human response further comprises a response to the eye position.

20. The method of claim 16 wherein:

the method comprises transmitting a haptic alarm to the person in response to the healthiness value.