System and method for detecting physiological state

• US 10,694,988 B2
• Filed: 02/17/2017
• Issued: 06/30/2020
• Est. Priority Date: 02/17/2016
• Status: Active Grant

• Current Assignee: NuraLogix Corporation
• Original Assignee: NuraLogix Corporation
• Inventors: Lee, Kang, Zheng, Pu
• Primary Examiner(s): Patel, Mahendra R
• Application Number:
  US16/076,522
• Publication Number:
  US 20190046099A1
• Time in Patent Office: 1,229 Days
• Field of Search: 382128, 348 77
• CPC Class Codes: A61B 5/0035 adapted for acquisition of images from more than one imaging mode, e.g. combining MRI and optical tomography
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/024 Detecting, measuring or recording pulse rate or heart rate A61B5/0205, A61B5/021 take precedence
• CPC Class Codes: A61B 5/02438 with portable devices, e.g. worn by the patient
• CPC Class Codes: A61B 5/0261 using optical means, e.g. infrared light
• CPC Class Codes: A61B 5/0263 using NMR
• CPC Class Codes: A61B 5/055 involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
• CPC Class Codes: A61B 5/145 Measuring characteristics of blood in vivo, e.g. gas concentration, pH value; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid, cerebral tissuemeasuring of blood pressure or blood flow A61B5/02; non-radiation detecting or locating of foreign bodies in blood A61B5/06
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/369 Electroencephalography [EEG] devices for psychotechnics A61B5/16
• CPC Class Codes: A61B 5/7207 of noise induced by motion artifacts
• CPC Class Codes: A61B 5/7257 using Fourier transforms
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30096 Tumor; Lesion
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/0014 using an image reference approach

Abstract

A system and method for health diagnostics and more specifically to an image-capture based system and method for detecting physiological state for a subject. The system provides a remote and non-invasive approach by which to detect physiological state with a high confidence. The system enables monitoring of hemoglobin concentration changes by optical imaging and related detection systems.

Claims

1. A system for detecting physiological states from a captured image sequence of a subject, the system comprising:

a camera configured to capture an image sequence of the subject, the image sequence comprising a query set of images;

a processing unit trained to determine a set of bitplanes of a plurality of images in the captured image sequence that represent hemoglobin concentration (HC) changes of the subject and that maximize signal differentiation between different physiological states, wherein the set of bitplanes that represent HC changes are determined for a selected plurality of regions of interest (ROI) of the subject that are relevant as an indicator of physiological state;

a classification machine, trained using a training set comprising HC changes of subjects with known physiological states, and configured to;

detect the subject'"'"'s physiological states based on HC changes in the set of bitplanes; and

output the detected physiological states,

wherein the processing unit is further configured to extract from the HC changes of one or more ROI a heart beat signal of the subject, process the heat beat signal to determine heart beat interval time series data of the subject, perform Fourier transforms on the heart beat interval time series data to obtain a stress level index, compare the stress level index with a predetermined normative stress index distribution profile to determine the subject'"'"'s comparative stress level, and provide the subject'"'"'s comparative stress level to the classification machine for use in detecting the subject'"'"'s physiological states.

2. The system of claim 1, wherein determining the set of bitplanes that maximize differentiation between different physiological states comprises selecting, using a machine learning technique, an RGB pixel bit-combination that maximizes a heart beat signal-to-noise ratio.

3. The system of claim 1, wherein detecting the subject'"'"'s physiological states based on HC changes comprises calculating an estimated statistical probability that the subject'"'"'s physiological state conforms to a known physiological state from the training set, and a normalized intensity measure of such determined physiological state.

4. The system of claim 3, wherein outputting the subject'"'"'s physiological states comprises grouping each of the detected physiological states into one of a plurality of groupings based on the calculated statistical probability and normalized intensity measure.

5. The system of claim 1, wherein the physiological states comprise stress levels, pain levels or fatigue levels.

6. The system of claim 1, wherein the camera comprises an optical sensor directly attached to the skin of the subject, and the processing unit is further configured to remove artifacts from motion and heat from the captured image sequence.

7. The system of claim 1, wherein the processing unit is further configured to process the captured image sequence to detect skin lesions that would be difficult to spot visually.

8. The system of claim 1, further comprising one of a magnetic resonance imaging unit, a near-infrared spectroscopy imaging unit or an electroencephalography imaging unit for capturing a second image sequence, and the processing unit is further configured to determine the subject'"'"'s transdermal blood flow changes from the second image sequence.

9. A method for detecting physiological states from a captured image sequence of a subject, the method comprising:

capturing, by a camera, an image sequence of the subject, the image sequence comprising a query set of images;

processing the captured image sequence, by a trained processing unit, to determine a set of bitplanes of a plurality of images in the captured image sequence that represent hemoglobin concentration (HC) changes of the subject and that maximize signal differentiation between different physiological states, the set of bitplanes that represent HC changes are determined for a selected plurality of regions of interest (ROI) of the subject that are relevant as an indicator of physiological state;

extracting, by the trained processing unit, from the HC changes of one or more ROI a heart beat signal of the subject;

processing, by the trained processing unit, the heat beat signal to determine heart beat interval time series data of the subject;

performing, by the trained processing unit, Fourier transforms on the heart beat interval time series data to obtain a stress level index;

comparing, by the trained processing unit, the stress level index with a predetermined normative stress index distribution profile to further determine the subject'"'"'s comparative stress level;

processing the set of bitplanes, by a classification machine trained using a training set comprising HC changes of subjects with known physiological states, to;

detect the subject'"'"'s physiological states based on HC changes in the set of bitplanes and the subject'"'"'s comparative stress level; and

output the detected physiological states.

10. The method of claim 9, wherein determining the set of bitplanes that maximize differentiation between different physiological states comprises selecting, using a machine learning technique, an RGB pixel bit-combination that maximizes a heart beat signal-to-noise ratio.

11. The method of claim 9, wherein detecting the subject'"'"'s physiological states based on HC changes comprises calculating an estimated statistical probability that the subject'"'"'s physiological state conforms to a known physiological state from the training set, and a normalized intensity measure of such determined physiological state.

12. The method of claim 11, wherein outputting the subject'"'"'s physiological states comprises grouping each of the detected physiological states into one of a plurality of groupings based on the calculated statistical probability and normalized intensity measure.

13. The method of claim 9, wherein the physiological states comprise stress levels, pain levels or fatigue levels.

14. The method of claim 9, wherein the camera comprises an optical sensor directly attached to the skin of the subject, and the method further comprises removing artifacts from motion and heat from the captured image sequence.

15. The method of claim 9, further comprising processing the captured image sequence to detect skin lesions that would be difficult to spot visually.

16. The method of claim 9, further comprising capturing a second image sequence, by one of a magnetic resonance imaging unit, a near-infrared spectroscopy imaging unit or an electroencephalography imaging unit, and processing the second image sequence to determine the subject'"'"'s transdermal blood flow changes from the second image sequence.