Method of analyzing, displaying, organizing and responding to vital signals

• US 10,108,325 B2
• Filed: 12/09/2015
• Issued: 10/23/2018
• Est. Priority Date: 12/11/2014
• Status: Active Grant

• Current Assignee: RDI Technologies, Inc.
• Original Assignee: RDI Technologies, Inc.
• Inventors: Hay, Jeffrey R., Johns, Jenna L.
• Primary Examiner(s): Chu, Randolph I
• Application Number:
  US14/757,256
• Publication Number:
  US 20160210747A1
• Time in Patent Office: 1,049 Days
• Field of Search: None
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/01 Measuring temperature of body parts ; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue clinical contact thermometers G01K13/20
• CPC Class Codes: A61B 5/024 Detecting, measuring or recording pulse rate or heart rate A61B5/0205, A61B5/021 take precedence
• CPC Class Codes: A61B 5/0816 Measuring devices for examining respiratory frequency measuring frequency of electric signals G01R23/00
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/4815 Sleep quality
• CPC Class Codes: A61B 5/7435 Displaying user selection data, e.g. icons in a graphical user interface
• CPC Class Codes: G01N 2291/028 Material parameters
• CPC Class Codes: G01N 29/44 Processing the detected response signal , e.g. electronic circuits specially adapted therefor digital signal processing per se G06F17/00
• CPC Class Codes: G06F 16/7335 Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate end-user interface involving hot spots associated with the video H04N21/4725; end-user interface for selecting a Region of Interest H04N21/4728
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20056 Discrete and fast Fourier transform, [DFT, FFT]
• CPC Class Codes: G06T 2207/20216 Image averaging
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06T 7/0016 involving temporal comparison
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/262 : using transform domain methods, e.g. Fourier domain methods
• CPC Class Codes: G06V 20/52 : Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/20 : Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: H04N 7/18 : Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A system for monitoring vital signs includes: an imaging device for acquiring video image files of a living individual; a data analysis system including a processor and memory; a computer program running in the data analysis system to automatically analyze the video images, autonomously identify an area in the images where periodic movements associated with a selected vital sign may be detected and quantified; and, an interface that outputs an electrical signal corresponding to the waveform of the selected vital sign. The system may include a Graphical User Interface, which may display a visual graph of the waveform and a single video frame or a video stream of the individual.

Claims

1. A system for monitoring vital signs comprising:

an imaging device for acquiring video image files of a living individual, each video file having a sequence of video frames, which are stored in memory and divided into pixels defined by two or more spatial coordinates within the frame, wherein each pixel tracks an intensity value of either light intensity changes from periodic motions of the individual or intensity change in thermal IR from temperature changes of the individual or the individual'"'"'s motion;

a data analysis system including a processor and memory;

a computer program running in said data analysis system to analyze said video images,

wherein said computer program identifies at least one pixel having the same spatial coordinates in a first frame as in a second frame, said at least one pixel indicating periodic motions or temperature changes of the individual, and said computer program detects a number of frames in the sequence separating a first occurrence of maximum or minimum amplitude in said at least one pixel from a next occurrence in the sequence of video frames of maximum or minimum amplitude in said at least one pixel, the number of frames representing a frame spacing value,

and said computer program then locates said at least one pixel having the same spatial coordinates in a plurality of frames separated by the frame spacing value, and produces a waveform of the periodic motion or temperature change of the individual that is based on said intensity value of said at least one pixel detected in each of the plurality of frames.

2. The system of claim 1 wherein said imaging device is selected from the group consisting of:

video cameras, optical sensors, IR sensors, smart phones, webcams, and digital microscopes.

3. The system of claim 1 wherein said computer program identifies the at least one pixel of interest by basing the frame spacing value on a rate of the vital signal for the individual stored in the memory.

4. The system of claim 1 further comprising an interface that outputs a digital signal corresponding to said waveform.

5. The system of claim 1 further comprising an interface that outputs an analog voltage corresponding to said waveform.

6. The system of claim 1 further comprising an interface that outputs an electrical signal corresponding to said waveform.

7. The system of claim 1 further comprising a Graphical User Interface (GUI) that displays data corresponding to a vital sign.

8. The system of claim 1 comprising both a video camera and IR sensor for the imaging device, wherein the video camera senses changes in a corresponding pixel based on light intensity changes associated with periodic movement of the chest, and the IR sensor senses temperature changes in a corresponding pixel associated with inhaling and exhaling air through the nose or mouth.

9. The system of claim 7 wherein said GUI further displays an image from said video file.

10. The system of claim 9 wherein said computer program receives a selection from a user for a particular point in time and said GUI displays the value of said vital sign and the video frame corresponding to said selected time.

11. The system of claim 9 wherein said GUI allows a user to replay data starting at a selected time so that said user may simultaneously view the video stream and the corresponding vital sign simultaneously.

12. The system of claim 1 wherein said computer program searches for an area of pixels within a user-defined perimeter within the video frame and said data analysis system monitors either light intensity changes from periodic motions of the individual or intensity change in thermal IR from temperature changes of the individual within the area of pixels within said user-defined perimeter.

13. The system of claim 12 wherein said computer program searches for an area of pixels in each of two discrete user-defined perimeters within the video frame, each with a separate perimeter, and said data analysis system monitors for said light intensity changes or temperature changes within each respective perimeter independently.

14. The system of claim 1 wherein said vital sign is selected from the group consisting of:

respiration rate;

respiration effort;

respiration waveform;

pulse rate;

pulse waveform;

pulse transit time;

temperature;

sleep quality index; and

state of stress.

19. The system of claim 7 wherein said GUI allows a user to select a particular time interval and said GUI displays the value of said vital sign and the video image(s) corresponding to said selected time interval.

20. The system of claim 1 wherein said computer program detects changes in the waveform indicative of vital sign irregularity and selectively stores the waveform and video data for a selected period of time before and after the occurrence of the irregularity.

21. The system of claim 20 wherein said computer program generates a notification of the irregularity.

15. A system for monitoring vital signs comprising:

an imaging device for acquiring video image files of a living individual, each video image file having a sequence of video frames, which are stored in memory and divided into pixels defined by two or more spatial coordinates within the frame, wherein each pixel tracks intensity changes expressed in amplitude which are associated with periodic motions or temperature changes of the individual;

a data analysis system including a processor and memory;

a computer program running in said data analysis system to analyze said video images, autonomously identify an area in the images comprising a set of pixels where periodic variations in intensity associated with a selected vital sign are detected and quantified, wherein said computer program detects said periodic movements or temperature changes by an array comparison procedure in which;

starting with a first frame [F_m], the computer program subtracts the intensity of one or more pixels in the frame from the intensity of one or more pixels from the set having the same spatial coordinates in frame [F_m+n], where n is an integer;

the computer program subtracts the intensity of said one or more pixels in frame [F_m+n] from the intensity in frame [F_m+2n]; and

the computer program subtracts the intensity of said one or more pixels in frame [F_m+2n] from the intensity in frame [F_m+3n], . . . until reaching a selected end point at frame [F_m+nMax], where n is an integer representing a frame spacing value for locating consecutive peaks in said waveform, and nMax is less than the total number of frames in said video file;

the computer program then sums the resulting frame differences to yield the maximum total change for each of said one or more pixels; and

,

wherein computer program produces a waveform of the periodic variation based on the intensities of said one or more pixels having the greatest frame-to-frame intensity difference, and the computer program determines the rate of said periodic variations by determining a number of frames in the sequence that separates a first occurrence of maximum or minimum amplitude in said one or more pixels from a next occurrence in the sequence of video frames of maximum or minimum amplitude in one or more pixels having the same spatial coordinates, the number of frames providing said frame spacing value.

16. The system of claim 15 wherein said computer program aligns the frame spacing value to track corresponding pixels only from video frames exhibiting a maximum or minimum amplitude of the waveform.

17. The system of claim 15 wherein said interface produces an output type selected from the group consisting of:

an analog voltage corresponding to said waveform;

an analog current corresponding to said waveform;

a digital signal corresponding to said waveform; and

a visual display of said waveform on a Graphical User Interface (GUI).

18. The system of claim 17 wherein said GUI further displays an image from said video file.

22. A method for monitoring vital signs of a living individual comprising:

acquiring video image files of a living individual, each video file having a sequence of video frames, which are stored in memory and divided into pixels defined by two or more spatial coordinates within the frame, wherein each pixel tracks an intensity value of either light intensity changes from periodic motions of the individual or intensity change in thermal IR from temperature changes of the individual or the individual'"'"'s motion; and

configuring a processor to execute a computer program to;

detect a number of frames in the sequence that separates a first occurrence of maximum or minimum amplitude in a pixel from a next occurrence of maximum or minimum amplitude in a corresponding pixel having the same spatial coordinates in a first frame as in a second frame, wherein the number of frames represents a frame spacing value;

locate pixels indicating periodic motion or temperature changes of the individual based on the frame spacing value;

produce a waveform of the periodic motion or temperature changes of the individual based on the intensity of said located pixels; and

adjust a thermostat in communication with said processor to change the temperature of the ambient environment around the individual.

23. The method of claim 22, wherein adjusting the thermostat is based on temperature changes of the individual.

24. The method of claim 22, wherein adjusting the thermostat is based on irregularity in the periodic motion of the individual.

25. The method of claim 22, further comprising displaying the waveform corresponding to a vital sign of the individual.

26. The method of claim 25, further comprising displaying an image of the individual from said video file.

27. The method of claim 26, wherein the image of the individual is displayed simultaneously with a waveform corresponding to a vital sign of the individual.