Non-touch optical detection of vital signs from amplified visual variations of reduced images of skin

• US 9,501,824 B2
• Filed: 08/11/2014
• Issued: 11/22/2016
• Est. Priority Date: 07/04/2014
• Status: Active Grant

• Current Assignee: ARC Devices, Ltd.
• Original Assignee: Arc Devices (Ni) Limited
• Inventors: Khachaturian, Mark, Smith, Michael G., Gross, Irwin, Gerst, Steven
• Primary Examiner(s): GORADIA, SHEFALI DINESH
• Application Number:
  US14/457,074
• Publication Number:
  US 20160035084A1
• Time in Patent Office: 834 Days
• US Class Current: 1/1
• CPC Class Codes: A61B 2560/0214 of power generation or supply
• CPC Class Codes: A61B 2560/0475 Special features of memory means, e.g. removable memory cards
• CPC Class Codes: A61B 2576/00 Medical imaging apparatus involving image processing or analysis A61B1/00009, A61B6/52 and A61B8/52 take precedence; image processing per se G06T; details of algorithms for analysing medical images G06T7/0012
• CPC Class Codes: A61B 5/0008 Temperature signals
• CPC Class Codes: A61B 5/0013 Medical image data A61B1/00011, A61B6/56, A61B8/56 take precedence
• CPC Class Codes: A61B 5/0059 using light, e.g. diagnosis by transillumination, diascopy, fluorescence A61B5/0093, A61B5/1455, A61B5/02416 and A61B5/0261 take precedence
• CPC Class Codes: A61B 5/0064 Body surface scanning
• CPC Class Codes: A61B 5/0075 by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy A61B5/0071 takes precedence
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/01 Measuring temperature of body parts ; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue clinical contact thermometers G01K13/20
• CPC Class Codes: A61B 5/015 By temperature mapping of body part
• CPC Class Codes: A61B 5/02055 Simultaneously evaluating both cardiovascular condition and temperature
• CPC Class Codes: A61B 5/021 Measuring pressure in heart or blood vessels A61B5/0205 takes precedence
• CPC Class Codes: A61B 5/024 Detecting, measuring or recording pulse rate or heart rate A61B5/0205, A61B5/021 take precedence
• CPC Class Codes: A61B 5/02416 using photoplethysmograph signals, e.g. generated by infrared radiation A61B5/14552 takes precedence
• CPC Class Codes: A61B 5/02433 for infrared radiation
• CPC Class Codes: A61B 5/0261 using optical means, e.g. infrared light
• CPC Class Codes: A61B 5/0816 Measuring devices for examining respiratory frequency measuring frequency of electric signals G01R23/00
• CPC Class Codes: A61B 5/1455 using optical sensors, e.g. spectral photometrical oximeters
• CPC Class Codes: A61B 5/14551 for measuring blood gases
• CPC Class Codes: A61B 5/14552 : Details of sensors specially adapted therefor
• CPC Class Codes: A61B 5/33 : specially adapted for cooperation with other devices
• CPC Class Codes: A61B 5/441 : Skin evaluation, e.g. for skin disorder diagnosis
• CPC Class Codes: A61B 5/6898 : Portable consumer electronic devices, e.g. music players, telephones, tablet computers
• CPC Class Codes: A61B 5/7225 : Details of analog processing, e.g. isolation amplifier, gain or sensitivity adjustment, filtering, baseline or drift compensation A61B5/30, A61B5/316 take precedence; transformers for specific applications H01F38/00; amplifiers H03F
• CPC Class Codes: A61B 5/725 : using specific filters therefor, e.g. Kalman or adaptive filters A61B5/316 take precedence
• CPC Class Codes: A61B 5/7257 : using Fourier transforms
• CPC Class Codes: A61B 5/7264 : Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems neural networks per se G06N3/00; expert systems per se G06N5/00
• CPC Class Codes: A61B 5/7267 : involving training the classification device
• CPC Class Codes: A61B 5/7275 : Determining trends in physiological measurement data; Predicting development of a medical condition based on physiological measurements, e.g. determining a risk factor
• CPC Class Codes: A61B 5/7278 : Artificial waveform generation or derivation, e.g. synthesising signals from measured signals A61B5/327 takes precedence
• CPC Class Codes: A61B 5/742 : using visual displays A61B5/339 takes precedence
• CPC Class Codes: A61B 5/7425 : Displaying combinations of multiple images regardless of image source, e.g. displaying a reference anatomical image with a live image geometric transforms for image registration G06T3/14; enhancement using image fusion G06T5/50; algorithms for registration of images G06T7/30
• CPC Class Codes: A61B 5/743 : Displaying an image simultaneously with additional graphical information, e.g. symbols, charts, function plots
• CPC Class Codes: G01J 2005/0077 : Imaging
• CPC Class Codes: G01J 5/0025 : Living bodies ear thermometers G01J5/0011; detecting, measuring or recording for diagnostic purposes A61B5/00
• CPC Class Codes: G01J 5/025 : Interfacing a pyrometer to an external device or network; User interface
• CPC Class Codes: G01J 5/0893 : Arrangements to attach devices to a pyrometer, i.e. attaching an optical interface; Spatial relative arrangement of optical elements, e.g. folded beam path G01J5/049 takes precedence
• CPC Class Codes: G01J 5/30 : Electrical features thereof
• CPC Class Codes: G01K 1/02 : Means for indicating or recording specially adapted for thermometers
• CPC Class Codes: G01K 13/223 : Infrared clinical thermometers, e.g. tympanic
• CPC Class Codes: G06F 18/23 : Clustering techniques
• CPC Class Codes: G06F 3/042 : by opto-electronic means
• CPC Class Codes: G06T 2207/10004 : Still image; Photographic image
• CPC Class Codes: G06T 2207/10016 : Video; Image sequence
• CPC Class Codes: G06T 2207/10024 : Color image
• CPC Class Codes: G06T 2207/10048 : Infrared image
• CPC Class Codes: G06T 2207/20024 : Filtering details
• CPC Class Codes: G06T 2207/20076 : Probabilistic image processing
• CPC Class Codes: G06T 2207/20132 : Image cropping
• CPC Class Codes: G06T 2207/30004 : Biomedical image processing
• CPC Class Codes: G06T 2207/30088 : Skin; Dermal
• CPC Class Codes: G06T 2207/30104 : Vascular flow; Blood flow; Perfusion
• CPC Class Codes: G06T 2207/30201 : Face
• CPC Class Codes: G06T 2210/22 : Cropping
• CPC Class Codes: G06T 3/40 : Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06T 5/00 : Image enhancement or restoration
• CPC Class Codes: G06T 5/10 : using non-spatial domain filtering
• CPC Class Codes: G06T 5/20 : using local operators
• CPC Class Codes: G06T 7/0012 : Biomedical image inspection
• CPC Class Codes: G06T 7/0016 : involving temporal comparison
• CPC Class Codes: G06T 7/13 : Edge detection
• CPC Class Codes: G06T 7/20 : Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/90 : Determination of colour characteristics
• CPC Class Codes: G06V 20/40 : in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 30/413 : Classification of content, e.g. text, photographs or tables
• CPC Class Codes: G06V 40/16 : Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/164 : using holistic features
• CPC Class Codes: G06V 40/168 : Feature extraction; Face representation
• CPC Class Codes: G06V 40/172 : Classification, e.g. identification
• CPC Class Codes: G16H 30/20 : for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 40/63 : for local operation
• CPC Class Codes: H04N 25/75 : Circuitry for providing, modifying or processing image signals from the pixel array
• CPC Class Codes: H04N 5/33 : Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

A microprocessor is operably coupled to a camera from which patient vital signs of reduced images of skin are determined. A temporal variation of reduced images of skin from the camera is generated from multiple filters and then amplified from which the patient vital sign, such as heart rate or respiratory rate, can be determined and then displayed or stored.

Claims

1. An apparatus of motion amplification to communicate biological vital signs, the apparatus comprising:

a cropper module that is operable to receive at least two images and crop the image to exclude a border area of the images, generating at least two cropped images;

a skin-pixel-identification module that is operably coupled to the cropper module and that identifies pixel values that are representative of the skin in at least two cropped images;

a first frequency filter module that is operably coupled to the skin-pixel-identifier and that includes a first frequency filter and that applies the first frequency filter to output of the skin-pixel-identifier;

a regional facial clusterial module that is operably coupled to the first frequency filter module and that applies spatial clustering to the output of the first frequency filter module;

a second frequency filter module that is operably coupled to the regional facial clusterial module and that includes a second frequency filter and that applies the second frequency filter to output of the regional facial clusterial module, generating a temporal variation;

a vital-sign generator that is operably coupled to the second frequency filter module that generates at least one vital sign from the temporal variation; and

a display device that is operably coupled to the vital-sign generator that is operable to display the at least one vital sign.

2. The apparatus of claim 1, wherein the skin-pixel-identifier further comprises:

an automatic seed point based clustering apparatus.

3. The apparatus of claim 1, wherein the regional facial clusterial module further comprises:

a fuzzy clusterer.

4. The apparatus of claim 1, wherein the first frequency filter module further comprises:

a high pass filter.

5. The apparatus of claim 1, wherein the first frequency filter module further comprises:

a low pass filter.

6. The apparatus of claim 1, wherein the first frequency filter module further comprises:

a bandpass filter.

7. The apparatus of claim 1, wherein the first frequency filter module further comprises:

a weighted bandpass filter.

8. The apparatus of claim 1, wherein the first frequency filter module further comprises:

a Gaussian filter.

9. The apparatus of claim 1 further comprising:

a storage device that is operably coupled to the vital-sign generator and that is operable to store the at least one vital sign in a volatile memory.

10. The apparatus of claim 1 further comprising:

a storage device that is operably coupled to the vital-sign generator and that is operable to transmit the at least one vital sign to another apparatus.

11. An apparatus of motion amplification to communicate biological vital signs, the apparatus comprising:

a cropper module that is operable to receive at least two images and operable to crop the images to exclude a border area of the images, generating at least two cropped images;

a first frequency filter module that includes a first frequency filter and that applies the first frequency filter to the at least two cropped images;

a regional facial clusterial module that is operably coupled to the first frequency filter module and that applies spatial clustering to output of the first frequency filter module;

a second frequency filter module that is operably to the regional facial clusterial module and that includes a second frequency filter and that applies the second frequency filter to output of the regional facial clusterial module, generating a temporal variation;

a vital-sign generator that is operably coupled to the second frequency filter module that generates at least one vital sign from the temporal variation; and

a display device that is operably coupled to the vital-sign generator that is operable to display the at least one vital sign.

12. The apparatus of claim 11 further comprising:

a skin-pixel-identifier that is operably coupled to the first frequency filter module that identifies pixel values that are representative of skin in the at least two images.

13. The apparatus of claim 11, wherein the regional facial clusterial module further comprises:

a fuzzy clusterer.

14. The apparatus of claim 11, wherein the regional facial clusterial module further comprises:

a seed point based clustering apparatus.

15. The apparatus of claim 11, wherein the first frequency filter module further comprises:

a one-dimensional spatial Fourier Transformation apparatus.

16. The apparatus of claim 11, wherein the first frequency filter module further comprises:

a high pass filter.

17. The apparatus of claim 11, wherein the first frequency filter module further comprises:

a low pass filter.

18. The apparatus of claim 11, wherein the first frequency filter module further comprises:

a Gaussian filter.

19. The apparatus of claim 11 further comprising:

a storage device that is operably coupled to the vital-sign generator and that is operable to store the at least one vital sign in a volatile memory.

20. The apparatus of claim 11 further comprising:

a storage device that is operably coupled to the vital-sign generator and that is operable to transmit the at least one vital sign to another apparatus.