Monitoring respiration with a thermal imaging system

• US 8,790,269 B2
• Filed: 05/09/2011
• Issued: 07/29/2014
• Est. Priority Date: 05/09/2011
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Xu, Beilei, Mestha, Lalit Keshav, Pennington, Graham
• Primary Examiner(s): Kahelin, Michael
• Assistant Examiner(s): Wang, Yunqing
• Application Number:
  US13/103,406
• Publication Number:
  US 20120289850A1
• Time in Patent Office: 1,177 Days
• Field of Search: 600529-538
• US Class Current: 600/529
• CPC Class Codes: A61B 2503/04 Babies, e.g. for SIDS detection
• CPC Class Codes: A61B 2576/00 Medical imaging apparatus involving image processing or analysis A61B1/00009, A61B6/52 and A61B8/52 take precedence; image processing per se G06T; details of algorithms for analysing medical images G06T7/0012
• CPC Class Codes: A61B 5/0008 Temperature signals
• CPC Class Codes: A61B 5/015 By temperature mapping of body part
• CPC Class Codes: A61B 5/08 Detecting, measuring or recording devices for evaluating the respiratory organs A61B5/0205 takes precedence
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing

Abstract

What is disclosed is a system and method for monitoring respiration of a subject or subject of interest using a thermal imaging system with single or multiple spectral bands set to a temperature range of a facial region of that person. Temperatures of extremities of the head and face are used to locate facial features in the captured thermal images, i.e., nose and mouth, which are associated with respiration. The RGB signals obtained from the camera are plotted to obtain a respiration pattern. From the respiration pattern, a rate of respiration is obtained. The system includes display and communication interfaces wherein alerts can be activated if the respiration rate falls outside a level of acceptability. The teachings hereof find their uses in an array of devices such as, for example, devices which monitor the respiration of an infant to signal the onset of a respiratory problem or failure.

Claims

1. A method for monitoring respiration using a thermal imaging system, the method comprising:

capturing thermal images of a subject of interest intended to be monitored for respiration using a thermal image camera set to a temperature of a facial region of said subject, each thermal image comprising a plurality of pixels each corresponding to a surface temperature of said facial region across a thermal wavelength band;

retrieving locational relationship of vectors and angles to determine said location of a facial feature consisting of a nose and a mouth by;

using a location of eyes relative to a location of ears of said subject to locate said nose, and

using a location of said ears and chin to locate said mouth;

analyzing said thermal images to determine said location of said nose and said mouth;

tracking physical motion of said facial feature over time to identify a physical motion pattern of respiration, said tracked physical motion being independent from determining a thermal pattern of respiration using thermal image analysis of said facial feature;

determining a respiration rate from said physical motion pattern of respiration;

performing any of;

comparing said determined respiration rate to a pre-defined respiration rate, and comparing said determined physical motion pattern of respiration pattern to a pre-defined physical motion pattern of respiration pattern; and

initiating an alarm in response to either of said comparisons falling outside a level of acceptability.

2. The method of claim 1, wherein analyzing said thermal images to determine said facial feature associated with respiration comprises:

using pixel temperature values to identify a location of an extremity of the head and face of said subject of interest; and

determining a location of said facial feature associated with respiration relative to said location of said identified head and face extremity.

3. The method of claim 2, wherein determining a location of said facial feature associated with respiration comprises:

retrieving from a database at least one locational relationship associated with said identified extremity of said subject'"'"'s head and face; and

using said locational relationship to determine said location of said facial feature associated with respiration.

4. The method of claim 1, wherein identifying said pattern of respiration comprises tracking over time any of:

R, G, and B, values of pixels associated with said facial feature associated with respiration obtained from the R,G,B, channels of said camera.

5. The method of claim 4, wherein determining said respiration rate comprises counting a number of peaks of said tracked pixel values over an amount of time.

6. The method of claim 1, wherein comparing said respiration rate to said pre-defined respiration rate comprises calculating a difference between said two respiration rates.

7. The method of claim 1, wherein comparing said determined physical motion pattern of respiration to said pre-defined physical motion pattern of respiration comprises determining a measure of mismatch between said two physical motion patterns of respiration.

8. The method of claim 1, further comprising compensating for a movement of said subject of interest in said captured video sequence by removing non-periodic motion of said facial feature.

9. The method of claim 1, wherein said alarm comprises any of:

a visible light, an audible noise, and a signal which activates a device.

10. A system for monitoring respiration using a thermal imaging system, the system comprising:

a thermal image camera set to a temperature of a facial region of a subject of interest intended to be monitored for respiration, said thermal camera capturing thermal images of said subject of interest, each thermal image comprising a plurality of pixels each corresponding to a surface temperature of said facial region across a thermal wavelength band;

a memory and a storage medium; and

a processor in communication with and said thermal image camera, said storage medium and said memory, said processor executing machine readable instructions for performing;

retrieving locational relationship of vectors and angles to determine said location of a facial feature consisting of a nose and a mouth by;

using a location of eyes relative to a location of ears of said subject to locate said nose, and

using a location of said ears and chin to locate said mouth;

analyzing said captured thermal images to determine said location of said nose and said mouth;

tracking physical motion of said facial feature over time to identify a physical motion pattern of respiration, said tracked physical motion being independent from determining a thermal pattern of respiration using thermal image analysis of said facial feature;

determining a respiration rate from said physical motion pattern of respiration;

performing any of;

comparing said determined respiration rate to a pre-defined respiration rate, and comparing said determined physical motion pattern of respiration pattern to a pre-defined physical motion pattern of respiration pattern; and

initiating an alarm in response to either of said comparisons falling outside a level of acceptability.

11. The system of claim 10, wherein analyzing said thermal images to determine said facial feature associated with respiration comprises:

using pixel temperature values to identify a location of an extremity of the head and face of said subject of interest; and

determining a location of said facial feature associated with respiration relative to said location of said identified head and face extremity.

12. The system of claim 11, wherein determining a location of said facial feature associated with respiration comprises:

retrieving from a database at least one locational relationship associated with said identified extremity of said subject'"'"'s head and face; and

using said locational relationship to determine said location of said facial feature associated with respiration.

13. The system of claim 10, wherein identifying said pattern of respiration comprises tracking over time any of:

R, G, and B, values of pixels associated with said facial feature associated with respiration obtained from the R,G,B, channels of said camera.

14. The system of claim 13, wherein determining said respiration rate comprises counting a number of peaks of said plotted pixel values over an amount of time.

15. The system of claim 10, wherein comparing said respiration rate to said pre-defined respiration rate comprises calculating a difference between said two respiration rates.

16. The system of claim 10, wherein comparing said determined physical motion pattern of respiration to said pre-defined physical motion pattern of respiration comprises determining a measure of mismatch between said two physical motion patterns of respiration.

17. The system of claim 10, further comprising compensating for a movement of said subject of interest in said captured video sequence by removing non-periodic motion of said facial feature.

18. The system of claim 10, wherein said alarm comprises any of:

a visible light, an audible noise, and a signal which activates a device.

19. A computer implemented method for monitoring respiration using a thermal imaging system, the method comprising:

capturing thermal images of a subject of interest intended to be monitored for respiration using a thermal image camera set to a temperature of a facial region of said subject, each thermal image comprising a plurality of pixels each corresponding to a surface temperature of said facial region across a thermal wavelength band;

retrieving locational relationship of vectors and angles to determine said location of a facial feature consisting of a nose and a mouth by;

using a location of eyes relative to a location of ears of said subject to locate said nose, and

using a location of said ears and chin to locate said mouth;

analyzing said thermal images to determine said location of said nose and said mouth;

tracking physical motion of said facial feature over time to identify a physical motion pattern of respiration, said tracked physical motion being independent from determining a thermal pattern of respiration using thermal image analysis of said facial feature;

determining a respiration rate from said physical motion pattern of respiration;

performing any of;

comparing said determined respiration rate to a pre-defined respiration rate, and comparing said determined physical motion pattern of respiration pattern to a pre-defined physical motion pattern of respiration pattern; and

initiating an alarm in response to either of said comparisons falling outside a level of acceptability.

20. The computer implemented method of claim 19, wherein analyzing said thermal images to determine said facial feature associated with respiration comprises:

using pixel temperature values to identify a location of an extremity of the head and face of said subject of interest; and

determining a location of said facial feature associated with respiration relative to said location of said identified head and face extremity.

21. The computer implemented method of claim 20, wherein determining a location of said facial feature associated with respiration comprises:

retrieving from a database at least one locational relationship associated with said identified extremity of said subject'"'"'s head and face; and

using said locational relationship to determine said location of said facial feature associated with respiration.

22. The computer implemented method of claim 19, wherein identifying said pattern of respiration comprises:

tracking over time any of;

R, G, and B, values of pixels associated with said facial feature associated with respiration obtained from the R,G,B, channels of said camera; and

counting a number of peaks of said plotted pixel values over an amount of time.

23. The computer implemented method of claim 19, wherein comparing said respiration rate to said pre-defined respiration rate comprises calculating a difference between said two respiration rates.

24. The computer implemented method of claim 19, wherein comparing said determined physical motion pattern of respiration to said pre-defined physical motion pattern of respiration comprises determining a measure of mismatch between said two physical motion patterns of respiration.

25. The computer implemented method of claim 19, further comprising compensating for a movement of said subject of interest in said captured video sequence by removing non-periodic motion of said facial feature.