Device, system and method for skin detection

• US 9,842,392 B2
• Filed: 12/02/2015
• Issued: 12/12/2017
• Est. Priority Date: 12/15/2014
• Status: Active Grant

• Current Assignee: Koninklijke Philips N.V.
• Original Assignee: Koninklijke Philips N.V.
• Inventors: De Haan, Gerard
• Primary Examiner(s): Nakhjavan, Shervin
• Application Number:
  US14/956,429
• Publication Number:
  US 20160171684A1
• Time in Patent Office: 741 Days
• Field of Search: None
• CPC Class Codes: A61B 5/02416 using photoplethysmograph signals, e.g. generated by infrared radiation A61B5/14552 takes precedence
• CPC Class Codes: A61B 5/14551 for measuring blood gases
• CPC Class Codes: A61B 5/4887 Locating particular structures in or on the body
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 18/2323 based on graph theory, e.g. minimum spanning trees [MST] or graph cuts
• CPC Class Codes: G06F 2218/00 Aspects of pattern recognition specially adapted for signal processing
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/431 Frequency domain transformation; Autocorrelation
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/7635 based on graphs, e.g. graph cuts or spectral clustering
• CPC Class Codes: G06V 40/15 Biometric patterns based on physiological signals, e.g. heartbeat, blood flow

Abstract

An input unit (20) obtains a sequence of image frames over time. A segmentation unit (22) segments image frames of the sequence of image frames. A tracking unit (24) tracks segments of the segmented image frame over time in the sequence of image frames. A clustering unit (26) clusters the tracked segments to obtain clusters representing skin of a subject by use of one or more image features of the tracked segments.

Claims

1. A system for skin detection of vital sign information, comprising:

a color video camera configured to acquire a sequence of color image frames;

a control computer processor configured to;

receive the sequence of image frames,

segment the images frames of said sequence of image frames,

track segments of the segmented image frames in said sequence of image frames,

cluster the tracked segments using temporal color variations of the tracked segments as a feature for clustering the tracked segments to obtain clusters representing skin of a subject, and

extract vital sign information from the clustered tracked segments; and

a user interface including a display configured to be controlled by control computer processor to display the extracted vital sign information.

2. The system as claimed in claim 1, wherein the control processor is further configured to perform a Fourier transformation of a spatially combined color value of at least part of the pixels in the tracked segments and to cluster two tracked segments into a single cluster if their amplitude peak frequency has substantially the same value and phase.

3. A device for skin detection comprising:

an input unit for obtaining a sequence of image frames acquired over time,

a segmentation unit for segmenting an image frame of said sequence of image frames,

a tracking unit for tracking segments of the segmented image frame over time in said sequence of image frames,

a clustering unit for clustering the tracked segments to obtain clusters representing skin of a subject by use of one or more image features of said tracked segments,

wherein said clustering unit is configured to compute the inner product between normalized time signals of different tracked segments and to cluster said tracked segments into a single cluster if said inner product exceeds a predetermined threshold.

4. The device as claimed in claim 3, wherein said segmentation unit is configured to over-segment the image frame that is used for segmentation.

5. The device as claimed in claim 3, wherein said clustering unit is further configured to use the hue of pixels of the tracked segments as feature for clustering the tracked segments.

6. The device as claimed in claim 3, wherein said segmentation unit is configured to perform a Harris corner detection.

7. The device as claimed in claim 3, wherein said segmentation unit is configured to perform a Delaunay triangulation.

8. A system for skin detection comprising:

a video camera configured to acquire the sequence of image frames over time; and

the device for skin detection as claimed in claim 3 based on the acquired sequence of image frames.

9. A device for skin detection comprising:

a camera configured to acquire a sequence of image frames of skin of a subject over time;

a user interface including a display; and

a computer programmed to;

segment the image frames of said sequence of image frames,

track segments of the segmented image frames of the sequence of image frames over time,

cluster the tracked segments to obtain clusters representing skin of a subject using one or more image features of the tracked segments,

repeatedly perform the segmentation, the tracking and the clustering, wherein the sequence of acquired image frames used for subsequent repetitions overlap in time,

repeatedly extract physiological information from the clustered tracked segments, and

controlling the user interface display to display the extracted physiological information.

10. The device as claimed in claim 9, wherein the computer is programmed to over-segment the image frames based on color, position and/or texture properties.

11. The device as claimed in claim 9, wherein the computer is programmed to track the segments of the segmented image frames using motion estimation.

12. The device as claimed in claim 9, wherein the physiological information includes vital signs of the subject.

13. A method for skin detection comprising:

obtaining a sequence of image frames of a subject acquired over time,

segmenting the image frames of said sequence of image frames to define segments,

tracking the segments of the segmented image frames over time in said sequence of image frames,

clustering the tracked segments to obtain clusters representing skin properties of the subject by use of one or more image features of said tracked segments, and

iteratively repeating the segmenting, tracking, and clustering to update the skin properties, wherein the sequence of image frames of subsequent repetitions overlap in time.

14. A non-transitory computer-readable medium carrying program code for controlling a computer to carry out the steps of the method as claimed in claim 13.

15. The method as claimed in claim 13, further including:

detecting vital signs of the subject in the updated skin properties, and

controlling a user interface to display the vital signs.

16. The method as claimed in claim 13, wherein the sequence of video frames is acquired with a color video camera and the clustering is based on temporal color variations of the tracked segments.

17. The method as claimed in claim 13, further including:

computing an inner product between normalized time signals of different tracked segments, the tracked segments being clustered into a single cluster if the inner product exceeds a predetermined threshold.