In-vehicle monitoring

• US 10,748,016 B2
• Filed: 04/24/2018
• Issued: 08/18/2020
• Est. Priority Date: 04/24/2017
• Status: Active Grant

• Current Assignee: Oxehealth Limited
• Original Assignee: Oxehealth Limited
• Inventors: Hutchinson, Nicholas Dunkley
• Primary Examiner(s): Johns, Andrew W
• Application Number:
  US15/961,279
• Publication Number:
  US 20180307927A1
• Time in Patent Office: 847 Days
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/02416 using photoplethysmograph signals, e.g. generated by infrared radiation A61B5/14552 takes precedence
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: A61B 5/6893 Cars
• CPC Class Codes: A61B 5/7207 of noise induced by motion artifacts
• CPC Class Codes: A61B 5/7485 Automatic selection of region of interest
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 2207/30076 Plethysmography
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30268 Vehicle interior
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/0016 involving temporal comparison
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/167 using comparisons between temporally consecutive images

Abstract

In a method of video monitoring of a subject, for example a driver of a vehicle, the video image is motion compensated by image registration techniques so that the subject'"'"'s position in each frame of the video image is stable. A region of interest is defined on the skin of the subject and used to obtain a PPG signal. To compensate for variations in illumination of the subject caused by the subject'"'"'s movement in the vehicle, the parameters of the calculated motion transformation used in the image registration are used to form an illumination model representing how the illumination of the subject would have changed because of the subject'"'"'s motion. The illumination model is a linear or quadratic function fitted to the image intensity in the region of interest. Residuals between the fitted model and the image intensity form an illumination-compensated signal in which the photoplethysmographic signal is more clearly present. The illumination-compensated signal is analysed to obtain a PPG signal and from this estimates of one or more vital signs such as heart rate or breathing rate are obtained.

Claims

1. A method of processing sequence of image frames forming a video image of a human subject to estimate a vital sign of the subject comprising the steps of:

analysing the video image to obtain a motion transformation whose parameters represent the movement of the subject;

defining at least one region of interest on the subject in the video image and obtaining a signal representing the image intensity in the at least one region of interest;

obtaining a predicted illumination signal by fitting a function of the motion transformation parameters to the signal representing the image intensity;

obtaining a residuals signal representing the difference between the signal representing the image intensity and the predicted illumination signal after said fitting; and

analysing the residuals signal to obtain an estimate of at least one vital sign of the subject.

2. A method according to claim 1 wherein the step of analysing the video image to obtain a motion transformation comprises performing a motion correction process which produces a motion corrected video image and the motion transformation.

3. A method according to claim 1 wherein the step of analysing the video image to obtain a motion transformation comprises performing an image registration process.

4. A method according to claim 1 wherein the region of interest is defined in the video image before the step of analysing the video image to obtain a motion transformation, and the step of analysing the video image to obtain a motion transformation comprises tracking the region of interest through the sequence of image frames.

5. A method according to claim 1 wherein the motion transformation is a projective transformation.

6. A method according to any claim 1 wherein the step of fitting a function of the motion transformation parameters to the signal representing the image intensity comprises linear or quadratic regression.

7. A method according to claim 1 wherein a plurality of regions of interest are defined on the subject.

8. A method according to claim 1 wherein the intensity signal is formed for the or each at least one region of interest by averaging or summing pixel values in the at least one region of interest.

9. A method according to claim 1 wherein the intensity signal contains a photoplethysmographic signal.

10. A method according to claim 1 wherein the video image is of a subject in a vehicle.

11. Apparatus for video image processing comprising an input for receiving a video image signal, an image processor adapted to process a video image in accordance with the method of claim 1, and an output for outputting the estimate of the at least one vital sign of the subject.

12. A video monitoring system comprising a video camera for capturing a video image of a subject and outputting a video image and apparatus for video image processing as defined in claim 11.

13. A vehicle having installed therein a video monitoring system according to claim 12, the video camera being positioned to capture an image of the operator of the vehicle.