Methods and systems for detecting stroke symptoms

• US 10,614,288 B2
• Filed: 12/30/2016
• Issued: 04/07/2020
• Est. Priority Date: 12/31/2015
• Status: Active Grant

• Current Assignee: Cerner Innovation, Inc. (Oracle Corporation)
• Original Assignee: Cerner Innovation, Inc. (Oracle Corporation)
• Inventors: Kusens, Michael, Kusens, Neil
• Primary Examiner(s): Kalapodas, Dramos
• Application Number:
  US15/395,762
• Publication Number:
  US 20170195637A1
• Time in Patent Office: 1,194 Days
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/11 Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb for measuring pulse A61B5/02 ;A61B5/1038 takes precedence; motion detection to correct for motion artifacts in physiological signals A61B5/721
• CPC Class Codes: A61B 5/1176 Recognition of faces
• CPC Class Codes: A61B 5/4064 Evaluating the brain A61B5/031, A61B5/369, A61B5/14553 take precedence
• CPC Class Codes: A61B 5/746 Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms alarms per se G08B
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/0016 involving temporal comparison
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/292 : Multi-camera tracking
• CPC Class Codes: G06V 20/52 : Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/647 : by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: G06V 40/161 : Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/172 : Classification, e.g. identification
• CPC Class Codes: G06V 40/20 : Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G08B 13/196 : using television cameras
• CPC Class Codes: G08B 13/19639 : Details of the system layout
• CPC Class Codes: G08B 21/0476 : Cameras to detect unsafe condition, e.g. video cameras
• CPC Class Codes: G08B 21/182 : Level alarms, e.g. alarms responsive to variables exceeding a threshold
• CPC Class Codes: G08B 25/009 : Signalling of the alarm condition to a substation whose identity is signalled to a central station, e.g. relaying alarm signals in order to extend communication range
• CPC Class Codes: G08B 5/22 : using electric transmission; using electromagnetic transmission
• CPC Class Codes: G16H 10/60 : for patient-specific data, e.g. for electronic patient records
• CPC Class Codes: G16H 15/00 : ICT specially adapted for medical reports, e.g. generation or transmission thereof
• CPC Class Codes: G16H 20/10 : relating to drugs or medications, e.g. for ensuring correct administration to patients
• CPC Class Codes: G16H 30/20 : for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 40/20 : for the management or administration of healthcare resources or facilities, e.g. managing hospital staff or surgery rooms
• CPC Class Codes: G16H 40/63 : for local operation
• CPC Class Codes: G16H 40/67 : for remote operation
• CPC Class Codes: G16H 50/30 : for calculating health indices; for individual health risk assessment
• CPC Class Codes: G16H 80/00 : ICT specially adapted for facilitating communication between medical practitioners or patients, e.g. for collaborative diagnosis, therapy or health monitoring
• CPC Class Codes: H04N 13/204 : using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/207 : using a single 2D image sensor
• CPC Class Codes: H04N 2013/0085 : Motion estimation from stereoscopic image signals
• CPC Class Codes: H04N 23/63 : by using electronic viewfinders
• CPC Class Codes: H04N 7/18 : Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 : for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/183 : for receiving images from a single remote source

Abstract

A stroke detection system analyzes images of a person'"'"'s face over time to detect asymmetric changes in the position of certain reference points that are consistent with sagging or drooping that may be symptomatic of a stroke or TIA. On detecting possible symptoms of a stroke or TIA, the system may alert caregivers or others, and log the event in a database. Identifying stroke symptoms automatically may enable more rapid intervention, and identifying TIA symptoms may enable diagnostic and preventative care to reduce the risk of a future stroke.

Claims

1. A computerized method for detecting stroke symptoms, the method performed by one or more processors of a computerized monitoring system and comprising:

receiving, from one or more 3D sensors having at least one camera, image data depicting a face of a person, wherein the image data comprises at least a series of two or more images of the face;

identifying, in the image data depicting the face of the person, a reference anatomical feature, the reference anatomical feature being an ear or a nose;

digitally superimposing an x-axis, a y-axis, and a z-axis over at least a portion of image data depicting the face of the person by placing at least one of the x-axis or the y-axis at the reference anatomical feature;

identifying, in the image data depicting the face of the person, soft tissue anatomical features of the face;

identifying, over time, positions of a plurality of reference points that correspond to the soft tissue anatomical features on the face of the person relative to the x-axis, the y-axis, and the z-axis using at least a portion of the image data;

detecting an asymmetric change in facial features over time based on an asymmetric change in the positions of at least some reference points within the plurality of reference points of soft tissue anatomical features relative to the x-axis, the y-axis, and the z-axis over time;

determining that the asymmetric change in facial features is consistent with a stroke symptom at least by identifying a minimum asymmetric change in positions of at least some reference points and determining that the minimum asymmetric change is maintained for a period of time; and

electronically communicating the determination that the asymmetric change in facial features of the person is consistent with the stroke symptom at least by alerting a designated recipient of the minimum asymmetric change.

2. The method of claim 1 further comprising communicating the series of two or more images to a central monitoring station.

3. The method of claim 2, wherein a plurality of series of images for a plurality of people being monitored are displayed on a primary display at the central monitoring station.

4. The method of claim 3, wherein electronically communicating the determination that the asymmetric change in facial features of the person is consistent with the stroke symptom further comprises issuing an alert to the central monitoring station.

5. The method of claim 4, wherein upon the central monitoring station receiving the alert, images of the person for whom the determination of the asymmetric change in facial features was made are displayed on an alert display of the central monitoring station.

6. A system for detecting stroke symptoms, the system comprising:

one or more 3D sensors located to provide the one or more 3D sensors with a view of a face of a person to be monitored, the one or more 3D sensors including at least one camera configured to collect a series of images of the face of the person;

a computerized monitoring system comprising one or more processors communicatively coupled to the one or more 3D sensors, the computerized monitoring system configured to;

receive image data comprising at least the series of images depicting the face of the person from the one or more 3D sensors;

identify, in the series of images depicting the face, a reference anatomical feature, the reference anatomical figure being an ear or a nose;

digitally superimpose an x-axis, a y-axis, and a z-axis over at least some images within the series of images depicting the face of the person by placing at least one of the x-axis or the y-axis at the reference anatomical feature;

identify, in the series of images depicting the face, soft tissue anatomical features of the face;

identify, over time, positions of a plurality of reference points that correspond to the soft tissue anatomical features on the face of the person relative to the x-axis, the y-axis, and the z-axis using the image data; and

detect an asymmetric change in facial features over time based on an asymmetric change in the positions of at least some reference points within the plurality of reference points relative to the x-axis, the y-axis, and the z-axis over time;

determine that the asymmetric change in facial features is consistent with a stroke symptom at least by identifying a minimum asymmetric change in positions of at least some reference points and determining that the minimum asymmetric change is maintained for a period of time; and

a computerized communication system communicatively coupled to the computerized monitoring system, the computerized communication system configured to send an alert to one or more designated recipients when the minimum asymmetric change in facial features of the person is determined to be consistent with the stroke system.

7. The system of claim 6 further comprising a central monitoring station communicatively coupled to the computerized communication system, the central monitoring station configured to display at least a portion of the series of images of the face of the person.

8. The system of claim 7, wherein the central monitoring station comprises a primary display and an alert display.

9. The system of claim 8, wherein the central monitoring station is configured to display an alert on the alert display.

10. The system of claim 8, wherein upon the computerized monitoring system determining that the minimum asymmetric change in facial features of the person is consistent with the stroke symptom, the computerized communication system sends an alert to the central monitoring station, and the central monitoring station moves a display of at least a portion of the series of images of the face of the person from the primary display to the alert display.

11. Non-transitory computer-readable storage media having embodied thereon instructions which, when executed by one or more computer processors, cause the computer processors to:

receive, from one or more 3D sensors having at least one camera, image data comprising a series of two or more images of a face of a person;

identify, in the series of two or more images of the face, a reference anatomical feature of the face, the reference anatomical figure being an ear or a nose;

digitally superimpose an x-axis, a y-axis, and a z-axis over at least some images within the series of two or more images of the face of the person by placing at least one of the x-axis or the y-axis at the reference anatomical feature of the face;

identify, in the series of two or more images of the face, soft tissue anatomical features of the face;

identify, over time, positions of a plurality of reference points that correspond to the soft tissue anatomical features on the face of the person relative to the x-axis, the y-axis, and the z-axis using at least a portion of the image data;

detect an asymmetric change in facial features over time based on an asymmetric change in the positions of at least some reference points within the plurality of reference points relative to the x-axis, the y-axis, and the z-axis over time;

determine that the asymmetric change in facial features is consistent with a stroke symptom at least by identifying a minimum asymmetric change in positions of at least some reference points and determining that the minimum asymmetric change is maintained for a period of time; and

electronically communicate a determination that the asymmetric change in facial features of the person is consistent with the stroke symptom at least by alerting a designated recipient of the minimum asymmetric change.

12. The computer-readable storage media of claim 11, wherein the instructions further cause the one or more computer processors to display a plurality of series of images for a plurality of people being monitored on a primary display at a central monitoring station.

13. The computer-readable storage media of claim 12, wherein electronically communicating the determination that the asymmetric change in facial features of the person is consistent with the stroke symptom further comprises sending an alert to the central monitoring station and wherein, upon receiving the alert, the central monitoring station duplicates the display of the series of images associated with the alert on an alert display of the central monitoring station.