Detecting unauthorized visitors

• US 10,210,378 B2
• Filed: 12/20/2017
• Issued: 02/19/2019
• Est. Priority Date: 12/31/2015
• Status: Active Grant

• Current Assignee: Cerner Innovation, Inc. (Oracle Corporation)
• Original Assignee: Cerner Innovation, Inc. (Oracle Corporation)
• Inventors: Kusens, Michael, Kusens, Neil
• Primary Examiner(s): Azarian, Seyed
• Application Number:
  US15/848,621
• Publication Number:
  US 20180114053A1
• Time in Patent Office: 426 Days
• Field of Search: 382100, 382103, 382106-107, 382128-134, 382154, 382155, 382162, 382168, 382173, 382181, 382190, 382199, 382203, 382209, 382214, 382219, 382232, 382254, 382274, 382276, 382286-291, 382305, 382312, 128903, 3482221, 348135, 348143
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/11 Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb for measuring pulse A61B5/02 ;A61B5/1038 takes precedence; motion detection to correct for motion artifacts in physiological signals A61B5/721
• CPC Class Codes: A61B 5/1176 Recognition of faces
• CPC Class Codes: A61B 5/4064 Evaluating the brain A61B5/031, A61B5/369, A61B5/14553 take precedence
• CPC Class Codes: A61B 5/746 Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms alarms per se G08B
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/0016 involving temporal comparison
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/292 : Multi-camera tracking
• CPC Class Codes: G06V 20/52 : Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/647 : by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: G06V 40/161 : Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/172 : Classification, e.g. identification
• CPC Class Codes: G06V 40/20 : Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G08B 13/196 : using television cameras
• CPC Class Codes: G08B 13/19639 : Details of the system layout
• CPC Class Codes: G08B 21/0476 : Cameras to detect unsafe condition, e.g. video cameras
• CPC Class Codes: G08B 21/182 : Level alarms, e.g. alarms responsive to variables exceeding a threshold
• CPC Class Codes: G08B 25/009 : Signalling of the alarm condition to a substation whose identity is signalled to a central station, e.g. relaying alarm signals in order to extend communication range
• CPC Class Codes: G08B 5/22 : using electric transmission; using electromagnetic transmission
• CPC Class Codes: G16H 10/60 : for patient-specific data, e.g. for electronic patient records
• CPC Class Codes: G16H 15/00 : ICT specially adapted for medical reports, e.g. generation or transmission thereof
• CPC Class Codes: G16H 20/10 : relating to drugs or medications, e.g. for ensuring correct administration to patients
• CPC Class Codes: G16H 30/20 : for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 40/20 : for the management or administration of healthcare resources or facilities, e.g. managing hospital staff or surgery rooms
• CPC Class Codes: G16H 40/63 : for local operation
• CPC Class Codes: G16H 40/67 : for remote operation
• CPC Class Codes: G16H 50/30 : for calculating health indices; for individual health risk assessment
• CPC Class Codes: G16H 80/00 : ICT specially adapted for facilitating communication between medical practitioners or patients, e.g. for collaborative diagnosis, therapy or health monitoring
• CPC Class Codes: H04N 13/204 : using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/207 : using a single 2D image sensor
• CPC Class Codes: H04N 2013/0085 : Motion estimation from stereoscopic image signals
• CPC Class Codes: H04N 23/63 : by using electronic viewfinders
• CPC Class Codes: H04N 7/18 : Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 : for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/183 : for receiving images from a single remote source

Abstract

An unauthorized visitor system collects an image of a person detected in a room of a patient. The system identifies reference points on the person'"'"'s face, for example, points along the cheeks, jowls, and/or brow. The system may compare the reference points to reference points of images associated with registered visitors. The system then determines, based on the comparison, if the person is a registered visitor. One or more designated recipients may be alerted if the person is not a registered visitor or if the person breaches a patient identification zone established around a particular patient. The system may also register the person in a database of visitors.

Claims

1. A system for detecting unauthorized visitors, the system comprising:

one or more 3D motion sensors located to provide the one or more 3D motion sensors with a view of a person and surrounding area to be monitored, the one or more 3D motion sensors configured to collect a series of images of the person and the surrounding area;

a computerized patient monitoring system communicatively coupled to the one or more 3D motion sensors, the computerized patient monitoring system configured to;

utilize information received from the one or more 3D motion sensors to identify the person being monitored;

determine when a visitor enters the view of the one or more 3D motion sensors; and

determine whether the visitor is an unauthorized visitor by identifying a plurality of reference points on the face of the visitor to compare positions, distances between, sizes, or shapes of the plurality of the reference points to positions, distances between, sizes, or shapes of a plurality of registered reference points on the faces of registered visitors, the plurality of registered reference points on the faces of registered visitors being stored in a database;

a central communication system communicatively coupled to the computerized patient monitoring system, the central communication system configured to;

receive an indication from the computerized patient monitoring system upon the computerized patient monitoring system determining that the visitor is an unauthorized visitor, and send an alert to one or more designated recipients; and

a central video monitoring system comprising a primary display communicatively coupled to the central communication system, the central video monitoring system configured to display at least a portion of the series of images of the visitor, wherein after the computerized patient monitoring system detects that the visitor is an unauthorized visitor, the central video monitoring system displays at least a portion of the series of images of the unauthorized visitor on the primary display,

wherein after the alert is sent to one or more designated recipients by the central communication system, the computerized patient monitoring system determines whether the unauthorized visitor made one or more corrective actions based on information from the one or more 3D motion sensors.

2. The system of claim 1, wherein the computerized patient monitoring system is further configured to identify the person being monitored utilizing skeletal tracking or blob detection.

3. The system of claim 1, wherein the central video monitoring system further comprises an alert display that is separate from the primary display.

4. The system of claim 1, further comprising determining the visitor is a registered visitor of the patient and is indicated as not being allowed to be in proximity to the patient.

5. The system of claim 1, further comprising determining the visitor is not a registered visitor.

6. The system of claim 5, further comprising creating a recognition profile for the visitor upon determining the visitor is not registered.

7. The system of claim 6, wherein when it is determined that the visitor is not a registered visitor, the computerized communication system sends an alert to the central video monitoring system.

8. A computerized method for detecting unauthorized visitors, the method implemented by one or more processors and comprising:

receiving, from one or more 3D motion sensors, an image of at least two persons detected in a room of a patient, wherein the at least two persons are detected using skeletal tracking;

determining the at least two persons are in proximity to each other;

determining at least one person is an unauthorized visitor by identifying reference points on the visitor using information from the one or more 3D motion sensors;

based on determining at least one person is an unauthorized visitor, sending a notification of the determination to a central communication system; and

after an alert is sent to one or more designated recipients by the central communication system, determining whether the at least one person who is an unauthorized visitor made one or more corrective actions based on information from the one or more 3D motion sensors.

9. The method of claim 8, further comprising determining one of the at least two persons is a caregiver for the patient.

10. The method of claim 8, further comprising determining one of the at least two persons is a registered visitor for the patient.

11. The method of claim 8, further comprising determining each of the at least two persons is an unauthorized visitor for the patient.

12. The method of claim 8, wherein one of the one or more designated recipients is a central video monitoring system communicatively coupled to the central communication system, and wherein images for a plurality of people being monitored are displayed on a primary display of the central video monitoring system.

13. The method of claim 12, wherein upon determining that the at least one person is an unauthorized visitor, the central video monitoring system moves a display of at least a portion of a series of images from the primary display to an alert display.

14. Non-transitory computer-readable storage media having embodied thereon instructions which, when executed by one or more computer processors, cause the processors to:

receive, from one or more 3D motion sensors a series of images of a person detected in a room of a patient, the person being in proximity to the patient;

determine that the person is an unauthorized visitor by identifying a plurality of reference points on the face of the person and comparing positions, distances between, sizes, or shapes of the plurality of the reference points to positions, distances between, sizes, or shapes of a plurality of registered reference points on the faces of registered visitors, the plurality of registered reference points on the faces of registered visitors being stored in a database;

electronically communicate the determination that the person is an unauthorized visitor; and

after an alert is sent to one or more designated recipients by a central communication system, determining whether the unauthorized visitor made one or more corrective actions based on information from the one or more 3D motion sensors.

15. The computer-readable storage media of claim 14, wherein one of the one or more designated recipients is a central video monitoring system, and wherein the central video monitoring system moves a display of at least a portion of the series of images of the person from a primary display to an alert display.

16. The computer-readable storage media of claim 14, wherein one of the one or more designated recipients is a central video monitoring system, and wherein the series of images are displayed on a primary display of the central video monitoring.

17. The computer-readable storage media of claim 14, wherein the instructions further cause the one or more computer processors to determine whether the person is registered and, upon determining the person is not registered, creating a profile for the person.

18. The computer-readable storage media of claim 14, wherein the person is determined to be an unauthorized visitor when the person is not identified as someone who is a registered visitor for the patient.

19. The computer-readable storage media of claim 14, wherein the person is determined to be an unauthorized visitor when the person is identified as a registered visitor and a profile associated with the registered visitor indicates that the registered visitor is not allowed to be in close proximity to the patient.