SYSTEMS AND METHODS FOR DETERMINING WHETHER AN INDIVIDUAL ENTERS A PRESCRIBED VIRTUAL ZONE USING SKELETAL TRACKING AND 3D BLOB DETECTION

• US 20190057592A1
• Filed: 10/22/2018
• Published: 02/21/2019
• Est. Priority Date: 06/01/2015
• Status: Active Grant

• Current Assignee: Cerner Innovation, Inc. (Oracle Corporation)
• Original Assignee: Cerner Innovation, Inc. (Oracle Corporation)
• Inventors: KUSENS, Neil
• Granted Patent: US 10,629,046 B2
• Time in Patent Office: Days
• CPC Class Codes: A61B 5/1117 Fall detection
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/34 Smoothing or thinning of the pattern; Morphological operations; Skeletonisation
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/53 Recognition of crowd images, e.g. recognition of crowd congestion
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G08B 21/043 detecting an emergency event, e.g. a fall
• CPC Class Codes: G08B 23/00 Alarms responsive to unspecified undesired or abnormal conditions
• CPC Class Codes: G08B 25/14 Central alarm receiver or annunciator arrangements
• CPC Class Codes: H05K 999/99 dummy group

Abstract

A method and system that allows healthcare providers to monitor disabled, elderly or other high-risk individuals to prevent or reduce falls and/or mitigate the impact of a fall by delivering automated notification of “at risk” behavior and falls by such an individual. Two systems are used to identify patients—a skeletal tracking system that identifies patients by biometric indicators and a virtual blob detection system. In the virtual blob system, the monitored individual is virtually represented as a blob object of at least a specific size by a computerized monitoring system, and such system detects and alerts when the blob object enters or crosses into a virtually defined or designated blob detection zone and remains in the zone for at least a predetermined period of time. These systems may be used concurrently and conflicts between the systems may be resolved.

Claims

1. A system for detecting when a monitored individual or any part of the monitored individual has crossed over a designated electronic perimeter, the system comprising:

one or more 3D camera, motion, and sound sensors located in a room with an individual to be monitored and configured to capture video data of the individual within the room;

a computerized monitoring system configured to;

electronically receive video data from the one or more 3D camera, motion, and sound sensors;

use skeletal tracking to electronically monitor the room for a crossing of a designated electronic perimeter by the individual based on the video data electronically received from the one or more 3D camera, motion, and sound sensors;

use virtual blob detection to electronically monitor the room for the crossing of the designated electronic perimeter by the individual based on the video data electronically received from the one or more 3D camera, motion, and sound sensors;

detect a conflict between determinations of whether the individual or part of the individual crossed the designated electronic perimeter made using skeletal tracking and virtual blob detection;

resolve the conflict between the determinations made using skeletal tracking and virtual blob detection according to a predetermined setting; and

based on resolution of the conflict, electronically transmit a determination that the individual or part of the individual has crossed the designated electronic perimeter.

2. The system of claim 1, wherein the predetermined setting prioritizes using skeletal tracking over using virtual blob detection, and wherein the determination that the individual or part of the individual has crossed the designated electronic perimeter that is electronically transmitted is made using skeletal tracking.

3. The system of claim 2, wherein the predetermined setting prioritizes using skeletal tracking over using virtual blob detection when the individual has previously been identified using skeletal tracking.

4. The system of claim 1, wherein the predetermined setting prioritizes using virtual blob detection over using skeletal tracking, and wherein the determination that the individual or part of the individual has crossed the designated electronic perimeter that is electronically transmitted is made using virtual blob detection.

5. The system of claim 1 further comprising a computerized communication system that is configured to receive an alert from the computerized monitoring system when the computerized monitoring system electronically detects that the individual or a part of the individual has crossed the designated electronic perimeter.

6. The system of claim 5, wherein the computerized monitoring system is configured to electronically alert the computerized communication system when the computerized monitoring system electronically detects that the individual or a part of the individual has crossed the designated electronic perimeter and remains in the designated electronic perimeter for a predetermined period of time.

7. The system of claim 6, wherein the computerized communication system is further configured to electronically issue an audible message to the individual to inform the individual that the individual has crossed the designated electronic perimeter.

8. The system of claim 1, further comprising a central monitoring alert display, wherein an alert is generated on the central monitoring alert display upon the computerized monitoring system electronically determining that the individual or part of the individual crossed the designated electronic perimeter.

9. The system of claim 1, wherein electronically transmitting the determination that the individual has crossed the designated electronic perimeter comprises notifying a designated person associated with the individual.

10. The system of claim 9, wherein notifying the designated person associated with the individual is further based on determining that the individual remains in the designated electronic perimeter for at least a predetermined period of time.

11. A computerized method for detecting when a monitored individual or any part of the monitored individual has crossed over a designated electronic perimeter, the method comprising:

electronically receiving video data from one or more 3D camera, motion, and sound sensors that are configured to capture video data of an individual being monitored;

using skeletal tracking to electronically monitor the room for a crossing of a designated electronic perimeter by the individual based on the video data electronically received from the one or more 3D camera, motion, and sound sensors;

using virtual blob detection to electronically monitor the room for the crossing of the designated electronic perimeter by the individual based on the video data electronically received from the one or more 3D camera, motion, and sound sensors;

detecting a conflict between determinations of whether the individual or part of the individual crossed the designated electronic perimeter made using skeletal tracking and virtual blob detection;

resolving the conflict between the determinations made using skeletal tracking and virtual blob detection according to a predetermined setting; and

based on resolution of the conflict, electronically transmitting a determination that the individual or part of the individual has crossed the designated electronic perimeter.

12. The computerized method of claim 11, wherein the predetermined setting prioritizes using skeletal tracking over using virtual blob detection, and wherein the determination that the individual or part of the individual has crossed the designated electronic perimeter that is electronically transmitted is made using skeletal tracking.

13. The computerized method of claim 12, wherein the predetermined setting prioritizes using skeletal tracking over using virtual blob detection when the individual has previously been identified using skeletal tracking.

14. The computerized method of claim 11, wherein the predetermined setting prioritizes using virtual blob detection over using skeletal tracking, and wherein the determination that the individual or part of the individual has crossed the designated electronic perimeter that is electronically transmitted is made using virtual blob detection.

15. The computerized method of claim 11, wherein electronically transmitting the determination that the individual has crossed the designated electronic perimeter comprises transmitting an alert to a computerized communication system.

16. The computerized method of claim 15, wherein the alert to the computerized communication system is transmitted upon determining that the individual or part of the individual has remained in the designated electronic perimeter for at least a predetermined period of time.

17. The computerized method of claim 15, wherein the computerized communication system is further configured to transmit the determination to a central monitoring alert display, wherein an alert is generated on the central monitoring alert display.

18. The computerized method of claim 11, wherein electronically transmitting the determination that the individual or part of the individual has crossed the designated electronic perimeter comprises electronically issuing an audible message to the individual to inform the individual that the individual has crossed the designated electronic perimeter.

19. The computerized method of claim 11, wherein electronically transmitting the determination that the individual has crossed the designated electronic perimeter comprises notifying a designated person associated with the individual.

20. The computerized method of claim 19, wherein notifying the designated person associated with the individual is further based on determining that the individual remains in the designated electronic perimeter after a predetermined period of time.