Method for determining whether an individual enters a prescribed virtual zone using skeletal tracking and 3D blob detection

• US 10,147,297 B2
• Filed: 10/09/2017
• Issued: 12/04/2018
• Est. Priority Date: 06/01/2015
• Status: Active Grant

• Current Assignee: Cerner Innovation, Inc. (Oracle Corporation)
• Original Assignee: Cerner Innovation, Inc. (Oracle Corporation)
• Inventors: Kusens, Neil
• Primary Examiner(s): Tabatabai, Abolfazl
• Application Number:
  US15/728,110
• Publication Number:
  US 20180068545A1
• Time in Patent Office: 421 Days
• Field of Search: 382103, 348169
• CPC Class Codes: A61B 5/1117 Fall detection
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/34 Smoothing or thinning of the pattern; Morphological operations; Skeletonisation
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/53 Recognition of crowd images, e.g. recognition of crowd congestion
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G08B 21/043 detecting an emergency event, e.g. a fall
• CPC Class Codes: G08B 23/00 Alarms responsive to unspecified undesired or abnormal conditions
• CPC Class Codes: G08B 25/14 Central alarm receiver or annunciator arrangements
• CPC Class Codes: H05K 999/99 dummy group

Abstract

A method and system that allows healthcare providers, hospitals, skilled nursing facilities and other persons to monitor disabled, elderly or other high-risk individuals to prevent or reduce falls and/or mitigate the impact of a fall by delivering automated notification of “at risk” behavior and falls by such an individual being monitored where assistance is required. Two systems are used to identify patients, a skeletal tracking system, which identifies patients by biometric indicators, and a virtual blob detection system. In the virtual blob system, the monitored individual is virtually represented as a blob object of at least a specific size by a computerized monitoring system and such system detects and alerts when the blob object enters or crosses into a virtually defined or designated blob detection zone and remains in the zone for at least a predetermined period of time. These systems may be used concurrently, or one system may be used primarily, with the other system acting as a failsafe.

Claims

1. A system for detecting when a monitored individual or any part of the monitored individual has crossed over a designated electronic perimeter, the system comprising:

one or more 3D camera, motion and sound sensors located in a room with an individual to be monitored and configured to capture video data of the individual within the room;

a computerized monitoring system configured to;

electronically receive video data from the one or more 3D camera, motion and sound sensors;

use skeletal tracking to search for a skeleton of the monitored individual based on video data from the one or more 3D camera, motion, and sound sensors;

upon being unable to detect the skeleton using skeletal tracking, track the monitored individual using virtual blob detection based on video data from the one or more 3D camera, motion, and sound sensors;

electronically determine whether at least part of the monitored individual has crossed a designated electronic perimeter using virtual blob detection; and

electronically issue an audible message to inform the individual that the individual has crossed the designated electronic perimeter.

2. The system of claim 1, further comprising a computerized communication system configured to receive an alert from the computerized monitoring system when the computerized monitoring system electronically detects that at least part of the monitored individual has crossed over the designated electronic perimeter.

3. The system of claim 2, wherein the computerized monitoring system is configured to electronically alert the computerized communication system hen the computerized monitoring system electronically detects that at least part of the monitored individual has crossed over the designated electronic perimeter and remains in the designated electronic perimeter for a predetermined period of time.

4. The system of claim 3, further comprising a central monitoring alert display, wherein an alert is generated on the central monitoring alert display when the computerized monitoring system electronically determines that the individual remains in the designated electronic perimeter after the predetermined period of time.

5. The system of claim 3, wherein the computerized communication system is further configured to notify a designated person associated with the individual when the computerized monitoring system electronically determines that the individual remains in the designated electronic perimeter after the predetermined period of time.

6. The system of claim 2, wherein the computerized communication system is configured to electronically issue an audible message to the monitored individual to inform the individual that they have crossed over the designated electronic perimeter.

7. The system of claim 2, further comprising a central monitoring alert display, wherein an alert is generated on the central monitoring alert display when the computerized monitoring system electronically determines that the individual crossed the designated electronic perimeter.

8. The system of claim 2, wherein the computerized communication system is further configured to notify a designated person associated with the individual when the computerized monitoring system electronically determines that the individual crossed the designated electronic perimeter.

9. The system of claim 1, further comprising a centralized monitoring primary display remote from the monitored individual and configured to receive a continuous feed of data from the computerized monitoring system and display a monitored area.

10. The system of claim 1, wherein the computerized monitoring system is configured to identify skeletal points for the monitored individual.

11. The system of claim 1, wherein the computerized monitoring system is configured to switch back and forth between skeletal tracking and virtual blob detection monitoring.

12. A system for electronic monitoring of when an individual or any part of the individual crosses over a designated electronic perimeter, the system comprising:

one or more 3D camera, motion and sound sensors located in a room with an individual to be monitored and configured to capture video data of the individual within the room;

a computerized monitoring system configured to;

electronically receive video data from the one or more 3D camera, motion and sound sensors;

using skeletal tracking, electronically monitor the room for a crossing of a designated electronic perimeter by the individual based on the video data electronically received from the one or more 3D camera, motion and sound sensors;

using virtual blob detection, electronically monitor the room for the crossing of the designated electronic perimeter by the individual based on the video data electronically received from the one or more 3D camera, motion and sound sensors, wherein electronically monitoring the room using virtual blob detection occurs simultaneously as electronic monitoring using skeletal tracking; and

electronically detect when at least part of the individual crosses the designated electronic perimeter using at least one of skeletal tracking or virtual blob detection.

13. The system of claim 12 further comprising a centralized monitoring primary display located remote from the individual and configured to receive data from the computerized monitoring system and display a monitored area of the room.

14. The system of claim 12 further comprising a computerized communication system configured to receive an alert from the computerized monitoring system when the computerized monitoring system electronically detects that at least part of the individual has crossed over the designated electronic perimeter.

15. The system of claim 14, wherein the computerized communication system is configured to issue an audible message to the individual to inform the individual that they have crossed over the designated electronic perimeter.

16. The system of claim 14, wherein the computerized communication system is configured to notify a designated person associated with the individual when the computerized monitoring system electronically determines that the individual crossed over the designated electronic perimeter.

17. A system for detecting when a monitored individual or any part of the monitored individual has crossed over a designated electronic perimeter, the system comprising:

one or more 3D camera, motion and sound sensors in a room with the monitored individual and configured to capture video data of the individual within the room;

a computerized monitoring system configured to;

electronically receive video data from the one or more 3D camera, motion and sound sensors;

use skeletal tracking to search for a skeleton of the monitored individual based on video data from the one or more 3D camera, motion and sound sensors;

upon being unable to detect the skeleton using skeletal tracking, track the monitored individual using virtual blob detection based on video data from the one or more 3D camera, motion and sound sensors;

electronically determine whether at least part of the monitored individual has crossed a designated electronic perimeter using virtual blob detection; and

a computerized communication system configured to receive an alert from the computerized monitoring system when at least part of the monitored individual has crossed the designated electronic perimeter and electronically send an audible message to the individual to inform the individual that the individual has crossed the designated electronic perimeter.