Method for determining whether an individual enters a prescribed virtual zone using skeletal tracking and 3D blob detection

• US 9,892,611 B1
• Filed: 06/01/2015
• Issued: 02/13/2018
• Est. Priority Date: 06/01/2015
• Status: Active Grant

• Current Assignee: Cerner Innovation, Inc. (Oracle Corporation)
• Original Assignee: Cerner Innovation, Inc. (Oracle Corporation)
• Inventors: Kusens, Neil
• Primary Examiner(s): Tabatabai, Abolfazl
• Application Number:
  US14/727,434
• Time in Patent Office: 988 Days
• Field of Search: 382103, 382107, 382236, 348169-172, 348352
• CPC Class Codes: A61B 5/1117 Fall detection
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/34 Smoothing or thinning of the pattern; Morphological operations; Skeletonisation
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/53 Recognition of crowd images, e.g. recognition of crowd congestion
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G08B 21/043 detecting an emergency event, e.g. a fall
• CPC Class Codes: G08B 23/00 Alarms responsive to unspecified undesired or abnormal conditions
• CPC Class Codes: G08B 25/14 Central alarm receiver or annunciator arrangements
• CPC Class Codes: H05K 999/99 dummy group

Abstract

A method and system that allows healthcare providers, hospitals, skilled nursing facilities and other persons to monitor disabled, elderly or other high-risk individuals to prevent or reduce falls and/or mitigate the impact of a fall by delivering automated notification of “at risk” behavior and falls by such an individual being monitored where assistance is required. Two systems are used to identify patients, a skeletal tracking system, which identifies patients by biometric indicators, and a virtual blob detection system. In the virtual blob system, the monitored individual is virtually represented as a blob object of at least a specific size by a computerized monitoring system and such system detects and alerts when the blob object enters or crosses into a virtually defined or designated blob detection zone and remains in the zone for at least a predetermined period of time. These systems may be used concurrently, or one system may be used primarily, with the other system acting as a failsafe.

Claims

1. A method for detecting when a monitored individual or any part of the monitored individual has crossed over a designated electronic perimeter, said method comprising the steps of:

(a) electronically receiving data by a computerized monitoring system from one or more 3D Camera, Motion and Sound sensors located in the room;

(b) tracking a monitored individual using skeletal tracking if a skeleton is detected;

(c) tracking the monitored individual using virtual blob detection if a skeleton is not detected; and

(d) electronically determining by the computerized monitoring system that the monitored individual has crossed a predetermined designated electronic perimeter by either electronic skeletal tracking of the monitored individual or by virtual blob detection.

2. The method for detecting of claim 1 further comprising the step of electronically alerting a computerized communication system by the computerized monitoring system when the computerized monitoring system electronically detects that the monitored individual or a part of the monitored individual has crossed over the designated electronic perimeter.

3. The method for detecting of claim 2 further comprising the step of electronically issuing an audible message to the individual by a computerized communication system to inform the individual that they have crossed over the designated electronic perimeter.

4. The method for detecting of claim 2 further comprising the step of generating an alert on a central monitoring alert display located remote from the monitored individual'"'"'s room if the computerized monitoring system electronically determines that the individual crossed over the designated electronic perimeter.

5. The method for detecting of claim 2 further comprising the step of notifying a designated person associated with the individual if the computerized monitoring system electronically determines that the individual crossed over the designated electronic perimeter.

6. The method for detecting of claim 1 further comprising the step of electronically alerting a computerized communication system by the computerized monitoring system when the computerized monitoring system electronically detects that the monitored individual or a part of the monitored individual has entered or crossed into the designated electronic perimeter and remains in the designated electronic perimeter for a predetermined period of time.

7. The method for detecting of claim 6 further comprising the step of electronically issuing an audible message to the individual by a computerized communication system to inform the individual that they have entered or crossed into the designated electronic perimeter.

8. The method for detecting of claim 6 further comprising the step of generating an alert on a central monitoring alert display located remote from the monitored individual'"'"'s room if the computerized monitoring system electronically determines that the individual fails to exit the designated electronic perimeter within the predetermined period of time.

9. The method for detecting of claim 6 further comprising the step of notifying a designated person associated with the individual if the computerized monitoring system electronically determines that the individual fails to exit the designated electronic perimeter within the predetermined period of time.

10. The method for detecting of claim 1 further comprising the step of electronically sending a continuous feed to remotely located centralized monitoring primary display showing the monitored area of the room by the computerized monitoring system.

11. A method for electronically monitoring for when an individual or any part of the individual crosses over a designated electronic perimeter, said method comprising the steps of:

(a) electronically receiving data by a computerized monitoring system from one or more 3D Camera, Motion and Sound sensors located in a room;

(b) tracking an individual using skeletal tracking if a skeleton is detected;

(c) tracking the individual using virtual blob detection if a skeleton is not detected; and

(d) electronically monitoring the room by the computerized monitoring system for a crossing of a designated electronic perimeter by the individual based on the data electronically received from the one or more 3D Camera, Motion and Sound sensors by either electronic skeletal tracking of the individual or by virtual blob detection.

12. The method for electronically monitoring of claim 11 wherein step (d) comprises the step of monitoring for perimeter crossings by the individual first using skeletal tracking based on skeletal points for the individual.

13. The method for electronically monitoring of claim 12 wherein if after a predetermined period of time no perimeter crossing is determined by the computerized monitoring system using skeletal tracking further comprising the step of switching to virtual blob detection monitoring for perimeter crossings by the computerized monitoring system.

14. The method for electronically monitoring of claim 11 wherein step (d) comprises the step of switching back and forth by the computerized monitoring system between skeletal tracking monitoring based on skeletal points for the individual and virtual blob detection monitoring for a perimeter crossing by the individual.

15. The method for detecting of claim 11 further comprising the step of electronically sending a continuous feed to remotely located centralized monitoring primary display showing the monitored area of the room by the computerized monitoring system.

16. The method for detecting of claim 11 further comprising the step of electronically alerting a computerized communication system by the computerized monitoring system when the computerized monitoring system electronically detects that the individual or a part of the individual has crossed over the designated electronic perimeter.

17. The method for detecting of claim 11 further comprising the step of electronically issuing an audible message to the individual by a computerized communication system to inform the individual that they have crossed over the designated electronic perimeter.

18. The method for detecting of claim 11 further comprising the step of generating an alert on a central monitoring alert display located remote from the individual'"'"'s room if the computerized monitoring system electronically determines that the individual crossed over the designated electronic perimeter.

19. The method for detecting of claim 11 further comprising the step of notifying a designated person associated with the individual if the computerized monitoring system electronically determines that the individual crossed over the designated electronic perimeter.

20. A method for electronically monitoring of when an individual or any part of the individual crosses over a designated electronic perimeter, said method comprising the steps of:

(a) electronically receiving data by a computerized monitoring system from one or more 3D Camera, Motion and Sound sensors located in a room; and

(b) electronically monitoring the room by the computerized monitoring system for a crossing of a designated electronic perimeter by the individual based on the data electronically received from the one or more 3D Camera, Motion and Sound sensors, simultaneously using skeletal tracking monitoring based on skeletal points for the individual and virtual blob detection monitoring by the computerized monitoring system for detecting a perimeter crossing by the individual.