System and method for home health care monitoring

• US 10,095,930 B2
• Filed: 05/10/2016
• Issued: 10/09/2018
• Est. Priority Date: 01/17/2012
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: Avigilon Fortress Corporation
• Inventors: Venetianer, Peter L., Myers, Gary W., Zhang, Zhong
• Primary Examiner(s): Abdi, Amara
• Application Number:
  US15/150,456
• Publication Number:
  US 20160253802A1
• Time in Patent Office: 882 Days
• CPC Class Codes: A61B 2505/07 Home care
• CPC Class Codes: A61B 5/0013 Medical image data A61B1/00011, A61B6/56, A61B8/56 take precedence
• CPC Class Codes: A61B 5/0046 Arrangements of imaging apparatus in a room, e.g. room provided with shielding or for improved access to apparatus
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/1072 measuring distances on the body, e.g. measuring length, height or thickness A61B5/1076 takes precedence
• CPC Class Codes: A61B 5/1073 Measuring volume, e.g. of limbs
• CPC Class Codes: A61B 5/1079 using optical or photographic means
• CPC Class Codes: A61B 5/1113 Local tracking of patients, e.g. in a hospital or private home
• CPC Class Codes: A61B 5/1116 Determining posture transitions
• CPC Class Codes: A61B 5/1117 Fall detection
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A61B 5/1176 Recognition of faces
• CPC Class Codes: A61B 5/7282 Event detection, e.g. detecting unique waveforms indicative of a medical condition A61B5/4094, A61B5/4818, A61B5/349, A61B5/0823 take precedence
• CPC Class Codes: A61B 5/746 Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms alarms per se G08B
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/0016 involving temporal comparison
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06T 7/55 : from multiple images
• CPC Class Codes: G06T 7/579 : from motion
• CPC Class Codes: G06T 7/62 : of area, perimeter, diameter or volume
• CPC Class Codes: G06T 7/73 : using feature-based methods
• CPC Class Codes: G06V 20/41 : Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/44 : Event detection
• CPC Class Codes: G06V 20/52 : Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/103 : Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/1365 : Matching; Classification
• CPC Class Codes: G08B 13/19615 : wherein said pattern is defined by the user
• CPC Class Codes: G08B 21/043 : detecting an emergency event, e.g. a fall
• CPC Class Codes: G08B 21/0476 : Cameras to detect unsafe condition, e.g. video cameras
• CPC Class Codes: H04N 7/18 : Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 : for receiving images from a plurality of remote sources

Abstract

A monitoring method and system are disclosed. In one embodiment, a method includes monitoring the health of a person in a home. The method includes capturing a video sequence from a first camera disposed within the home, including capturing two-dimensional image data for the video sequence; receiving depth data corresponding to the two-dimensional data, and associating the depth data with the video sequence as metadata; setting a plurality of events to monitor associated with the person, the events defined to include actions captured from the first camera, at least a first event including the person'"'"'s body being in a particular bodily position and performing video content analysis on the video sequence to determine whether the events have occurred. The video content analysis includes automatically detecting a potential human object from the video sequence based on the two-dimensional image data; using the depth data to determine a size and bodily position of the potential human object; and based on the size of the potential object, confirming that the potential human object is an actual human, thereby confirming the potential human object as a target. The method further includes determining that the first event has occurred based on the determined bodily position of the target.

Claims

1. A method of monitoring the health of a person in a home, the method comprising:

capturing a video sequence from a first camera disposed within the home, including capturing two-dimensional image data for the video sequence;

receiving depth data corresponding to the two-dimensional data, and associating the depth data with the video sequence as metadata;

setting a plurality of events to monitor associated with the person, the events defined to include actions captured from the first camera, at least a first event including the person'"'"'s body being in a particular bodily position;

performing video content analysis on the video sequence to determine whether the events have occurred, wherein the video content analysis includes;

automatically detecting a potential human object from the video sequence based on the two-dimensional image data;

using the depth data to determine a size, a volume and bodily position of the potential human object, wherein the volume is determined using a plurality of pivot points on a surface of the potential human object, a self-occlusion line, and a corresponding mirror image of each pivot point on the self-occlusion line; and

based on the size of the potential object, confirming that the potential human object is an actual human, thereby confirming the potential human object as a target; and

determining that the first event has occurred based on the determined bodily position of the target.

2. The method of claim 1, wherein determining the bodily position of the potential human object includes determining whether the object is standing, sitting, or lying down.

3. The method of claim 1, wherein the first event includes one or more of:

(1) a person standing;

(2) a person sitting;

or (3) a person lying down.

4. The method of claim 3, wherein the first event includes:

(1) a person standing for at least a particular amount of time;

or (2) a person lying down for at least a particular amount of time.

5. The method of claim 1, further comprising:

setting an alarm to go off if it is determined that the first event has occurred.

6. A method of monitoring people, the method comprising:

capturing a video sequence from a first camera disposed in a particular location within a building, including capturing two-dimensional image data;

receiving depth data for the video sequence, the depth data corresponding to the two-dimensional image data;

associating the depth data with the video sequence as metadata;

setting a plurality of events to monitor associated with the people, at least a first event including a person'"'"'s body being in a particular bodily position;

performing video content analysis on the video sequence to determine whether the events have occurred, wherein the video content analysis includes;

(a) automatically detecting a potential human object from the video sequence based on the two-dimensional image data;

(b) using the depth data to determine a size, a volume and bodily position of the detected potential human object, wherein the volume is determined using a plurality of pivot points on a surface of the potential human object, a self-occlusion line, and a corresponding mirror image of each pivot point on the self-occlusion line; and

(c) based on the determined size of the detected potential human object, confirming that the detected potential human object is an actual human, thereby confirming the detected potential human object as a target; and

determining that the first event has occurred based on the determined bodily position of the target.

7. The method of claim 6, wherein the only depth data used during any of the steps (a) through (c) is the depth data for the objects determined to be potential human objects.

8. The method of claim 6, wherein determining the bodily position of the detected potential human object includes determining whether the object is standing, sitting, or lying down.

9. The method of claim 8, wherein the first event includes:

(1) a person standing for at least a particular amount of time;

or (2) a person lying down for at least a particular amount of time.

10. The method of claim 6, further comprising:

setting an alarm to go off if it is determined that the first event has occurred.

11. A video surveillance system comprising:

a first video camera that captures two-dimensional image data and depth data; and

a video content analysis system configured to;

receive a video sequence from the first video camera, including receiving two-dimensional image data and depth data for the video sequence;

associate the depth data with the video sequence as metadata;

set a plurality of events to monitor associated with people;

perform video content analysis on the of video sequence to determine whether the events have occurred, wherein the video content analysis includes;

(a) automatically detecting a potential human object from the video sequence based on the two-dimensional image data;

(b) using the depth data to determine a size, a volume, and bodily position of the potential human object, wherein the volume is determined using a plurality of pivot points on a surface of the potential human object, a self-occlusion line, and a corresponding mirror image of each pivot point on the self-occlusion line; and

(c) based on the size of the potential object, confirming that the potential human object is an actual human, thereby confirming the potential human object as a target; and

determining that a first event has occurred based on the determined bodily position of the target.

12. The video surveillance system of claim 11, wherein the only depth data used during any of the steps (a) through (c) is the depth data for the objects determined to be potential human objects.

13. The video surveillance system of claim 11, wherein determining the bodily position of the potential human object includes determining whether the object is standing, sitting, or lying down.

14. The video surveillance system of claim 13, wherein the first event includes:

(1) a person standing for at least a particular amount of time;

or (2) a person lying down for at least a particular amount of time.