System and method for home health care monitoring

• US 9,338,409 B2
• Filed: 01/17/2013
• Issued: 05/10/2016
• Est. Priority Date: 01/17/2012
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: Avigilon Fortress Corporation (Motorola Solutions, Inc.)
• Inventors: Zhang, Zhong, Myers, Gary W., Venetianer, Peter L.
• Primary Examiner(s): Abdi, Amara
• Application Number:
  US13/744,266
• Publication Number:
  US 20130184592A1
• Time in Patent Office: 1,209 Days
• US Class Current: 1/1
• CPC Class Codes: A61B 2505/07 Home care
• CPC Class Codes: A61B 5/0013 Medical image data A61B1/00011, A61B6/56, A61B8/56 take precedence
• CPC Class Codes: A61B 5/0046 Arrangements of imaging apparatus in a room, e.g. room provided with shielding or for improved access to apparatus
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/1072 measuring distances on the body, e.g. measuring length, height or thickness A61B5/1076 takes precedence
• CPC Class Codes: A61B 5/1073 Measuring volume, e.g. of limbs
• CPC Class Codes: A61B 5/1079 using optical or photographic means
• CPC Class Codes: A61B 5/1113 Local tracking of patients, e.g. in a hospital or private home
• CPC Class Codes: A61B 5/1116 Determining posture transitions
• CPC Class Codes: A61B 5/1117 Fall detection
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A61B 5/1176 Recognition of faces
• CPC Class Codes: A61B 5/7282 Event detection, e.g. detecting unique waveforms indicative of a medical condition A61B5/4094, A61B5/4818, A61B5/349, A61B5/0823 take precedence
• CPC Class Codes: A61B 5/746 Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms alarms per se G08B
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/0016 involving temporal comparison
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06T 7/55 : from multiple images
• CPC Class Codes: G06T 7/579 : from motion
• CPC Class Codes: G06T 7/62 : of area, perimeter, diameter or volume
• CPC Class Codes: G06T 7/73 : using feature-based methods
• CPC Class Codes: G06V 20/41 : Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/44 : Event detection
• CPC Class Codes: G06V 20/52 : Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/103 : Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/1365 : Matching; Classification
• CPC Class Codes: G08B 13/19615 : wherein said pattern is defined by the user
• CPC Class Codes: G08B 21/043 : detecting an emergency event, e.g. a fall
• CPC Class Codes: G08B 21/0476 : Cameras to detect unsafe condition, e.g. video cameras
• CPC Class Codes: H04N 7/18 : Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 : for receiving images from a plurality of remote sources

Abstract

A home health care monitoring method and system are disclosed. In one embodiment, a method includes: capturing a plurality of video sequences from a plurality of respective cameras disposed in different locations within a patient'"'"'s home, including capturing two-dimensional image data and depth data for each video sequence; defining a plurality of events to monitor associated with the patient, the events including, during a predetermined time period, at least (1) the patient'"'"'s body being in a particular position for at least a predetermined amount of time; performing depth-enhanced video content analysis on the plurality of video sequences to determine whether the event (1) has occurred. The video content analysis may include: for each of at least two cameras: automatically detecting a human object based on the two-dimensional image data; using the depth data to confirm whether the human object is in the particular position; and based on the confirmation, tracking an amount of time that the human object is in the particular position; and then determining that the event (1) has occurred based on the collective tracked amount of time.

Claims

1. A method of monitoring the health of a person in home, the method comprising:

capturing a plurality of video sequences from a plurality of respective cameras disposed in different locations within the patient'"'"'s home, including capturing two-dimensional image data and depth data for each video sequence;

defining a plurality of events to monitor associated with the patient, the events including, during a predetermined time period, at least (1) the patient'"'"'s body being in a particular position for at least a predetermined amount of time;

performing depth-enhanced video content analysis on the plurality of video sequences to determine whether the event (1) has occurred, wherein the video content analysis includes;

for each of at least two cameras;

automatically detecting a human object based on the two-dimensional image data;

using the depth data associated with the detected human object to confirm whether the human object is in the particular position, without using depth data associated with other parts of the two-dimensional image in the confirmation; and

based on the confirmation, tracking an amount of time that the human object is in the particular position; and

determining that the event (1) has occurred based on the collective tracked amount of time.

2. The method of claim 1, further comprising, for each camera:

determining a height of the human object based on the depth data; and

using the height to confirm that the human object is in the particular position.

3. The method of claim 2, further comprising, for each camera:

determining a volume of the human object based on the depth data, and using the volume to verify that the human object is the person.

4. The method of claim 3, further comprising, for each camera:

verifying that the human object is the patient by comparing the volume of the human object to a stored known volume of the patient.

5. The method of claim 2, further comprising, for each camera:

performing facial recognition on the human object; and

verifying that the human object is the patient based on the facial recognition.

6. The method of claim 1, wherein the particular position is one or more of sitting, standing up, or lying down.

7. The method of claim 6, wherein the events include:

(2) the person spending at least a predetermined amount of time in a particular room.

8. The method of claim 7, wherein the events include:

(3) the person taking medication.

9. The method of claim 1, further comprising:

setting an alarm to go off based on whether event (1) occurs.

10. A home health monitoring system, comprising:

a plurality of cameras in different rooms in a home, the different cameras configured to capture two-dimensional image data and depth data from a plurality of respective video images;

a central computer system configured to perform video content analysis based on the two-dimensional image data and the depth data; and

one or more communication media configured to transmit data from the plurality of cameras to the central computer system,

wherein the video content analysis includes;

setting a plurality of events to monitor associated with an inhabitant in the home, the events including, during a predetermined time period, (1) the patient'"'"'s body being in a particular position for at least a predetermined period of time;

performing video content analysis on the plurality of video sequences to determine whether the event (1) has occurred, wherein the video content analysis includes;

for each of at least two cameras;

automatically detecting a human object based on the two-dimensional image data;

using the depth data associated with the detected human object to confirm whether the human object is in the particular position, without using depth data associated with other parts of the two dimensional image in the confirmation;

based on the confirmation, tracking an amount of time that the human object is in the particular position; and

determining that the event (1) has occurred based on the tracked amount of time.

11. The home health monitoring system of claim 10, further comprising:

a computer storing the depth data as metadata, wherein using the depth data to confirm whether the human object is in the particular position includes analyzing the metadata.