System and method for building automation using video content analysis with depth sensing
First Claim
Patent Images
1. A method of monitoring a building comprising:
- taking a video within a location in the building with a video sensor, the video comprising a plurality of frames, each frame including 2D image data;
for each frame, receiving depth data associated with the 2D image data, the depth data corresponding to one or more distances from the video sensor to features represented by the 2D image data;
analyzing the 2D image data including performing 2D analysis on the 2D image data to detect one or more image blobs in the video;
using the depth data, projecting an image blob of the one or more image blobs onto a plurality of Z-planes, thereby creating a plurality of blob slices;
separating the Z-planes into a ground plane and non-round planes based on a height threshold, so that at least a first blob slice on the ground plane is at or below the height threshold, and a second blob slice and additional blob slices on the non-ground planes are above the height threshold;
creating a refined blob that includes the second blob slice, the additional blob slices, and only a portion of the first blob slice;
performing object detection on the refined blob, to determine that the blob corresponds to a human object, thereby detecting a person in the video;
based on the detected person and one or more additional detected people, counting a number of people at the location in the building; and
controlling a system of the building in response to the number of people counted.
5 Assignments
0 Petitions
Accused Products
Abstract
A method and system for monitoring buildings (including houses and office buildings) by performing video content analysis based on two-dimensional image data and depth data are disclosed. Occupation and use of such buildings may be monitored with higher accuracy to provide higher energy efficiency usage, to control operation of components therein, and/or provide better security. Height data may be obtained from depth data to provide greater reliability in object detection, object classification and/or event detection.
-
Citations
26 Claims
-
1. A method of monitoring a building comprising:
-
taking a video within a location in the building with a video sensor, the video comprising a plurality of frames, each frame including 2D image data; for each frame, receiving depth data associated with the 2D image data, the depth data corresponding to one or more distances from the video sensor to features represented by the 2D image data; analyzing the 2D image data including performing 2D analysis on the 2D image data to detect one or more image blobs in the video; using the depth data, projecting an image blob of the one or more image blobs onto a plurality of Z-planes, thereby creating a plurality of blob slices; separating the Z-planes into a ground plane and non-round planes based on a height threshold, so that at least a first blob slice on the ground plane is at or below the height threshold, and a second blob slice and additional blob slices on the non-ground planes are above the height threshold; creating a refined blob that includes the second blob slice, the additional blob slices, and only a portion of the first blob slice; performing object detection on the refined blob, to determine that the blob corresponds to a human object, thereby detecting a person in the video; based on the detected person and one or more additional detected people, counting a number of people at the location in the building; and controlling a system of the building in response to the number of people counted. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23)
-
-
24. A method of monitoring a building comprising:
-
taking a video within a location in the building, the video comprising a plurality of frames, each frame including 2D image data; for each frame, receiving depth data associated with the 2D image data, the depth data corresponding to one or more distances from a depth sensor to features represented by the 2D image data; analyzing the 2D image data to detect at least a first image blob in the video; using the depth data, projecting the image blob onto a plurality of Z-planes, thereby creating a plurality of blob slices; based on a height threshold, separating the blob slices into a ground plane blob slice, and non-ground plane blob slices; creating a refined blob that includes non-ground plane blob slices, and only a portion of the ground plane blob slice; performing object detection on the refined blob, to determine that the blob corresponds to a human object, thereby detecting a person in the video; and controlling a system of the building in response to the detected person. - View Dependent Claims (25, 26)
-
Specification