Image detection and processing for building control

• US 9,082,202 B2
• Filed: 08/31/2013
• Issued: 07/14/2015
• Est. Priority Date: 09/12/2012
• Status: Active Grant

• Current Assignee: Building Robotics, Inc. (Siemens AG)
• Original Assignee: Enlighted, Inc. (Siemens AG)
• Inventors: Kovesi, Peter Dominic, Petridis, Evan Danny, Mohan, Tanuj
• Primary Examiner(s): Wu, Jingge
• Application Number:
  US14/016,055
• Publication Number:
  US 20140072211A1
• Time in Patent Office: 682 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20224 Image subtraction
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

Occupancy detection for building environmental control is disclosed. One apparatus includes at least one image sensor and a controller, wherein the controller is operative to obtain a plurality of images from the at least one image sensor, determine a color difference image of a color difference between consecutive images of the plurality of images, determining an area of the color difference image wherein the color difference is greater than a threshold, create an occupancy candidate based on filtering of the determined area if the determined area is less than light change threshold, generate one or more centroids that represent areas of the occupancy candidate that are greater than a second threshold, track the one or more centroids over a period of time, detect occupancy if one or more of the centroids is detected to have moved more than a movement threshold over the period of time.

Claims

1. An apparatus, comprising:

at least one image sensor;

a controller, wherein at least one of the controller, or an external controller electronically connected to the controller are operative to;

obtain a plurality of images from the at least one image sensor;

determine a color difference image of a color difference between consecutive images of the plurality of images;

determining an area of the color difference image wherein the color difference is greater than a threshold;

create an occupancy candidate based on filtering of the determined area if the determined area is less than a light change threshold;

generate one or more centroids that represent areas of the occupancy candidate that are greater than a second threshold;

track the one or more centroids over a period of time; and

detect occupancy if one or more of the centroids is detected to have moved more than a movement threshold over the period of time.

2. The apparatus of claim 1, wherein determining a color difference image of a color difference between consecutive images, and determining an area of the color difference image wherein the color difference is greater than a threshold comprises the at least one of the controller, or the external controller electronically connected to the controller are operative to:

convert the plurality of images to a grey scale;

determine an absolute value of a difference in the grey scale between consecutive images; and

determine an area of the plurality of images wherein the absolute value is greater than a threshold.

3. The apparatus of claim 1, wherein at least one of the controller, or the external controller are further operative to control at least one environmental parameter of an area of interest based on the detected occupancy.

4. The apparatus of claim 2, wherein at least one of the controller, or an external controller electronically connected to the controller are further operative to down-sample the plurality of images before converting the plurality of images to a grey scale.

5. The apparatus of claim 2, wherein converting the plurality of images to a grey scale comprises processing a particular color channel.

6. The apparatus of claim 1, wherein if the determined area is greater than the light change threshold, then determining that a lighting change has occurred, not a detection of occupancy.

7. The apparatus of claim 1, wherein filtering of the determined area comprises summation filtering of the determined area.

8. The apparatus of claim 1, further comprising powering an infrared (IR) illuminator proximate to the at least one image senor of the apparatus if lighting proximate to the at least one image sensor is sensed to be below a lighting threshold.

9. The apparatus of claim 8, further comprising reducing a frame rate of the plurality of images.

10. The apparatus of claim 9, wherein the IR illuminator is operative to strobe with a pattern that is synchronous with a frame capture times sequence of the plurality of images.

11. The apparatus of claim 1, wherein at least one of the controller, or an external controller electronically connected to the controller are further operative to measure temporal sequences of movement of the centroids, and compare the measured temporal sequences to a database of temporal sequences known to be associated with humans, allowing an estimate of a likelihood that one or more humans appear in the plurality of images.

12. The apparatus of claim 1, wherein at least one of the controller, or an external controller electronically connected to the controller are further operative to:

analyze the plurality of images acquired of an area of interest in which occupancy is being detected;

select at least a target area within the plurality of images which is best suited as an ambient target area for ambient light level sensing;

calculate a reflectivity estimate for each of at least one target area using knowledge of values of applied lighting;

calculate ambient lighting estimates for the at least one target area using the calculated reflectivity estimates for the at least one target area; and

continuously calculate a total light level in the area of interest using the ambient lighting estimates of at least one target area from the plurality of images.

13. A method of detecting motion, comprising:

obtaining a plurality of images;

determining a color difference image of a color difference between consecutive images of the plurality of images;

determining an area of the color difference image wherein the color difference is greater than a threshold;

creating an occupancy candidate based on filtering of the determined area if the determined area is less than light change threshold;

generating one or more centroids that represent areas of the occupancy candidate that are greater than a second threshold;

tracking the one or more centroids over a period of time; and

detecting occupancy if one or more of the centroids is detected to have moved more than a movement threshold over the period of time.

14. The method of claim 13, wherein determining a color difference image of a color difference between consecutive images, and determining an area of the color difference image wherein the color difference is greater than a threshold comprises

converting the plurality of images to a grey scale;

determining an absolute value of a difference in the grey scale between consecutive images;

determining an area of the plurality of images wherein the absolute value is greater than a threshold.

15. The method of claim 14, further comprising down-sampling the plurality of images before converting the plurality of images to a grey scale.

16. The method of claim 14, wherein converting the plurality of images to a grey scale comprises processing a particular color channel.

17. The method of claim 13, wherein if the determined area is greater than light change threshold, then determining that a lighting changes has occurred not detection of occupancy.

18. The method of claim 13, wherein filtering of the determined area comprises summation filtering of the determined area.

19. The method of claim 13, further comprising powering an infrared (IR) illuminator proximate to a device that is obtaining a plurality of images if lighting proximate to the device is sensed to be below a lighting threshold.

20. The method of claim 19, further comprising reducing a frame or image rate of the plurality of images.

21. The method of claim 20, wherein the IR illuminator is strobed with a pattern that is synchronous with a frame capture times sequence of the plurality of images.

22. The method of claim 14, further comprising measuring temporal sequences of movement of the centroids, and comparing the measured temporal sequences to a database of temporal sequences known to be associated with humans, allowing an estimate of a likelihood that one or more humans appear in the plurality of images.

23. The method of claim 13, further comprising:

analyzing the plurality of images acquired of an area of interest in which occupancy is being detected;

selecting at least a target area within the plurality of images which is best suited as an ambient target area for ambient light level sensing;

calculating a reflectivity estimate for each of at least one target area using knowledge of values of applied lighting;

calculating ambient lighting estimates for the at least one target area using the calculated reflectivity estimates for the at least one target area; and

continuously calculating a total light level in the area of interest using the ambient lighting estimates of at least one target area from the plurality of images.