Method and system for counting people using depth sensor

• US 10,009,579 B2
• Filed: 11/21/2012
• Issued: 06/26/2018
• Est. Priority Date: 11/21/2012
• Status: Active Grant

• Current Assignee: Pelco Incorporated (Motorola Solutions, Inc.)
• Original Assignee: Pelco Incorporated (Motorola Solutions, Inc.)
• Inventors: Wang, Lei, Aghdasi, Farzin
• Primary Examiner(s): Yenke, Brian
• Application Number:
  US13/683,886
• Publication Number:
  US 20140139633A1
• Time in Patent Office: 2,043 Days
• CPC Class Codes: G06V 20/53 Recognition of crowd images, e.g. recognition of crowd congestion
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A sensor system according to an embodiment of the invention may process depth data and visible light data for a more accurate detection. Depth data assists where visible light images are susceptible to false positives. Visible light images (or video) may similarly enhance conclusions drawn from depth data alone. Detections may be object-based or defined with the context of a target object. Depending on the target object, the types of detections may vary to include motion and behavior. Applications of the described sensor system include motion guided interfaces where users may interact with one or more systems through gestures. The sensor system described may also be applied to counting systems, surveillance systems, polling systems, retail store analytics, or the like.

Claims

1. A method, comprising:

obtaining a frame of depth data from a depth sensor, the depth sensor mounted to provide a top view of a scene;

discerning foreground objects from background objects from within the frame of depth data;

for a given foreground object, calculating expected relative head size, in pixels of the frame of depth data, of the given foreground object at a depth of the given foreground object, the expected relative head size calculated using a pre-determined head size, a width in pixel units of the frame of depth data, an angle the depth sensor covers, and a depth of the given foreground object determined using the frame of depth data obtained; and

determining if the given foreground object matches a single-scale reference model of a target object, the single-scale reference model of the target object determined based upon the expected relative head size at the depth of the foreground object,

wherein the expected relative head size is calculated according to the following equation;

2. The method of claim 1, further comprising:

defining properties of a detection window based on the expected relative head size; and

generating a scene population count based on an application of the detection window to the given foreground object.

3. The method of claim 2, further comprising resetting the scene population count to zero in an event of no motion within a scene for a predetermined temporal period.

4. The method of claim 2, further comprising obtaining visible images from a visible spectrum camera and using information in the visible spectrum camera to check accuracy of the count.

5. The method of claim 4, wherein information from the visible spectrum camera includes motion information, a time attribute, and location features.

6. The method of claim 5, further comprising communicating metadata to a data center.

7. The method of claim 2, further comprising reporting the scene population count, statistics or complementary data.

8. The method of claim 1, further comprising:

obtaining multiple frames of depth data from the depth sensor; and

tracking, from frame to frame of the multiple frames of depth data, one or more foreground objects determined to match the reference model from among the foreground objects.

9. The method of claim 1, further comprising imaging the scene using the depth sensor.

10. A non-transitory computer readable medium having program instructions stored thereon, the program instructions being executable by a processor and, when loaded and executed by the processor, causing the processor to:

obtain a frame of depth data from a depth sensor, the depth sensor mounted to provide a top view of a scene;

identify a given foreground object, from among multiple foreground objects and background objects, from within the frame of depth data,

for a given foreground object, calculate expected relative head size, in pixels of the frame of depth data, of the given foreground object at a depth of the given foreground object, the expected relative head size calculated using a pre-determined head size, a width in pixel units of the frame of depth data, an angle the depth sensor covers, and a depth of the given foreground object determined using the frame of depth data obtained;

determine if the given foreground object matches a single-scale reference model of a target object, the single-scale reference model of the target object determined based upon the expected relative head size at the depth of the foreground object;

apply a machine learning application to generate a classification determination of the foreground object; and

maintain a classification determination count,

wherein the expected relative head size is calculated according to the following equation;

11. A system, comprising:

a depth sensor configured to image depth of objects to acquire depth data;

a memory, in communication with the depth sensor, configured to store the depth data;

a processor, in communication with the memory, configured to execute program instructions that cause the processor to;

obtain a frame of depth data from the depth sensor, the depth sensor mounted to provide a top view of a scene;

discern foreground objects from background objects from within the frame of depth data;

for a given foreground object, calculate expected relative head size, in pixels of the frame of depth data, of the given foreground object at a depth of the given foreground object, the expected relative head size calculated using a predetermined head size, a width in pixel units of the frame of depth data, an angle the depth sensor covers, and a depth of the given foreground object determined using the frame of depth data obtained; and

determine if the given foreground object matches a single-scale reference model of a target object, the single-scale reference model of the target object determined based upon the expected relative head size at the depth of the foreground object,

wherein the expected relative head size is calculated according to the following equation;

12. The system of claim 11, wherein the processor is further configured to:

define properties of a detection window based on the expected relative head size; and

generate a scene population count based on application of the detection window to the given foreground object.

13. The system of claim 12, wherein the processor is further configured to reset the scene population count to zero in an event of no motion within a scene for a predetermined temporal period.

14. The system of claim 12, wherein the processor is further configured to obtain visible images from a visible spectrum camera and use information in the visible spectrum camera to check accuracy of the count.

15. The system of claim 14, wherein information from the visible spectrum camera includes motion information, a time attribute, and location features.

16. The system of claim 15, wherein the processor is further configured to communicate metadata to a data center.

17. The system of claim 12, wherein the processor is further configured to report the scene population count, statistics or complementary data.

18. The system of claim 11, wherein the processor is further configured to:

obtain multiple frames of depth data from the depth sensor; and

track, from frame to frame of the multiple frames of depth data, one or more foreground objects determined to match the reference model from among the foreground objects.

19. The system of claim 11, wherein the processor is further configured to image the scene using the depth sensor.