METHOD AND SYSTEM FOR AUDIENCE DIGITAL MONITORING

• US 20120027299A1
• Filed: 07/20/2011
• Published: 02/02/2012
• Est. Priority Date: 07/20/2010
• Status: Active Grant

• Current Assignee: SET Corp. (Leidos Holdings, Inc.)
• Original Assignee: SET Corp. (Leidos Holdings, Inc.)
• Inventors: RAN, Yang
• Granted Patent: US 8,630,455 B2
• Time in Patent Office: Days
• US Class Current: 382/173
• CPC Class Codes: G06V 20/53 Recognition of crowd images, e.g. recognition of crowd congestion

Abstract

Aspects of the present invention allow for real-time people monitoring method and system for estimation of the size and flow density of a given group of people located in a given area of space. The method and system may be used to monitor live or recorded camera input, foreground segmentation, human tracking, height estimation, and dwell estimation. In particular, human detection may be achieved using a spatio-temporal variance analysis calculation methodology for moving target detection.

Claims

1. A method of audience monitoring, comprising:

determining a mask for an image having a plurality of image pixels, the image pixels including a plurality of mask pixels, a plurality of background pixels, and a plurality of foreground pixels, the mask having the plurality of mask pixels, a background having the plurality of background pixels, and a foreground having the plurality of foreground pixels;

removing at least a portion of the plurality of mask pixels and at least a portion of the plurality of background pixels from the image pixels;

calibrating at least a portion of the plurality of foreground pixels;

determining a sum total of the calibrated foreground pixels; and

determining a number of people present in the audience based on the calculated sum total of calibrated foreground pixels.

2. The method of claim 1, wherein calibrating the plurality of foreground pixels comprises:

forming a virtual parallelogram having parallel sides in the image, the virtual parallelogram including a closest point having a coordinate y₁ along a direction parallel to one of the parallel sides and having a weight w₁, a farthest point having a coordinate y₃ along the direction and having a weight w₂, and a vanishing point having a coordinate y_vanish along the direction, wherein the vanishing point y_vanish corresponds to a projected meeting point of the parallel sides of the virtual parallelogram in space;

calculating the coordinate y_vanish according to;

3. The method of claim 2, wherein calibrating the plurality of foreground pixels further comprises:

determining a closest point having a coordinate z₁ along a direction perpendicular to one of the parallel sides of the virtual parallelogram and having a weight w′

₁, a farthest point having a coordinate z₃ along the perpendicular direction and having a weight w′

₂, and a vanishing point having a coordinate z_vanish along the perpendicular direction, wherein the vanishing point z_vanish corresponds to a projected meeting point of lines that are perpendicular to the parallel sides of the virtual parallelogram in space;

calculating the coordinate z_vanish according to;

4. The method of claim 3, wherein determining the sum total of the calibrated foreground pixels comprises:

determining a mask matrix including the plurality of mask pixels;

determining a weight matrix including respective weights of each of the calibrated foreground pixels; and

calculating the sum total of the calibrated foreground pixels according to;

5. The method of claim 4, wherein determining the number of people present in the audience comprises:

dividing the sum total of the calibrated foreground pixels by an average number of pixels per person.

6. The method of claim 3, further comprising:

determining a mask matrix including the plurality of mask pixels;

determining pixel bands that include any of the plurality of image pixels along a plurality of scanning lines of the image and forming a blob of pixels;

determining a blob weight matrix including respective weights of each blob of pixels; and

calculating a sum total {right arrow over (n)} of calibrated foreground pixels as;

7. The method of claim 6, wherein determining the number of people present in the audience comprises:

dividing the sum total of the calibrated foreground pixels by an average number of pixels per person.

8. A system for audience monitoring, comprising:

means for determining a mask for an image having a plurality of image pixels, the image pixels including a plurality of mask pixels, a plurality of background pixels, and a plurality of foreground pixels, the mask having the plurality of mask pixels, a background having the plurality of background pixels, and a foreground having the plurality of foreground pixels;

means for removing at least a portion of the plurality of mask pixels and at least a portion of the plurality of background pixels from the image pixels;

means for calibrating at least a portion of the plurality of foreground pixels;

means for determining a sum total of the calibrated foreground pixels; and

means for determining a number of people present in the audience based on the calculated sum total of calibrated foreground pixels.

9. The system for audience monitoring according to claim 8, wherein the means for calibrating the plurality of foreground pixels comprises:

means for forming a virtual parallelogram having parallel sides in the image, the virtual parallelogram including a closest point having a coordinate y₁ along a direction parallel to one of the parallel sides and having a weight w₁, a farthest point having a coordinate y₃ along the direction and having a weight w₂, and a vanishing point having a coordinate y_vanish along the direction, wherein the vanishing point y_vanish corresponds to a projected meeting point of the parallel sides of the virtual parallelogram in space;

means for calculating the coordinate y_vanish according to;

10. The system for audience monitoring according to claim 8, wherein the means for calibrating the plurality of foreground pixels further comprises:

means for determining a closest point having a coordinate z₁ along a direction perpendicular to one of the parallel sides of the virtual parallelogram and having a weight w′

₁, a farthest point having a coordinate z₃ along the perpendicular direction and having a weight w′

₂, and a vanishing point having a coordinate z_vanish along the perpendicular direction, wherein the vanishing point z_vanish corresponds to a projected meeting point of lines that are perpendicular to the parallel sides of the virtual parallelogram in space;

means for calculating the coordinate z_vanish according to;

11. The system for audience monitoring according to claim 10, wherein the means for determining the sum total of the calibrated foreground pixels comprises:

means for determining a mask matrix including the plurality of mask pixels;

means for determining a weight matrix including respective weights of each of the calibrated foreground pixels; and

means for calculating the sum total of the calibrated foreground pixels according to;

12. The system for audience monitoring according to claim 10, further comprising:

means for determining a mask matrix including the plurality of mask pixels;

means for determining pixel bands that include any of the plurality of image pixels along a plurality of scanning lines of the image and forming a blob of pixels;

means for determining a blob weight matrix including respective weights of each blob of pixels; and

means for calculating a sum total h of calibrated foreground pixels as;

13. A computer program product comprising a computer usable medium having control logic stored therein for causing a computer to monitor an audience, the control logic comprising:

first computer readable program code means for determining a mask for an image having a plurality of image pixels, the image pixels including a plurality of mask pixels, a plurality of background pixels, and a plurality of foreground pixels, the mask having the plurality of mask pixels, a background having the plurality of background pixels, and a foreground having the plurality of foreground pixels;

second computer readable program code means for removing at least a portion of the plurality of mask pixels and at least a portion of the plurality of background pixels from the image pixels;

third computer readable program code means for calibrating at least a portion of the plurality of foreground pixels;

fourth computer readable program code means for determining a sum total of the calibrated foreground pixels; and

fifth computer readable program code means for determining a number of people present in the audience based on the calculated sum total of calibrated foreground pixels.

14. A system for audience monitoring, the system comprising:

a processor;

a user interface functioning via the processor; and

a repository accessible by the processor, wherein the processor is configured to;

determine a mask for an image having a plurality of image pixels, the image pixels including a plurality of mask pixels, a plurality of background pixels, and a plurality of foreground pixels, the mask having the plurality of mask pixels, a background having the plurality of background pixels, and a foreground having the plurality of foreground pixels;

remove at least a portion of the plurality of mask pixels and at least a portion of the plurality of background pixels from the image pixels;

calibrate at least a portion of the plurality of foreground pixels;

determine a sum total of the calibrated foreground pixels; and

determine a number of people present in the audience based on the calculated sum total of calibrated foreground pixels.

15. The system of claim 14, wherein the processor is housed on a terminal.

16. The system of claim 15, wherein the terminal is selected from a group consisting of a personal computer, a minicomputer, a main frame computer, a microcomputer, a hand held device, and a telephonic device.

17. The system of claim 14, wherein the processor is housed on a server.

18. The system of claim 17, wherein the server is selected from a group consisting of a personal computer, a minicomputer, a microcomputer, and a main frame computer.

19. The system of claim 17, wherein the server is coupled to a network.

20. The system of claim 19, wherein the server is coupled to the network via a coupling.