Method and system for audience digital monitoring

• US 8,630,455 B2
• Filed: 07/20/2011
• Issued: 01/14/2014
• Est. Priority Date: 07/20/2010
• Status: Active Grant

• Current Assignee: SET Corp. (Leidos Holdings, Inc.)
• Original Assignee: SET Corp. (Leidos Holdings, Inc.)
• Inventors: Ran, Yang
• Primary Examiner(s): Seth, Manav
• Application Number:
  US13/187,181
• Publication Number:
  US 20120027299A1
• Time in Patent Office: 909 Days
• Field of Search: None
• US Class Current: 382/103
• CPC Class Codes: G06V 20/53 Recognition of crowd images, e.g. recognition of crowd congestion

Abstract

Aspects of the present invention allow for real-time people monitoring method and system for estimation of the size and flow density of a given group of people located in a given area of space. The method and system may be used to monitor live or recorded camera input, foreground segmentation, human tracking, height estimation, and dwell estimation. In particular, human detection may be achieved using a spatio-temporal variance analysis calculation methodology for moving target detection.

Claims

1. A method of audience monitoring, including:

capturing an image having a plurality of image pixels, the image pixels including a plurality of mask pixels, a plurality of background pixels, and a plurality of foreground pixels;

determining a mask having the plurality of mask pixels, a background having the plurality of background pixels, and a foreground having the plurality of foreground pixels;

removing mask pixels and background pixels from the image pixels;

calibrating the foreground pixels;

determining a sum total of foreground pixels; and

determining a number of people present in the audience based on the calculation of the sum total of foreground pixels.

2. The method of claim 1, wherein calibrating the foreground pixels includes:

forming a theoretical parallelogram having parallel sides in the image, the parallelogram including a closest point having a coordinate y₁ along a direction parallel to one of the parallel sides and having a weight w₁, a farthest point having a coordinate y₃ along the direction and having a weight w₂, and a vanishing point having a coordinate y_vanish along the direction, wherein the vanishing point y_vanish corresponds to a projected meeting point of the parallel sides of the parallelogram in space;

calculating the coordinate v vanish according to;

3. The method of claim 2, wherein calibrating the foreground pixels further includes:

determining a closest point having a coordinate z₁ along a direction perpendicular to one of the parallel sides of the parallelogram and having a weight w′

₁, a farthest point having a coordinate z₃ along the perpendicular direction and having a weight w′

₂, and a vanishing point having a coordinate z_vanish along the perpendicular direction, wherein the vanishing point z_vanish corresponds to a projected meeting point of lines that are perpendicular to the parallel sides of the parallelogram in space;

calculating the coordinate z_vanish according to;

4. The method of claim 3, wherein determining the sum total of foreground pixels comprises:

determining a mask matrix including the mask pixels;

determining a weight matrix including respective weights of each foreground pixel; and

calculating the following equation;

5. The method of claim 4, wherein determining the number of people present in the audience comprises dividing the sum total of foreground pixels by an average number of pixels per person.

6. The method of claim 3, further comprising:

determining a mask matrix including the mask pixels;

determining pixel bands that include pixels along a plurality of scanning lines of the image and forming a blob;

determining a blob weight matrix including respective weights of each blob; and

calculating a number of pixels as;

7. The method of claim 6, wherein determining the number of people present in the audience comprises:

dividing the sum total of the calibrated foreground pixels by an average number of pixels per person.

8. A system for audience monitoring, comprising:

means for determining a mask for an image having a plurality of image pixels, the image pixels including a plurality of mask pixels, a plurality of background pixels, and a plurality of foreground pixels, the mask having the plurality of mask pixels, a background having the plurality of background pixels, and a foreground having the plurality of foreground pixels;

means for removing at least a portion of the plurality of mask pixels and at least a portion of the plurality of background pixels from the image pixels;

means for calibrating at least a portion of the plurality of foreground pixels;

means for determining a sum total of the calibrated foreground pixels; and

means for determining a number of people present in the audience based on the calculated sum total of calibrated foreground pixels.

9. The system for audience monitoring according to claim 8, wherein the means for calibrating the plurality of foreground pixels comprises:

means for forming a virtual parallelogram having parallel sides in the image, the virtual parallelogram including a closest point having a coordinate y₁ along a direction parallel to one of the parallel sides and having a weight w₁, a farthest point having a coordinate y₃ along the direction and having a weight w₂, and a vanishing point having a coordinate y_vanish along the direction, wherein the vanishing point y_vanish corresponds to a projected meeting point of the parallel sides of the virtual parallelogram in space;

means for calculating the coordinate y_vanish according to;

10. The system for audience monitoring according to claim 8, wherein the means for calibrating the plurality of foreground pixels further comprises:

means for determining a closest point having a coordinate z₁ along a direction perpendicular to one of the parallel sides of the virtual parallelogram and having a weight w′

₁, a farthest point having a coordinate z₃ along the perpendicular direction and having a weight w′

₂, and a vanishing point having a coordinate z_vanish along the perpendicular direction, wherein the vanishing point z_vanish corresponds to a projected meeting point of lines that are perpendicular to the parallel sides of the virtual parallelogram in space;

means for calculating the coordinate z_vanish according to;

11. The system for audience monitoring according to claim 10, wherein the means for determining the sum total of the calibrated foreground pixels comprises:

means for determining a mask matrix including the plurality of mask pixels;

means for determining a weight matrix including respective weights of each of the calibrated foreground pixels; and

means for calculating the sum total of the calibrated foreground pixels according to;

12. The system for audience monitoring according to claim 10, further comprising:

means for determining a mask matrix including the plurality of mask pixels;

means for determining pixel bands that include any of the plurality of image pixels along a plurality of scanning lines of the image and forming a blob of pixels;

means for determining a blob weight matrix including respective weights of each blob of pixels; and

means for calculating a sum total {right arrow over (n)} of calibrated foreground pixels as;

13. A computer program product comprising a non-transitory computer usable medium having control logic stored therein for causing a computer to monitor an audience, the control logic comprising:

first computer readable program code means for determining a mask for an image having a plurality of image pixels, the image pixels including a plurality of mask pixels, a plurality of background pixels, and a plurality of foreground pixels, the mask having the plurality of mask pixels, a background having the plurality of background pixels, and a foreground having the plurality of foreground pixels;

second computer readable program code means for removing at least a portion of the plurality of mask pixels and at least a portion of the plurality of background pixels from the image pixels;

third computer readable program code means for calibrating at least a portion of the plurality of foreground pixels;

fourth computer readable program code means for determining a sum total of the calibrated foreground pixels; and

fifth computer readable program code means for determining a number of people present in the audience based on the calculated sum total of calibrated foreground pixels.

14. A system for audience monitoring, the system comprising:

a processor;

a user interface functioning via the processor; and

a repository accessible by the processor, wherein the processor is configured to;

determine a mask for an image having a plurality of image pixels, the image pixels including a plurality of mask pixels, a plurality of background pixels, and a plurality of foreground pixels, the mask having the plurality of mask pixels, a background having the plurality of background pixels, and a foreground having the plurality of foreground pixels;

remove at least a portion of the plurality of mask pixels and at least a portion of the plurality of background pixels from the image pixels;

calibrate at least a portion of the plurality of foreground pixels;

determine a sum total of the calibrated foreground pixels; and

determine a number of people present in the audience based on the calculated sum total of calibrated foreground pixels.

15. The system of claim 14, wherein the processor is housed on a terminal.

16. The system of claim 15, wherein the terminal is selected from a group consisting of a personal computer, a minicomputer, a main frame computer, a microcomputer, a hand held device, and a telephonic device.

17. The system of claim 14, wherein the processor is housed on a server.

18. The system of claim 17, wherein the server is selected from a group consisting of a personal computer, a minicomputer, a microcomputer, and a main frame computer.

19. The system of claim 17, wherein the server is coupled to a network.

20. The system of claim 19, wherein the server is coupled to the network via a coupling.