Method and system for media audience measurement and spatial extrapolation based on site, display, crowd, and viewership characterization

• US 20090158309A1
• Filed: 12/12/2007
• Published: 06/18/2009
• Est. Priority Date: 12/12/2007
• Status: Abandoned Application

• Current Assignee: Videomining Corporation
• Original Assignee: Videomining Corporation
• Inventors: Jung, Namsoon, Sharma, Rajeev, Moon, Hankyu
• Application Number:
  US12/001,611
• Publication Number:
  US 20090158309A1
• Time in Patent Office: Days
• US Class Current: 725/12
• CPC Class Codes: G06V 20/53 Recognition of crowd images, e.g. recognition of crowd congestion
• CPC Class Codes: H04H 60/33 Arrangements for monitoring the users' behaviour or opinions
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45

Abstract

The present invention provides a comprehensive method to design an automatic media viewership measurement system, from the problem of sensor placement for an effective sampling of the viewership to the method of extrapolating spatially sampled viewership data. The system elements that affect the viewership—site, display, crowd, and audience—are identified first. The site-viewership analysis derives some of the crucial elements in determining an effective data sampling plan: visibility, occupancy, and viewership relevancy. The viewership sampling map is computed based on the visibility map, the occupancy map, and the viewership relevancy map; the viewership measurement sensors are placed so that the sensor coverage maximizes the viewership sampling map. The crowd-viewership analysis derives a model of the viewership in relation to the system parameters so that the viewership extrapolation can effectively adapt to the time-changing spatial distribution of the viewership; the step identifies crowd dynamics, and its invariant features as the crucial elements that extract the influence of the site, display, and the crowd to the temporal changes of viewership. The extrapolation map is formulated around these quantities, so that the site-wide viewership can be effectively estimated from the sampled viewership measurement.

Claims

1. A method for designing an automatic media measurement system based on information from visual sensors, comprising the following steps of:

a) deriving a viewership sampling map based on visibility, occupancy, and viewership relevancy pertaining to a site and a display,

b) placing viewership measurement sensors based on the viewership sampling map,

c) measuring a sampled viewership in a first time period by processing the visual data from the viewership measurement sensors using a viewership measurement algorithm,

d) determining a time-varying viewership extrapolation map based on crowd dynamics invariant features extracted from a crowd dynamics measurement, and

e) estimating a site-wide viewership of a second time period based on the sampled viewership, using the viewership extrapolation map.

2. The method according to claim 1, wherein the method further comprises a step of deriving the viewership sampling map based on

a) a visibility map derived from a display-human perception analysis,

b) an occupancy map derived from a crowd occupancy analysis, and

c) a viewership relevancy map derived from a comparison of ground truth viewership data and an initial viewership measurement.

3. The method according to claim 2, wherein the display-human perception analysis further comprises a step of finding a relationship between display characteristics and a degree of the ease of viewing in relation to the position of a human audience relative to the display,

wherein the visibility map is computed given a position and a direction of the display.

4. The method according to claim 2, wherein the crowd occupancy analysis further comprises a step of predicting a crowd flow based on the site constraints and computing the occupancy map,

whereby the crowd occupancy analysis measures the crowd flow over time and computes the occupancy map based on time-integrated measured data.

5. The method according to claim 2, wherein the method further comprises a step of computing a statistical correlation between the ground truth viewership data and the initial viewership measurement using the viewership measurement algorithm and the viewership measurement sensors, for computing the viewership relevancy map.

6. The method according to claim 1, wherein the method further comprises a step of placing the viewership measurement sensors so that a spatial coverage of the sensors optimizes the viewership sampling map,

whereby the optimization is constrained by the sensor placement constraints imposed by the physical structure of the site, the cost of an installation, and the requirement to hide the viewership measurement sensors.

7. The method according to claim 1, wherein the crowd dynamics measurement further comprises a step of computing a crowd velocity tensor field at each sampling grid on the site floor by computing the covariance matrix of the crowd motion vectors.

8. The method according to claim 7, wherein the method further comprises a step of estimating the crowd motion vectors using dense optical field estimation.

9. The method according to claim 7, wherein the method further comprises a step of estimating the crowd motion vectors by tracking each person in the crowd and computing the trajectories.

10. The method according to claim 7, wherein the method further comprises a step of computing the crowd motion anisotropy at each sampling grid by dividing the first Eigenvalue by the sum of the two Eigenvalues.

11. The method according to claim 7, wherein the crowd dynamics measurement further comprises a step of utilizing at least a top-down view visual sensor.

12. The method according to claim 7, wherein the crowd dynamics measurement further comprises a step of utilizing the viewership measurement sensors.

13. The method according to claim 7, wherein the method further comprises a step of utilizing the histogram of the dynamic features of the crowd dynamics at each sampling grid, for the crowd dynamics invariant features.

14. The method according to claim 7, wherein the crowd dynamics invariant features extraction further comprises a step of computing the crowd dynamics distribution over the site,

whereby the crowd dynamics distribution is processed by computing the joint histogram of crowd motion anisotropy and the average speed of the crowd over the sampling grid.

15. The method according to claim 1, wherein the method further comprises a step of formulating the viewership extrapolation map as parameterized by the crowd dynamics invariant features of the crowd dynamics.

16. The method according to claim 1, wherein the method further comprises a step of determining the viewership extrapolation map by estimating the viewership map in each time period.

17. The method according to claim 16, wherein the method further comprises a step of representing the viewership map by the sampled viewership to the site-wide viewership ratio,

wherein the viewership extrapolation map is a multiplication by a time-varying constant whose value is an estimated value of the sampled viewership to the site-wide viewership ratio.

18. The method according to claim 16, wherein the method further comprises a step of learning a functional relation between the crowd dynamics and the viewership map by training a learning machine.

19. The method according to claim 1, wherein the viewership measurement algorithm further comprises the following steps of:

a) employing face detection to find faces in captured video frames,

b) employing face tracking to keep the identities of an audience in the scene, and

c) estimating a 3-dimensional facial pose to detect the viewership of an audience.

20. An apparatus for designing an automatic media measurement system based on information from visual sensors, comprising:

a) means for deriving a viewership sampling map based on visibility, occupancy, and viewership relevancy pertaining to a site and a display,

b) means for placing viewership measurement sensors based on the viewership sampling map,

c) means for measuring a sampled viewership in a first time period by processing the visual data from the viewership measurement sensors using a viewership measurement algorithm,

d) means for determining a time-varying viewership extrapolation map based on crowd dynamics invariant features extracted from a crowd dynamics measurement, and

e) means for estimating a site-wide viewership of a second time period based on the sampled viewership, using the viewership extrapolation map.

21. The apparatus according to claim 20, wherein the apparatus further comprises means for deriving the viewership sampling map based on

a) a visibility map derived from a display-human perception analysis,

b) an occupancy map derived from a crowd occupancy analysis, and

c) a viewership relevancy map derived from a comparison of a ground truth viewership data and an initial viewership measurement.

22. The apparatus according to claim 21, wherein the display-human perception analysis further comprises means for finding a relationship between display characteristics and a degree of the ease of viewing in relation to the position of a human audience relative to the display,

wherein the visibility map is computed given a position and a direction of the display.

23. The apparatus according to claim 21, wherein the crowd occupancy analysis further comprises means for predicting a crowd flow based on the site constraints and computing the occupancy map,

whereby the crowd occupancy analysis measures the crowd flow over time and computes the occupancy map based on time-integrated measured data.

24. The apparatus according to claim 21, wherein the apparatus further comprises means for computing a correlation between the ground truth viewership data and the initial viewership measurement using the viewership measurement algorithm and the viewership measurement sensors for the viewership relevancy map.

25. The apparatus according to claim 20, wherein the apparatus further comprises means for placing the viewership measurement sensors so that a spatial coverage of the sensors optimizes the viewership sampling map,

whereby the optimization is constrained by the sensor placement constraints imposed by the physical structure of the site and the cost of an installation.

26. The apparatus according to claim 20, wherein the crowd dynamics measurement further comprises means for computing a crowd velocity tensor field at each sampling grid on the site floor by computing the covariance matrix of the crowd motion vectors.

27. The apparatus according to claim 26, wherein the apparatus further comprises means for estimating the crowd motion vectors using a dense optical field estimation.

28. The apparatus according to claim 26, wherein the apparatus further comprises means for estimating the crowd motion vectors by tracking each person in the crowd and computing the trajectories.

29. The apparatus according to claim 26, wherein the apparatus further comprises means for computing the crowd motion anisotropy at each sampling grid by dividing the first Eigenvalue by the sum of the two Eigenvalues.

30. The apparatus according to claim 26, wherein the crowd dynamics measurement further comprises means for utilizing at least a top-down view visual sensor.

31. The apparatus according to claim 26, wherein the crowd dynamics measurement further comprises means for utilizing the viewership measurement sensors.

32. The apparatus according to claim 26, wherein the apparatus further comprises means for utilizing the histogram of the dynamic features of the crowd dynamics at each sampling grid for the crowd dynamics invariant features.

33. The apparatus according to claim 26, wherein the crowd dynamics invariant features extraction further comprises means for computing the crowd dynamics distribution over the site,

whereby the crowd dynamics distribution is processed by computing the joint histogram of crowd motion anisotropy and the average speed of the crowd over the sampling grid.

34. The apparatus according to claim 20, wherein the apparatus further comprises means for formulating the viewership extrapolation map as parameterized by the crowd dynamics invariant features of the crowd dynamics.

35. The apparatus according to claim 20, wherein the apparatus further comprises means for determining the viewership extrapolation map by estimating the viewership map in each time period.

36. The apparatus according to claim 35, wherein the apparatus further comprises means for representing the viewership map by the sampled viewership to the site-wide viewership ratio,

wherein the viewership extrapolation map is a multiplication by a time-varying constant whose value is an estimated value of the sampled viewership to the site-wide viewership ratio.

37. The apparatus according to claim 35, wherein the apparatus further comprises means for learning a functional relation between the crowd dynamics and the viewership map by training a learning machine.

38. The apparatus according to claim 20, wherein the viewership measurement algorithm further comprises:

a) means for employing face detection to find faces in captured video frames,

b) means for employing face tracking to keep the identities of audience in the scene, and

c) means for estimating a 3-dimensional facial pose to detect the viewership of an audience,

whereby the viewership measurement sensors comprise video cameras.