Detecting a composition of an audience

• US 7,609,853 B2
• Filed: 12/11/2002
• Issued: 10/27/2009
• Est. Priority Date: 12/11/2002
• Status: Active Grant

• Current Assignee: Nielsen Company (US), LLC
• Original Assignee: The Nielsen Company LLC (MOOD Media Corporation)
• Inventors: Nelson, Daniel J., Ramaswamy, Arun, Srinivasan, Venugopal
• Primary Examiner(s): Alavi; Amir
• Application Number:
  US10/538,692
• Publication Number:
  US 20060200841A1
• Time in Patent Office: 2,512 Days
• Field of Search: 382/103, 382/115, 382/116, 382/118, 382/154, 382/168, 382/227, 382/236, 382/274, 382/284, 725/1, 725/10, 725/12, 725/14, 725/15, 367/87, 367/104, 348/E7.07, 348/E7.083, 348/E17.001
• US Class Current: 382/103
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: H04H 60/33 Arrangements for monitoring the users' behaviour or opinions
• CPC Class Codes: H04H 60/43 for identifying broadcast channels
• CPC Class Codes: H04H 60/45 for identifying users
• CPC Class Codes: H04N 21/258 Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data arrangements for services using the result on the distributing side of broadcast systems H04H60/66; profiles in network data switching protocols H04L67/30
• CPC Class Codes: H04N 21/42201 biosensors, e.g. heat sensor for presence detection, EEG sensors or any limb activity sensors worn by the user input arrangements for interaction with the human body based on nervous system activity detection G06F3/015
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/433 Content storage operation, e.g. storage operation in response to a pause request, caching operations
• CPC Class Codes: H04N 21/4335 Housekeeping operations, e.g. prioritizing content for deletion because of storage space restrictions storage management, e.g. defragmentation G06F3/0604; unloading stored programs G06F9/445; storage management in file systems G06F16/10; buffering arrangements in a network node or in an end terminal in packet networks H04L49/90
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/44222 Analytics of user selections, e.g. selection of programs or purchase activity monitoring of user selections in data processing systems G06F11/34; arrangements for monitoring the user's behaviour or opinions in broadcast systems H04H60/33
• CPC Class Codes: H04N 7/17309 Transmission or handling of upstream communications

Abstract

Methods and apparatus for detecting a composition of an audience of an information presenting device are disclosed. One disclosed method captures images of the audience, determines a number of people within the images and prompts the audience to identify its members if a change in the number of people in the audience is visually detected. This disclosed method avoids both under sampling of the audience and excessive prompting of the audience by only requesting entry of identification data when an audience change occurs. Methods and apparatus to determine the number of persons in an image or series of image are also disclosed.

Claims

1. A method of automatically detecting a composition of an audience of an information presenting device comprising:

capturing at least one image of the audience;

determining a number of people within the at least one image; and

prompting the audience to identify its members if a change in the number of people is detected based on the number of people determined to be within the at least one image, wherein determining the number of people within the at least one image comprises;

(a) determining at least one difference image based on at least two captured images;

(b) developing at least one shape from the at least one difference image;

(c) identifying at least one center of the at least one shape;

(d) adding a symbol having a predetermined size and indicative of the at least one center of the at least one shape to a histogram;

(e) repeating (a)-(d); and

(f) if any symbol in the histogram grows beyond a predetermined threshold, counting that symbol as a person.

2. A method as defined in claim 1 wherein capturing the at least one image of the audience comprises digitizing the at least one image.

3. A method as defined in claim 1 wherein determining the at least one difference image based on the at least two of the captured images comprises:

(a) determining a first difference image between a first image and a second image; and

(b) determining a second difference image between the second image and a third image.

4. A method as defined in claim 3 further comprising developing an amalgamated image from the first and second difference images.

5. A method as defined in claim 4 wherein developing the at least one shape from the at least one difference image comprises developing the at least one shape from the amalgamated image.

6. A method as defined in claim 1 wherein identifying the at least one center of the at least one shape comprises identifying a center of a first shape and a center of a second shape.

7. A method as defined in claim 6 wherein adding the symbol having the predetermined size and indicative of the at least one center of the at least one shape to the histogram comprises adding a first symbol indicative of the center of the first shape to the histogram and adding a second symbol indicative of the center of the second shape to the histogram.

8. A method as defined in claim 7 wherein repeating (a)-(d) comprises stacking a third symbol on the first symbol if a second difference image contains a shape having a center that substantially corresponds to the center of the first shape.

9. A method as defined in claim 1 further comprising excluding a shape from a group of possible human shapes based on a test.

10. A method as defined in claim 9 wherein the test comprises at least one of a location test and a size test.

11. A method as defined in claim 1 further comprising identifying an energy value associated with the at least one difference image, and, performing (f) if the energy value exceeds a predetermined threshold.

12. A method as defined in claim 1 wherein developing the at least one shape from the at least one difference image comprises executing a convex hull process.

13. A method as defined in claim 1 wherein, if any symbol in the histogram does not grow within a predetermined length of time, that symbol is eliminated from the histogram.

14. A method as defined in claim 1 further comprising:

if a number of members identified by the audience is different from the determined number of people after a predetermined number of prompts of the audience, adjusting a value to avoid excessive prompting of the audience.

15. A method as defined in claim 14 wherein adjusting the value comprises increasing a previous audience count by a difference between the number of members identified by the audience and the number of people determined from the at least one image.

16. A method as defined in claim 15 further comprising recording the difference between the number of members identified by the audience and the number of people determined from the at least one image as a number of unidentified audience members.

17. A method as defined in claim 1 further comprising periodically exporting recorded data to a device to identify audiences and programs being consumed by the audiences.

18. A method as defined in claim 1 further comprising identifying a program being consumed by the audience.

19. A method as defined in claim 18 wherein identifying a program being consumed by the audience further comprises:

identifying a consumption time; and

identifying a source of the program being consumed by the audience.

20. A method as defined in claim 19 further comprising recording the source and the consumption time.

21. A method as defined in claim 19 wherein identifying the source of the program comprises identifying a tuned channel.

22. A method as defined in claim 19 further comprising using the consumption time and the source to identify the program from a program guide.

23. An apparatus to detect a composition of an audience of an information presenting device comprising:

a camera to capture at least one image of the audience;

a processor;

a memory storing computer readable instructions which, when executed, cause the processor to;

determine a number of people within the at least one image; and

develop a prompt signal requesting the audience to identify its members if a change in the number of people is visually detected; and

an output device responsive to the prompt signal to output an indication requesting the audience to identify its members, wherein the processor determines the number of people within the at least one image by;

(a) determining at least one difference image between at least two images captured by the camera;

(b) developing at least one shape from the at least one difference image;

(c) identifying at least one center of the at least one shape;

(d) adding a symbol having a predetermined size and indicative of the at least one center of the at least one shape to a histogram;

(e) repeating (a)-(d); and

(f) if any symbol in the histogram grows beyond a predetermined threshold, counting the symbol as a person.

24. An apparatus as defined in claim 23 wherein the output device is a visual display and the prompt signal is a visual signal.

25. An apparatus as defined in claim 23 wherein the camera digitizes the at least one image.

26. An apparatus as defined in claim 23 wherein, if a number of members identified by the audience is different from the determined number of people after a predetermined number of prompts of the audience, the computer readable instructions cause the processor to adjust a value to avoid excessive prompting of the audience.

27. An apparatus as defined in claim 26 wherein the processor adjusts the value by increasing a previous audience count by a difference between the number of members identified by the audience and the number of people determined from the at least one image.

28. An apparatus as defined in claim 27 wherein the computer readable instructions cause the processor to record the difference between the number of members identified by the audience and the number of people determined from the at least one image as a number of unidentified audience members.

29. An apparatus as defined in claim 23 wherein the computer readable instructions cause the processor to periodically export recorded data to a device to identify audiences and programs being consumed by the audiences.

30. An apparatus as defined in claim 23 wherein the computer readable instructions cause the processor to identify the program being consumed by the audience.

31. An apparatus as defined in claim 30 wherein identifying a program being consumed by the audience further comprises:

identifying a consumption time; and

identifying a source of the program being consumed by the audience.

32. An apparatus as defined in claim 31 wherein the computer readable instructions cause the processor to record the source and the consumption time.

33. An apparatus as defined in claim 31 wherein identifying the source of the program further comprises identifying a tuned channel.

34. An apparatus as defined in claim 31 further comprising using the consumption time and the source to identify the program from a program guide.

35. A computer readable medium storing computer readable instructions which, when executed, cause at least one computer to:

capture at least one image of an audience consuming a program;

determine a number of people within the at least one image; and

output a prompt signal requesting the audience to identify its members if a change in the number of people is detected from the at least one image, wherein the at least one computer determines the number of people within the at least one image by;

(a) determining at least one difference image between at least two captured images;

(b) developing at least one shape from the at least one difference image;

(c) identifying at least one center of the at least one shape;

(d) adding a symbol having a predetermined size and indicative of the at least one center of the at least one shape to a histogram;

(e) repeating (a)-(d); and

(f) if any symbol in the histogram grows beyond a predetermined threshold, counting the symbol as a person.

36. A computer readable medium as defined in claim 35 wherein, if a number of members identified by the audience is different from the determined number of people after a predetermined number of prompts of the audience, the computer readable instructions cause the at least one computer to adjust a value to avoid excessive prompting of the audience.

37. A computer readable medium as defined in claim 36 wherein the at least one computer adjusts the value by increasing a previous audience count by a difference between the number of members identified by the audience and the number of people determined from the at least one image.

38. A computer readable medium as defined in claim 37 wherein the computer readable instructions cause the at least one computer to record the difference between the number of members identified by the audience and the number of people determined from the at least one image as a number of unidentified audience members.

39. A computer readable medium as defined in claim 35 wherein the computer readable instructions cause the at least one computer to periodically export recorded data to a device to identify audiences and programs being consumed by the audiences.

40. A computer readable medium as defined in claim 35 wherein the computer readable instructions cause the at least one computer to identify the program being consumed by the audience.

41. A computer readable medium as defined in claim 40 wherein identifying a program being consumed by the audience further comprises:

identifying a consumption time; and

identifying a source of the program being consumed by the audience.

42. A computer readable medium as defined in claim 41 wherein the computer readable instructions cause the at least one computer to record the source and the consumption time.

43. A computer readable medium as defined in claim 41 wherein identifying the source of the program comprises identifying a tuned channel.

44. A computer readable medium as defined in claim 41 further comprising using the consumption time and the source to identify the program from a program guide.