Methods and apparatus for detecting on-screen media sources

• US 8,019,162 B2
• Filed: 10/24/2007
• Issued: 09/13/2011
• Est. Priority Date: 06/20/2006
• Status: Active Grant

• Current Assignee: Nielsen Company (US) LLC (RR Donnelley & Sons Company)
• Original Assignee: The Nielsen Company LLC (MOOD Media Corporation)
• Inventors: Nelson, Daniel, Zhang, Min, Dykstra, Regina, Cooper, Scott, Ramaswamy, Arun, Wright, David Howell
• Primary Examiner(s): Le; Vu
• Assistant Examiner(s): Park; Soo Jin
• Application Number:
  US11/923,141
• Publication Number:
  US 20080127253A1
• Time in Patent Office: 1,420 Days
• Field of Search: 725/35, 725/9, 725/19, 725/141, 382/199
• US Class Current: 382/199
• CPC Class Codes: G06V 20/635 Overlay text, e.g. embedded captions in a TV program
• CPC Class Codes: H04H 60/31 Arrangements for monitoring the use made of the broadcast services
• CPC Class Codes: H04H 60/37 for identifying segments of broadcast information, e.g. scenes or extracting programme ID
• CPC Class Codes: H04H 60/44 for identifying broadcast stations
• CPC Class Codes: H04H 60/45 for identifying users
• CPC Class Codes: H04H 60/48 for recognising items expressed in broadcast information
• CPC Class Codes: H04H 60/59 of video recognising characters or patterns in general G06F18/00, G06V20/00
• CPC Class Codes: H04N 21/252 Processing of multiple end-users' preferences to derive collaborative data
• CPC Class Codes: H04N 21/25883 being end-user demographical data, e.g. age, family status or address arrangements for identifying locations of users in broadcast systems H04H60/52
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/442 Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk arrangements for monitoring broadcast services or broadcast-related services H04H60/29; arrangements for identifying or recognising characteristics with a direct linkage to broadcast information H04H60/35; monitoring of user activities for profile generation for accessing a video database G06F16/739; monitoring in wireless networks H04W24/00
• CPC Class Codes: H04N 21/44204 Monitoring of content usage, e.g. the number of times a movie has been viewed, copied or the amount which has been watched monitoring of user activities for profile generation for accessing a video database G06F16/739; protecting generic digital content where the protection is independent of the precise nature of the content G06F21/10; arrangements for monitoring the use made of the broadcast services in broadcast systems H04H60/31
• CPC Class Codes: H04N 21/4667 Processing of monitored end-user data, e.g. trend analysis based on the log file of viewer selections
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 21/478 Supplemental services, e.g. displaying phone caller identification, shopping application
• CPC Class Codes: H04N 21/6582 Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number arrangements where receivers interact with the broadcast H04H20/38
• CPC Class Codes: H04N 5/4448 for frame-grabbing

Abstract

Methods and apparatus for detecting on-screen media sources are disclosed. An example method includes monitoring a region of interest of a video signal for a first image, comparing the first image to a stored image, and confirming the media device change when the stored image differs from the first image and is associated with a secondary criterion.

Claims

1. A computer implemented method to identify an active household media source device, comprising:

receiving a video signal;

analyzing a portion of the video signal associated with a display region of interest, wherein the video signal is provided to a display device by the active household media source device;

comparing the portion of the video signal to a template programmed state diagram comprising a plurality of device modes, each device mode corresponding to an image and associated with a candidate next mode, to identify the active household media source device from a plurality of household media source devices that are not providing the video signal to the display device; and

generating an output including the identified active household media source device.

2. A method as defined in claim 1, wherein the template comprises an expected video image at the region of interest.

3. A method as defined in claim 2, further comprising associating the active household media source device with the video signal when the expected video image is detected in the region of interest.

4. A method as defined in claim 1, wherein analyzing the portion of the video signal comprises quantizing the portion of the video signal to generate a quantized video signal.

5. A method as defined in claim 4, further comprising applying a quantization threshold to generate the quantized video signal, the threshold specifying at least one of a color density, a pixel intensity, or a pixel density.

6. A method as defined in claim 1, wherein analyzing the portion of the video signal comprises detecting an edge of the portion of the video signal.

7. A method as defined in claim 6, wherein the edge of the portion of the video signal comprises a quiet area surrounding the portion of the video signal.

8. A method as defined in claim 1, wherein the display device comprises a television.

9. A computer implemented method to identify video output from an alternate household media source device, comprising:

monitoring a region of interest of a video signal for a first image;

comparing the first image to a stored image programmed state diagram comprising a plurality of device modes, each device mode corresponding to an image and associated with a candidate next mode, the first image associated with one of a plurality of household media source devices capable of outputting the video signal to a display device;

confirming current video output from the alternate household media source device when the image associated with the candidate next mode differs from the first image and is associated with a secondary criterion to identify the alternate household media source device from a plurality of household media source devices that are not providing a video signal to the display device; and

generating an output including the confirmed alternate household media source device.

10. A method as defined in claim 9, wherein the secondary criterion comprises at least one of a second image located in a second region of interest, a quiet area, a presence of a third region of interest, an absence of a fourth region of interest, or a remote control signal.

11. A method as defined in claim 9, wherein the image associated with the candidate next mode comprises a first pixel sum value and the first image comprises a second pixel sum value.

12. A method as defined in claim 11, further comprising calculating a dot product of the first and second pixel sums.

13. A method as defined in claim 12, wherein the dot product yields a positive output when the image associated with the candidate next mode and the first image are similar.

14. A method as defined in claim 12, wherein the dot product yields a negative output when the image associated with the candidate next mode and the first image are different.

15. A method as defined in claim 12, wherein the dot product constructively adds when the image associated with the candidate next mode is similar to the first image, and the dot product destructively adds when the image associated with the candidate next mode is different from the first image.

16. A method as defined in claim 9, further comprising calculating a criterion to determine if the first image differs from the image associated with the candidate next mode.

17. A method as defined in claim 16, wherein the criterion comprises a ratio of a dot product of the first image and the image associated with the candidate next mode, and the number of pixels in the first image.

18. A method to identify a media device mode comprising:

monitoring a region of interest in a video signal at a first time;

comparing the monitored region of interest to a programmed state diagram comprising a plurality of device modes to determine a media device mode candidate, each device mode corresponding to an image and associated with at least one of the media device mode candidates;

monitoring the region of interest at a second time; and

comparing the monitored region of interest to the programmed state diagram to confirm the media device mode candidate as the media device mode and to identify an active household media source device from a plurality of household media source devices that are not providing the video signal to a display device.

19. A method as defined in claim 18, wherein the programmed state diagram comprises the plurality of device modes for at least one of a set-top box, a television, a video cassette recorder, a digital video disk player, a people meter, or a game console.

20. An apparatus to monitor a video signal comprising:

a region of interest extractor to extract an image signal from a region of the video signal;

an image quantizer to quantize the extracted image signal; and

an object comparator to compare the quantized image signal with a programmed mode sequence state diagram based on firmware programming of a household media device to identify an operational viewing mode and to identify an active household media source device from a plurality of household media source devices that are not providing the video signal to a display device.

21. An apparatus as defined in claim 20, further comprising a quantizer threshold module to provide a quantization threshold to the image quantizer to at least one of decrease a pixel density, alter a pixel luminosity, or remove a pixel color.

22. An apparatus to process a video signal comprising:

a framegrabber to capture the video signal at a first rate;

a media source detector to quantize the video signal to improve image detection of a first and second region of interest; and

an on screen display reader to compare detected images in the first and second regions of interest to a programmed mode sequence state diagram to identify an active household media source device from a plurality of household media source devices that are not providing the video signal to a display device when the compared images in the first and second regions of interest match a sequence of images associated with the programmed mode sequence state diagram.

23. An apparatus as defined in claim 22, wherein the media source detector further comprises a quantizer threshold to adjust at least one of a pixel density, a pixel color, or a pixel luminosity.

24. A computer implemented method, comprising:

capturing a screenshot generated by a media device;

determining image regions of interest based on an operating mode of the media device and a programmed state diagram corresponding to the media device; and

identifying a current operating mode of the media device by analyzing at least one of the image regions of interest within the screenshot to identify an active household media source device from a plurality of household media source devices that are not providing a video signal to a display device.

25. A method as defined in claim 24, further comprising identifying a plurality of candidate next operating modes based on the identified current operating mode of the media device.