Video identification through detection of proprietary rights logos in media

• US 8,175,413 B1
• Filed: 03/05/2009
• Issued: 05/08/2012
• Est. Priority Date: 03/05/2009
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ioffe, Sergey, Kulewski, Krzysztof
• Primary Examiner(s): PERUNGAVOOR, SATHYANARAYA V
• Application Number:
  US12/398,868
• Time in Patent Office: 1,160 Days
• Field of Search: 382/283
• US Class Current: 382/283
• CPC Class Codes: G06V 10/10 Image acquisition document image scanning and transmission H04N1/00; control of digital cameras H04N23/60
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/635 Overlay text, e.g. embedded captions in a TV program

Abstract

Proprietary rights logos are detected in a video. The video is divided into a plurality of regions that are analyzed for generic proprietary rights logo features. A confidence mask is generated that comprises a plurality of scaling factors, each scaling factor corresponding to a region of the video and indicating a likelihood that the corresponding region of the video includes a proprietary rights logo. The scaling factors of the confidence mask are applied to the video data to generate an altered video. The altered video is analyzed to determine a confidence measure that the video includes a reference proprietary rights logo.

Claims

1. A method for detecting at least one known logo in a video, the video comprising a plurality of regions, each region comprising video data, the method executed by at least one computer system and comprising:

generating a confidence mask comprising a plurality of scaling factors, each scaling factor corresponding to a region of the video and indicating a likelihood that the corresponding region of the video includes a logo;

applying the scaling factors of the confidence mask to the video data for the corresponding regions of the video to generate an altered video;

identifying an interest point in the video;

computing a feature vector for the interest point, wherein the feature vector is a vector of real numbers describing visual characteristics of the video proximate to the interest point;

accessing a database of stored feature vectors for known logos; and

determining from the altered video a confidence measure as to whether the video includes a known logo by calculating a degree of similarity between the stored feature vector of the known logo and the computed feature vector for the video.

2. The method of claim 1, wherein generating a confidence mask comprises determining whether pixel values for a first group of pixels are substantially constant over a duration of time relative to pixel values for a second group of pixels proximate to the first group of pixels.

3. The method of claim 1, wherein generating a confidence mask comprises determining whether a region of the video includes an edge.

4. The method of claim 3, wherein determining whether a region of the video includes an edge comprises calculating a spatial or temporal gradient for pixel values.

5. The method of claim 3, wherein determining if a region of the video includes an edge comprises:

calculating time-averaged pixel values; and

calculating a spatial or temporal gradient for time-averaged pixel values.

6. The method of claim 1, wherein generating a confidence mask comprises:

calculating a motion vector for a pixel; and

determining whether a magnitude of the motion vector is below a threshold.

7. The method of claim 1, further comprising applying a noise filter to the video to create a reduced-noise video file.

8. The method of claim 1, wherein applying the scaling factors of the confidence mask to the video data for the corresponding regions of the video to generate an altered video further comprises scaling pixels values for a region by an associated scaling factor.

9. The method of claim 8, wherein scaling pixels values for a region by an associated scaling factor comprises performing a pixel-wise multiplication of pixel values in the region by the associated scaling factor.

10. The method of claim 1, wherein determining from the altered video a confidence measure as to whether the video includes a known logo further comprises determining that an observed position, scale, or orientation of the interest point corresponds to a known position, scale, or orientation of a known logo.

11. The method of claim 10, wherein determining that an observed position, scale, or orientation of the interest point corresponds to a known position, scale, or orientation of the known logo comprises:

overlaying a frame of the altered video including the interest point with a stored template for the known logo; and

computing a degree of correlation between the stored template and the frame of the altered video.

12. The method of claim 1, further comprising calculating a duration of appearance for the known logo.

13. The method of claim 12, further comprising outputting a logo report including at least one selected from the group consisting of:

a number of known logos detected, the confidence measure for each detected known logo, and the duration of appearance for each detected known logo.

14. A non-transitory computer-readable storage medium storing computer executable code, the computer-executable code when executed by a processor causing the processor to perform a process for detecting at least one known logo in a video, the video comprising a plurality of regions, each region comprising video data, the process comprising:

generating a confidence mask comprising a plurality of scaling factors, each scaling factor corresponding to a region of the video and indicating a likelihood that the corresponding region of the video includes a logo;

applying the scaling factors of the confidence mask to the video data for the corresponding regions of the video to generate an altered video;

identifying an interest point in the video;

computing a feature vector for the interest point, wherein the feature vector is a vector of real numbers describing visual characteristics of the video proximate to the interest point;

accessing a database of stored feature vectors for known logos; and

determining from the altered video a confidence measure as to whether the video includes a known logo by calculating a degree of similarity between a stored feature vector and the computed feature vector for the video.

15. The computer-readable storage medium of claim 14, wherein generating a confidence mask comprises determining whether pixel values for a first group of pixels are substantially constant over a duration of time relative to pixel values for a second group of pixels proximate to the first group of pixels.

16. The computer-readable storage medium of claim 14, wherein generating a confidence mask comprises determining whether a region of the video includes an edge.

17. The computer-readable storage medium of claim 16, wherein determining whether a region of the video includes an edge comprises calculating a spatial or temporal gradient for pixel values.

18. The computer-readable storage medium of claim 16, wherein determining if a region of the video includes an edge comprises:

calculating time-averaged pixel values; and

calculating a spatial or temporal gradient for time-averaged pixel values.

19. The computer-readable storage medium of claim 14, wherein generating a confidence mask comprises:

calculating a motion vector for a pixel; and

determining whether a magnitude of the motion vector is below a threshold.

20. The computer-readable storage medium of claim 14, the process further comprising applying a noise filter to the video to create a reduced-noise video file.

21. The computer-readable storage medium of claim 14, wherein applying the scaling factors of the confidence mask to the video data for the corresponding regions of the video to generate an altered video further comprises scaling pixels values for a region by an associated scaling factor.

22. The computer-readable storage medium of claim 21, wherein scaling pixels values for a region by an associated scaling factor comprises performing a pixel-wise multiplication of pixel values in the region by the associated scaling factor.

23. The computer-readable storage medium of claim 14, wherein determining from the altered video a confidence measure as to whether the video includes a known logo further comprises determining that an observed position, scale, or orientation of the interest point corresponds to a known position, scale, or orientation of a known logo.

24. The computer-readable storage medium of claim 23, wherein determining that an observed position, scale, or orientation of the interest point corresponds to a known position, scale, or orientation of the known logo comprises:

overlaying a frame of the altered video including the interest point with a stored template for the known logo; and

computing a degree of correlation between the stored template and the frame of the altered video.

25. The computer-readable storage medium of claim 24, further comprising outputting a logo report including at least one selected from the group consisting of:

a number of known logos detected, the confidence measure for each detected known logo, and the duration of appearance for each detected known logo.

26. The computer-readable storage medium of claim 14, further comprising calculating a duration of appearance for the known logo.