Video entity recognition in compressed digital video streams

• US 20060187358A1
• Filed: 04/04/2006
• Published: 08/24/2006
• Est. Priority Date: 03/07/2003
• Status: Active Grant

• Current Assignee: RPX Corporation
• Original Assignee: Technology Patents & Licensing Incorporated
• Inventors: Eldering, Charles A., Lienhart, Rainer W.
• Granted Patent: US 7,809,154 B2
• Time in Patent Office: Days
• US Class Current: 348/661
• CPC Class Codes: G06F 16/785 using colour or luminescence
• CPC Class Codes: G06F 16/7864 using domain-transform features, e.g. DCT or wavelet transform coefficients
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0241 Advertisements
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 20/48 Matching video sequences
• CPC Class Codes: G06V 20/635 Overlay text, e.g. embedded captions in a TV program
• CPC Class Codes: H04H 60/37 for identifying segments of broadcast information, e.g. scenes or extracting programme ID
• CPC Class Codes: H04H 60/59 of video recognising characters or patterns in general G06F18/00, G06V20/00
• CPC Class Codes: H04N 21/2143 located in a single building, e.g. hotel, hospital or museum arrangements specially adapted for plural spots in a confined site in broadcast systems H04H20/63; adaptations for transmission by electric cable for domestic distribution in television systems H04N7/106
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/2547 Third Party Billing, e.g. billing of advertiser
• CPC Class Codes: H04N 21/25891 being end-user preferences retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/2668 Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063
• CPC Class Codes: H04N 21/41415 involving a public display, viewable by several users in a public space outside their home, e.g. movie theatre, information kiosk
• CPC Class Codes: H04N 21/4331 Caching operations, e.g. of an advertisement for later insertion during playback
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/44016 involving splicing one content stream with another content stream, e.g. for substituting a video clip
• CPC Class Codes: H04N 21/458 : Scheduling content for creating a personalised stream, e.g. by combining a locally stored advertisement with an incoming stream; Updating operations, e.g. for OS modules ; time-related management operations arrangements for replacing or switching information during the broadcast or during the distribution H04H20/10
• CPC Class Codes: H04N 21/812 : involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/8352 : involving content or source identification data, e.g. Unique Material Identifier [UMID]
• CPC Class Codes: H04N 21/84 : Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/8405 : represented by keywords
• CPC Class Codes: H04N 21/8455 : involving pointers to the content, e.g. pointers to the I-frames of the video stream
• CPC Class Codes: H04N 21/8456 : by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 21/8547 : involving timestamps for synchronizing content
• CPC Class Codes: H04N 7/088 : the inserted signal being digital
• CPC Class Codes: H04N 7/0887 : for the transmission of programme or channel identifying signals
• CPC Class Codes: H04N 7/162 : Authorising the user terminal, e.g. by paying; Registering the use of a subscription channel, e.g. billing

Abstract

A method and system for detection of video segments in compressed digital video streams is presented. The compressed digital video stream is examine to determine synchronization points, and the compressed video signal is analyzed following detection of the synchronization points to create video fingerprints that are subsequently compared against a library of stored fingerprints.

Claims

1. A method of detecting a known video entity within a compressed digital video stream, the method comprising:

(a) receiving the compressed digital video stream;

(b) determining synchronization points within the compressed digital video stream;

(c) creating statistical parameterized representations of the compressed digital video stream for windows following the synchronization points in the video stream;

(d) comparing the statistical parameterized representation windows to windows of a plurality of fingerprints, wherein each of the plurality of fingerprints includes associated statistical parameterized representations of a known video entity; and

(e) detecting a known video entity in the compressed digital video stream when at least one of the plurality of fingerprints has at least a threshold level of similarity to a fingerprint of the video stream.

2. The method of claim 1, wherein the synchronization points are obtained using timing indicators within the compressed digital video stream.

3. The method of claim 1, wherein the compressed digital video stream is spatially compressed.

4. The method of claim 1, wherein the statistical parameterized representations are based on coefficient histograms.

5. The method of claim 1, wherein the statistical parameterized representations are based on weighted coefficient histograms.

6. The method of claim 1, wherein the statistical parameterized representations are based on code word histograms.

7. The method of claim 1, wherein the statistical parameterized representations are based on histograms of the separation of the end of blocks.

8. The method of claim 1, wherein the compressed digital video stream is temporally compressed.

9. The method of claim 1, wherein the statistical parameterized representations are based on histograms of I frame separations.

10. The method of claim 1, wherein the statistical parameterized representations are based on the number of B, P, and I frames within a time segment.

11. The method of claim 1, wherein the statistical parameterized representations are based on motion compensation vectors.

12. The method of claim 1, wherein the statistical parameterized representations are spatially reduced coefficients.

13. The method of claim 12, wherein the spatially reduced coefficients are based on the DC coefficients derived from I, P, or B frames of the compressed digital video stream.

14. The method of claim 12, wherein the spatially reduced coefficients are based on a linear combination of DC and AC coefficients derived from I, P, or B frames of the compressed digital video stream.

15. The method of claim 1, further comprising:

(f) processing uncompressed video when a particular fingerprint of the plurality of fingerprints has at least a threshold level of similarity to the video stream.

16. The method of claim 1, wherein the known video entities are advertisements.

17. The method of claim 1, wherein the known video entities are at least some subset of advertisement intros, advertisement outros, channel idents, and sponsorships.

18. The method of claim 1, wherein said detecting of step (e) includes detecting an advertisement opportunity in the video stream when said comparing of step (d) indicates that a fingerprint associated with an advertisement or an advertisement intro has at least a threshold level of similarity with the statistical parameterized representations of the video stream.

19. The method of claim 1, further comprising:

(f) inserting a targeted advertisement subsequent to said detecting an advertisement opportunity in the video stream.

20. A method for detecting a known video entity within a video stream, the method comprising:

receiving a video stream;

continually creating statistical parameterized representations for windows of the video stream;

continually comparing the statistical parameterized representation windows to windows of a plurality of fingerprints, wherein each of the plurality of fingerprints includes associated statistical parameterized representations of a known video entity; and

detecting a known video entity in the video stream when a particular fingerprint of the plurality of fingerprints has at least a threshold level of similarity with the video stream.

21. A system for detecting a known video entity within a compressed digital video stream, the system comprising:

a receiver to receive a compressed digital video stream;

memory for storing a plurality of fingerprints, wherein each of the plurality of fingerprints includes associated statistical parameterized representations of a known video entity; and

a processor configured to determine synchronization points within a compressed digital video stream;

create statistical parameterized representations of the compressed digital video stream for windows following the synchronization points;

compare the statistical parameterized representation windows following the synchronization points to windows of the plurality of fingerprints; and

detect a known video entity in the video stream when at least one of the plurality of fingerprints has at least a threshold level of similarity to a fingerprint of the video stream.

22. The system of claim 21, wherein the synchronization points are obtained points are obtained using timing indicators within the compressed digital video stream.

23. The system of claim 21, wherein the statistical parameterized representations include at least some subset of coefficient histograms, weighted coefficient histograms, code word histograms, histograms of separation of end blocks, histograms of I frame separations, the number of B, P, and I frames within a time segment, motion compensation vectors, and spatially reduced coefficients.

24. The system of claim 21, wherein the known video segments include at least some subset of advertisements, advertisement intros, sponsorship messages, advertisement outros, and channel idents.

25. The system of claim 21, wherein said processor detects an advertisement opportunity in the video stream when the comparing indicates that a fingerprint associated with an advertisement or an advertisement intro has at least a threshold level of similarity with the statistical parameterized representations of the video stream.

26. The system of claim 25, further comprising:

a video inserter to insert targeted advertisements once the incoming video stream is associated with a known advertisement.

27. An article of manufacture for detecting a known video entity within a compressed digital video stream, the article of manufacture comprising a computer-readable medium holding computer-executable instructions for performing a method comprising:

(a) receiving the compressed digital video stream;

(b) determining synchronization points within the compressed digital video stream;

(c) creating statistical parameterized representations of the compressed digital video stream for windows following the synchronization points in the video stream;

(d) comparing the statistical parameterized representation windows to windows of a plurality of fingerprints, wherein each of the plurality of fingerprints includes associated statistical parameterized representations of a known video entity; and

(e) detecting a known video entity in the compressed digital video stream when at least one of the plurality of fingerprints has at least a threshold level of similarity to a fingerprint of the video stream.

28. The article of manufacture of claim 27, the computer-executable instructions performing a method wherein the synchronization points are obtained using timing indicators within the compressed digital video stream.

29. The article of manufacture of claim 27, the computer-executable instructions performing a method wherein the compressed digital video stream is spatially compressed.

30. The article of manufacture of claim 27, the computer-executable instructions performing a method wherein the statistical parameterized representations include at least some subset of coefficient histograms, weighted coefficient histograms, code word histograms, histograms of separation of end blocks, histograms of I frame separations, the number of B, P, and I frames within a time segment, motion compensation vectors, and spatially reduced coefficients.

31. The article of manufacture of claim 27, the computer-executable instructions performing a method further comprising:

(f) processing uncompressed video when a particular fingerprint of the plurality of fingerprints has at least a threshold level of similarity to the video stream.

32. The article of manufacture of claim 27, the computer-executable instructions performing a method wherein the known video entities are advertisements.

33. The article of manufacture of claim 27, the computer-executable instructions performing a method wherein the known video entities are at least some subset of advertisement intros, advertisement outros, channel idents, and sponsorships.

34. The article of manufacture of claim 27, the computer-executable instructions performing a method wherein said detecting of step (e) includes detecting an advertisement opportunity in the video stream when said comparing of step (d) indicates that a fingerprint associated with an advertisement or an advertisement intro has at least a threshold level of similarity with the statistical parameterized representations of the video stream.

35. The article of manufacture of claim 27, the computer-executable instructions performing a method further comprising:

(f) inserting a targeted advertisement subsequent to said detecting an advertisement opportunity in the video stream.