Video entity recognition in compressed digital video streams

• US 8,374,387 B2
• Filed: 11/16/2011
• Issued: 02/12/2013
• Est. Priority Date: 03/07/2003
• Status: Active Grant

• Current Assignee: RPX Corporation
• Original Assignee: Technology Patents & Licensing Incorporated
• Inventors: Eldering, Charles A., Lienhart, Rainer W.
• Primary Examiner(s): CHAWAN, SHEELA C
• Application Number:
  US13/297,576
• Publication Number:
  US 20120063636A1
• Time in Patent Office: 454 Days
• Field of Search: 382/100, 382/181, 382/232, 382/233, 382/168, 382/124, 348/E5.005, 348/E5.002, 348/E7.071, 348/E5.009, 348/E5.108, 348/E7.07, 348/E5.137, 348/E5.109, 348/335, 348/E5.007, 348/E7.036, 375/240.28, 375/240.25, 375/E7.211, 375/E7.024, 375/E7.025, 375/E7.268, 375/240.01, 375/E7.267, 375/E7.094, 375/E7.129, 375/E7.25, 375/E7.278, 375/E7.222, 725/32, 725/33, 725/34, 725/35, 725/36
• US Class Current: 382/100
• CPC Class Codes: G06F 16/785 using colour or luminescence
• CPC Class Codes: G06F 16/7864 using domain-transform features, e.g. DCT or wavelet transform coefficients
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0241 Advertisements
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 20/48 Matching video sequences
• CPC Class Codes: G06V 20/635 Overlay text, e.g. embedded captions in a TV program
• CPC Class Codes: H04H 60/37 for identifying segments of broadcast information, e.g. scenes or extracting programme ID
• CPC Class Codes: H04H 60/59 of video recognising characters or patterns in general G06F18/00, G06V20/00
• CPC Class Codes: H04N 21/2143 located in a single building, e.g. hotel, hospital or museum arrangements specially adapted for plural spots in a confined site in broadcast systems H04H20/63; adaptations for transmission by electric cable for domestic distribution in television systems H04N7/106
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/2547 Third Party Billing, e.g. billing of advertiser
• CPC Class Codes: H04N 21/25891 being end-user preferences retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/2668 Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063
• CPC Class Codes: H04N 21/41415 involving a public display, viewable by several users in a public space outside their home, e.g. movie theatre, information kiosk
• CPC Class Codes: H04N 21/4331 Caching operations, e.g. of an advertisement for later insertion during playback
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/44016 involving splicing one content stream with another content stream, e.g. for substituting a video clip
• CPC Class Codes: H04N 21/458 : Scheduling content for creating a personalised stream, e.g. by combining a locally stored advertisement with an incoming stream; Updating operations, e.g. for OS modules ; time-related management operations arrangements for replacing or switching information during the broadcast or during the distribution H04H20/10
• CPC Class Codes: H04N 21/812 : involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/8352 : involving content or source identification data, e.g. Unique Material Identifier [UMID]
• CPC Class Codes: H04N 21/84 : Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/8405 : represented by keywords
• CPC Class Codes: H04N 21/8455 : involving pointers to the content, e.g. pointers to the I-frames of the video stream
• CPC Class Codes: H04N 21/8456 : by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 21/8547 : involving timestamps for synchronizing content
• CPC Class Codes: H04N 7/088 : the inserted signal being digital
• CPC Class Codes: H04N 7/0887 : for the transmission of programme or channel identifying signals
• CPC Class Codes: H04N 7/162 : Authorising the user terminal, e.g. by paying; Registering the use of a subscription channel, e.g. billing

Abstract

A method and system for detection of video segments in compressed digital video streams is presented. The compressed digital video stream is examine to determine synchronization points, and the compressed video signal is analyzed following detection of the synchronization points to create video fingerprints that are subsequently compared against a library of stored fingerprints.

Claims

1. A method of detecting a video entity in a media stream, the method comprising:

receiving a first media stream comprising a plurality of video frames and including at least one known video entity;

compressing the first media stream, the compressing comprising automatically generating a first fingerprint of statistical parameters of at least a portion of the first media stream; and

submitting the first fingerprint and at least one item of identification information corresponding to the first media stream to a database of fingerprints of known media content.

2. The method of claim 1, further comprising:

facilitating access to the first media stream.

3. The method of claim 1, further comprising:

receiving a second media stream comprising unidentified content;

comparing statistical parameters of the second media stream to the fingerprints in the database of fingerprints of known media content; and

determining, based on the comparing, whether the second media stream contains at least one known video entity.

4. The method of claim 3, further comprising:

denying access to the second media stream if the second media stream is determined to contain the at least one known video entity; and

facilitating access to the second media stream if the second media stream is determined to not contain the at least one known video entity.

5. The method of claim 3, further comprising:

if the second media stream is determined to not contain the at least one known video entity;

compressing the second media stream and automatically generating a second fingerprint of statistical parameters of at least a portion of the second media stream; and

submitting the second fingerprint and at least one item of identification information corresponding to the second media stream to the database of fingerprints of known video entities.

6. The method of claim 3, further comprising:

if the second media stream is determined to contain the at least one known video entity;

automatically selecting a third media stream based at least in part on the at least one item of identification information corresponding to the second media stream; and

providing access to the third media stream.

7. The method of claim 3, further comprising:

assigning a confidence value based on the comparing, wherein the confidence value represents a threshold level of similarity between the statistical parameters of the second media stream and the fingerprints in the database of fingerprints of known media content; and

utilizing the confidence value in determining whether the second media stream contains the at least one the known video entity.

8. The method of claim 3, wherein the first fingerprint is generated based on a comparison of features between at least two portions of the first media stream.

9. The method of claim 3, wherein the first fingerprint is generated based on color coherence vectors of at least a portion of the first media stream.

10. The method of claim 3, wherein the first fingerprint is generated based on color histograms of at least a portion of the first media stream.

11. The method of claim 3, wherein the first fingerprint is generated based on the relative color at least a portion of the first media stream.

12. A method of detecting a video entity in a media stream, the method comprising:

receiving a first media streams comprising a plurality of video frames and a known video entity;

compressing the first media stream, the compressing comprising automatically generating a first fingerprint of statistical parameters of at least a portion the first media stream;

storing the first fingerprint of the first media stream;

receiving a second media stream having unidentified content and at least one item of identification information corresponding to the second media stream;

generating a second fingerprint of at least a portion of the second media stream;

comparing the second fingerprint to a plurality of stored fingerprints, the plurality of stored fingerprints including the first fingerprint; and

determining, based on the comparing, whether the second media stream contains at least one known video entity.

13. The method of claim 12, further comprising:

denying access to the second media stream if the second media stream is determined to contain the at least one known video entity; and

facilitating access to the second media stream if the second media stream is determined to not contain the at least one known video entity.

14. The method of claim 12, further comprising:

if the second media stream is determined to not contain the at least one known video entity;

storing the second fingerprint and the at least one item of identification information corresponding to the second media stream.

15. The method of claim 12, further comprising:

if the second media stream is determined to contain the at least one known video entity;

automatically selecting a third media stream based at least in part on the at least one item of identification information corresponding to the second media stream; and

providing access to the third media stream.

16. A computer program product, comprising a computer usable, non-transitory medium having a computer readable program code embodied therein, said computer readable program code adapted to be executed to implement a method detecting a video entity in a video stream having a plurality of video frames, said method comprising:

storing a plurality of known fingerprints, wherein each of the known fingerprints includes a calculated feature corresponding to at least one frame of known video content;

receiving the video stream;

defining a plurality of regions of interest in the video frames of the received video stream;

determining a fingerprint of at least one of the video frames based on the defined regions of interest;

comparing the determined fingerprint to the plurality of known fingerprints; and

determining, based on the comparing, that the received video stream contains the video entity.

17. The computer program product of claim 16, further comprising:

assigning a confidence value based on the comparing, wherein the confidence value represents a threshold level of similarity between the determined fingerprint and the plurality of known fingerprints; and

utilizing the confidence value in determining that the received video stream contains at least one frame of the known video content.

18. The computer program product of claim 16, wherein the plurality of known fingerprints corresponds to the defined regions of interest.

19. The computer program product of claim 16, further comprising:

determining the plurality of known fingerprints from the at least one frame of known video content prior to storing the plurality of known fingerprints.

20. A computer program product, comprising a computer usable, non-transitory medium having a computer readable program code embodied therein, said computer readable program code adapted to be executed to implement a method detecting a video entity in a media stream, said method comprising:

receiving a first media stream comprising a plurality of video frames and including at least one known video entity;

compressing the first media stream, the compressing comprising automatically generating a first fingerprint of statistical parameters of at least a portion of the first media stream; and

submitting the first fingerprint and at least one item of identification information corresponding to the first media stream to a database of fingerprints of known media content.