Program segmentation of linear transmission

• US 10,116,902 B2
• Filed: 02/26/2010
• Issued: 10/30/2018
• Est. Priority Date: 02/26/2010
• Status: Active Grant

• Current Assignee: Comcast Cable Communications LLC (Comcast Corporation)
• Original Assignee: Comcast Cable Communications LLC (Comcast Corporation)
• Inventors: Tzoukermann, Evelyne, Farrell, Ryan M., Zhou, Hongzhong, Jojic, Oliver, Shevade, Bageshree
• Primary Examiner(s): Tran, Thai
• Assistant Examiner(s): Hasan, Syed
• Application Number:
  US12/713,883
• Publication Number:
  US 20110211812A1
• Time in Patent Office: 3,168 Days
• Field of Search: 386250, 386241, 386249, 386346, 386 E5001, 386 E5043, 386280, 386290, 386239, 386242, 348460, 348553, 348731, 348 E5007, 348 E5096, 348 E7071, 348 E7073, 358908, 360 13, 360 721, 360 722, 375 E7003, 375 E7016, 707E17028, 707723, 707749, 707769, 725 25, 725 28, 725100, 725131, 725134, 725139, 725151, 725145, 725146, 725 22, 725 18, 725 9, 725 32, 725 34, 725135, 725141, 725 14, 725 19, 725 20, 715203, 382229, 382181, 704254, 704255, 704270, 704278, 700 90, 709231, 709232, 709233, 709247
• CPC Class Codes: H04N 21/21815 comprising local storage units
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/4334 Recording operations recording of a television signal H04N5/76; arrangements for recording or accumulating broadcast information or broadcast-related information H04H60/27
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/44016 involving splicing one content stream with another content stream, e.g. for substituting a video clip
• CPC Class Codes: H04N 21/47202 for requesting content on demand, e.g. video on demand
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 7/17318 Direct or substantially direct transmission and handling of requests

Abstract

Content streams may be segmented to provide automatic extraction and storage of content items without intervening commercials or other unrelated content. These content items may then be stored in a database and made accessible to subscribers through, for example, an on-demand service. Automatic segmentation may include the identification of program boundaries, segmentation of a content stream based on the boundaries and the subsequent classification of the segments into content types. For example, audio and video duplication detection may be used to identify commercials since commercials tend to repeat frequently over a relatively short amount of time. A system may further identify an end of program indicator in a video stream to determine when a program ends. Accordingly, if a program ends after a scheduled end time, a recording device (e.g., the program is being recorded) may automatically extend the recording time to capture the entire program.

Claims

1. A method comprising:

determining, by a computing device and based on content scheduling information associated with a media stream;

content from the media stream, wherein the content comprises non-commercial content and commercial content; and

a content type associated with the non-commercial content;

determining, based on the content type, one or more expected visual elements corresponding to the content type;

determining, based on a comparison between the one or more expected visual elements and the content from the media stream, a non-commercial portion of the content from the media stream;

determining that a quantity of repeating elements in a second portion of the content from the media stream satisfies a threshold, wherein the second portion is different from the non-commercial portion; and

storing, after determining that the quantity satisfies the threshold, an updated version of the content from the media stream, wherein the updated version omits one or more of the repeating elements.

2. The method of claim 1,

wherein the threshold is associated with repeating scenes within the non-commercial content.

3. The method of claim 2, further comprising detecting a transition between the content from the media stream and another content of the media stream based on determining a joint entropy between a plurality of frames in the media stream separated by a cut.

4. The method of claim 2, wherein the determining that the quantity satisfies the threshold comprises determining whether the quantity of repeating elements occurs within a predetermined time period.

5. The method of claim 2, wherein the threshold is greater than a quantity of the repeating scenes within the non-commercial content, and

wherein the method further comprises generating the updated version based on a cut frequency.

6. The method of claim 1, wherein the content scheduling information comprises:

the content type associated with the non-commercial content; and

a program time schedule associated with the non-commercial content.

7. The method of claim 1, further comprising detecting a content boundary of the non-commercial content, wherein the detecting the content boundary comprises:

detecting a juncture in the media stream where a logo occurs or stops occurring; and

detecting particular audio content indicating the content boundary.

8. The method of claim 1, further comprising causing playback of the updated version of the content.

9. The method of claim 1, further comprising:

determining, by the computing device and based on the content scheduling information;

a different content from the media stream, wherein the different content comprises a different non-commercial content and commercial content; and

a different content type associated with the different non-commercial content; and

determining, based on the different content type, one or more expected visual elements corresponding to the different content type.

10. The method of claim 1, wherein the one or more expected visual elements comprises expected repeating visual elements, and

wherein the determining that the quantity satisfies the threshold comprises applying a weight value based on the expected repeating visual elements.

11. The method of claim 10, wherein the determining that the quantity satisfies the threshold comprises reducing the weight value for the expected repeating visual elements.

12. The method of claim 1, wherein the repeating elements comprise one or more of:

repeating images or repeating audio.

13. A method comprising:

determining, by a computing device and based on content scheduling information associated with a media stream;

a portion of the media stream, wherein the portion comprises commercial content and a program; and

an expected program category associated with the program;

determining, by the computing device and based on transitions in the portion, a plurality of segments in the portion of the media stream;

determining, based on the expected program category, an expected visual element;

determining, based on a comparison between the expected visual element and the plurality of the segments, that one or more of the segments comprise the expected visual element; and

generating an updated version of the portion of the media stream, wherein the updated version;

comprises the one or more segments that comprise the expected visual element, and

omits one or more other segments of the plurality of the segments.

14. The method of claim 13, wherein the determining the plurality of segments in the portion of the media stream comprises:

detecting a juncture in the portion of the media stream where a logo occurs or stops occurring; and

segmenting the portion of the media stream at the detected juncture.

15. The method of claim 13, further comprising:

determining a quantity of repeating elements in the one or more other segments; and

after determining that the quantity of repeating elements being greater than a duplicate threshold, determining the repeating elements as being part of the commercial content within the plurality of segments,

wherein the duplicate threshold is associated with repeating scenes within the program.

16. The method of claim 15, wherein the determining the repeating elements as being part of the commercial content within the plurality of segments is further responsive to whether the quantity of repeating elements occurs within a predetermined time period.

17. The method of claim 13, further comprising detecting the transitions in the portion of the media stream based on an image analysis of one or more frames in the portion of the media stream, wherein the detecting the transitions in the portion comprises determining a joint entropy between a plurality of frames in the portion of the media stream separated by a cut.

18. The method of claim 13, wherein the content scheduling information comprises:

a first program time schedule associated with the program;

a second program time schedule associated with a second program; and

program category information for the program and the second program.

19. An apparatus comprising:

one or more processors; and

memory storing computer-readable instructions that, when executed by the one or more processors, cause the apparatus to;

determine, based on content scheduling information associated with a media stream;

content from the media stream, wherein the content comprises non-commercial content and commercial content; and

a content type associated with the non-commercial content;

determine, based on the content type, one or more expected visual elements corresponding to the content type;

determine, based on a comparison between the one or more expected visual elements and the content from the media stream, a non-commercial portion of the content from the media stream;

determine that a quantity of repeating elements in a second portion of the content from the media stream satisfies a threshold, wherein the second portion is different from the non-commercial portion; and

store, after determining that the quantity satisfies the threshold, an updated version of the content from the media stream, wherein the updated version omits one or more of the repeating elements.

20. The apparatus of claim 19, wherein the computer-readable instructions, when executed by the one or more processors, further cause the apparatus to:

detect a juncture in the content from the media stream where a logo occurs or stops occurring; and

detect, based on the detected juncture, a content boundary of the non-commercial content.

21. The apparatus of claim 19, wherein the content scheduling information comprises:

the content type associated with the non-commercial content; and

a program time schedule associated with the non-commercial content.

22. The apparatus of claim 19, wherein the threshold is associated with repeating scenes within the non-commercial content.

23. The apparatus of claim 19, wherein the repeating elements comprise one or more of:

repeating images or repeating audio.