Automatic segmentation of video

• US 8,423,555 B2
• Filed: 07/09/2010
• Issued: 04/16/2013
• Est. Priority Date: 07/09/2010
• Status: Active Grant

• Current Assignee: Comcast Cable Communications LLC (Comcast Corporation)
• Original Assignee: Comcast Cable Communications LLC (Comcast Corporation)
• Inventors: Ambwani, Geetu, Davis, Anthony R.
• Primary Examiner(s): BURKE, JEFF A
• Application Number:
  US12/833,486
• Publication Number:
  US 20120011109A1
• Time in Patent Office: 1,012 Days
• Field of Search: 707/711, 707/741, 707/746, 707/771, 707/780, 707/913
• US Class Current: 707/746
• CPC Class Codes: G06F 16/435 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data
• CPC Class Codes: G06F 16/90344 by using string matching techniques
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04N 21/278 Content descriptor database or directory service for end-user access details of content or meta data based information retrieval of video data in video databases G06F16/739
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/440236 by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text

Abstract

Content items may be segmented and labeled by topic to provide for the capture, analysis, indexing, retrieval and/or distribution of information within information rich media, such as audio or video, with greater functionality, accuracy and speed. The segments and other related information may be stored in a database and made accessible to users through, for example, a search service and/or an on-demand service. Automatic segmentation may include receiving a text representation, calculating relevance intervals based on the text representation, determining a nodal representation based on the relevance intervals, and determining segments of the content item based on the nodal representation.

Claims

1. A method comprising:

receiving text associated with a content item, the text including a plurality of term strings, each term string including one or more terms;

determining, by a computing device, a set of relevance intervals for a corresponding set of terms in the text, wherein each relevance interval of the set of relevance intervals identifies one or more sections of the content item that are relevant to a corresponding term;

generating a first set of nodes for a first term string from the plurality of term strings, wherein the first set of nodes includes a node for each term in the first term string, and at least one node for a term different from the terms in the first term string but whose corresponding relevance interval includes the first term string;

generating a second set of nodes for a second term string from the plurality of term strings, wherein the second set of nodes includes a node for each term in the second term string, and at least one node for a term different from the terms in the second term string but whose corresponding relevance interval includes the second term string; and

determining a plurality of connections between nodes in the first set of nodes and the second set of nodes, wherein the plurality of connections includes a subset for a first node from the first set of nodes, wherein the subset for the first node includes a connection for each node from the second set of nodes; and

using the first set of nodes, second set of nodes or plurality of connections to identify one or more segment boundaries within the content item.

2. The method of claim 1, wherein each term string of the plurality of term strings is a sentence.

3. The method of claim 1, wherein a first connection of the subset for the first node is a connection strength that includes a contextualized measurement of the relatedness between the first node and a second node in the second set of nodes.

4. The method of claim 1, further comprising:

filtering the plurality of connections based on at least one of a number of connections between the first set of nodes and the second set of nodes, or a cumulative value of connection strengths associated with the plurality of connections.

5. The method of claim 1, further comprising:

determining that a segment boundary occurs between the first set of nodes and the second set of nodes based on the plurality of connections.

6. The method of claim 5, further comprising:

determining a segment of the content item based on the segment boundary.

7. The method of claim 6, further comprising:

transmitting information based on the segment of the content item to a user device.

8. The method of claim 6, further comprising:

determining a topic for the segment of the content item based on at least one of the first set of nodes, the second set of nodes, or the text.

9. The method of claim 8, wherein determining the topic is further based on at least one of a frequency of occurrences of terms in the text or a determined central node based on a term centrality measurement.

10. An apparatus comprising:

a processor; and

memory operatively coupled to the processor and storing computer readable instructions that, when executed, cause the apparatus to;

receive text associated with a content item, the text including a plurality of term strings, each term string including one or more terms;

determine a set of relevance intervals for a corresponding set of terms in the text, wherein each relevance interval of the set of relevance intervals identifies one or more sections of the content item that are relevant to a corresponding term;

generate a first set of nodes for a first term string from the plurality of term strings, wherein the first set of nodes includes a node for each term in the first term string, and at least one node for a term different from the terms in the first term string but whose corresponding relevance interval includes the first term string;

generate a second set of nodes for a second term string from the plurality of term strings, wherein the second set of nodes includes a node for each term in the second term string, and at least one node for a term different from the terms in the second term string but whose corresponding relevance interval includes the second term string; and

determine a plurality of connections between nodes in the first set of nodes and the second set of nodes, wherein the plurality of connections includes a subset for a first node from the first set of nodes, wherein the subset for the first node includes a connection for each node from the second set of nodes; and

use the first set of nodes, the second set of nodes or the plurality of connections to identify one or more segment boundaries within the content item.

11. The apparatus of claim 10, wherein a first connection of the subset for the first node is a connection strength that includes a contextualized measurement of the relatedness between the first node and a second node in the second set of nodes.

12. The apparatus of claim 10, wherein the memory further stores computer readable instructions that, when executed, cause the apparatus to:

filter the plurality of connections based on at least one of a number of connections between the first set of nodes and the second set of nodes, or a cumulative value of connection strengths associated with the plurality of connections.

13. The apparatus of claim 10, wherein the memory further stores computer readable instructions that, when executed, cause the apparatus to:

determine that a segment boundary occurs between the first set of nodes and the second set of nodes based on the plurality of connections.

14. The apparatus of claim 13, wherein the memory further stores computer readable instructions that, when executed, cause the apparatus to:

determine a segment of the content item based on the segment boundary.

15. The apparatus of claim 14, wherein the memory further stores computer readable instructions that, when executed, cause the apparatus to:

transmit information based on the segment of the content item to a user device.

16. The apparatus of claim 14, wherein the memory further stores computer readable instructions that, when executed, cause the apparatus to:

determine a topic for the segment of the content item based on at least one of the first set of nodes, the second set of nodes, or the text.

17. The apparatus of claim 16, wherein determining the topic is further based on at least one of a frequency of occurrences of terms in the text or a determined central node based on a term centrality measurement.

18. One or more non-transitory computer readable media storing computer readable instructions that, when executed, cause a processor to:

receive text associated with a content item, the text including a plurality of term strings, each term string including one or more terms;

determine a set of relevance intervals for a corresponding set of terms in the text, wherein each relevance interval of the set of relevance intervals identifies one or more sections of a content item that are relevant to a corresponding term;

generate a first set of nodes for a first term string from the plurality of term strings, wherein the first set of nodes includes a node for each term in the first term string, and at least one node for a term different from the terms in the first term string but whose corresponding relevance interval includes the first term string;

generate a second set of nodes for a second term string from the plurality of term strings, wherein the second set of nodes includes a node for each term in the second term string, and at least one node for a term different from the terms in the second term string but whose corresponding relevance interval includes the second term string; and

determine a plurality of connections between nodes in the first set of nodes and the second set of nodes, wherein the plurality of connections includes a subset for a first node from the first set of nodes, wherein the subset for the first node includes a connection for each node from the second set of nodes; and

use the first set of nodes, the second set of nodes or the plurality of connections to identify one or more segment boundaries within the content item.