System for organizing videos based on closed-caption information
First Claim
Patent Images
1. A system for indexing video comprising:
- a video directory;
a shot editor connected to said video directory for browsing and editing the video directory to produce a shotlist;
a producer connected to said shot editor for producing a mergelist and a keyframelist from the shotlist;
a shot group editor connected to said producer for editing the mergelist and the keyframelist;
a closed-caption decision connected to said shot group editor for determining if closed-caption text is available;
a processor connected to said closed-caption decision for processing the closed-caption text to extract data, wherein said data includes change signals; and
a closed-caption generator connected to said processor for generating subject segments and speaker segments using the data, wherein said closed-caption generator further defines shots and stories using said change signals.
3 Assignments
0 Petitions
Accused Products
Abstract
A system for organizing digital videos to archive and access them at different levels of abstraction uses data available from a closed-caption text along with off-the-shelf natural language processing tools to segment the video into self-contained story sections and speaker blocks. If the subject changes are marked, the system uses these points to divide the video into distinct stories which are represented as nodes attached to the root node in a tree structure and groups speaker segments belonging to a story under the story node as its children.
-
Citations
26 Claims
-
1. A system for indexing video comprising:
-
a video directory;
a shot editor connected to said video directory for browsing and editing the video directory to produce a shotlist;
a producer connected to said shot editor for producing a mergelist and a keyframelist from the shotlist;
a shot group editor connected to said producer for editing the mergelist and the keyframelist;
a closed-caption decision connected to said shot group editor for determining if closed-caption text is available;
a processor connected to said closed-caption decision for processing the closed-caption text to extract data, wherein said data includes change signals; and
a closed-caption generator connected to said processor for generating subject segments and speaker segments using the data, wherein said closed-caption generator further defines shots and stories using said change signals. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. A system for indexing video comprising:
-
a video directory;
a shot editor connected to said video directory for browsing and editing the video directory to produce a shotlist;
a producer connected to said shot editor for producing a mergelist and a keyframelist from the shotlist;
a shot group editor connected to said producer for editing the mergelist and the keyframelist;
a closed-caption decision connected to said shot group editor for determining if closed-caption text is available;
a processor connected to said closed-caption decision for processing the closed-caption text to extract data, wherein the closed-caption text comprises lines of text and frame numbers, wherein said lines of text include speaker markers for marking a beginning of different speaker segments; and
a closed-caption generator connected to said processor for generating subject segments and speaker segments using the data, wherein the closed-caption generator generates the subject segments and the speaker segments by determining whether different subject segments in the data are marked by subject markers, wherein if it is determined that different subject segments in the data are marked by subject markers, further comprising;
a separator for separating the different subject segments into separate nodes using the subject markers, and for separating the different speaker segments into separate speaker segments using the speaker markers; and
an organized tree creator for creating an organized tree comprising the separate nodes and attaching corresponding separate speaker segments to each separate node. - View Dependent Claims (9, 10)
a related segments finder for finding related segments in the different speaker segments using common elements and grouping the related segments into separate nodes.
-
-
10. The system of claim 9, wherein the common elements comprise proper nouns in the closed-caption text.
-
11. A method of indexing video comprising the steps of:
-
providing a video directory;
browsing and editing said video directory using a shot editor to produce a shotlist;
producing a mergelist file and a keyframelist from the shotlist using a producer;
editing the mergelist file and the keyframelist using a shot group editor;
determining if closed-caption text of said video is available, wherein if closed-caption text is determined to be available;
processing said closed-caption text to extract data, wherein said data includes change signals; and
segmenting the video into subject segments and speaker segments using the data, wherein segmenting further includes defining shots and stories using said change signals. - View Dependent Claims (12, 13, 14, 15, 16, 17, 18)
aligning the closed-caption text to the video;
generating sentences from the closed-caption text;
extracting noun phrases and proper nouns from the sentences; and
storing said noun phrases and proper nouns in a noun data file.
-
-
16. The method of claim 11, wherein the mergelist file comprises groups of visually similar shots.
-
17. The method of claim 11, wherein the keyframelist comprises additional distinct images from the shotlist.
-
18. The method of claim 11, wherein the shotlist comprises pairs of begin and end frames.
-
19. A method of indexing video comprising the steps of:
-
providing a video directory;
browsing and editing said video directory using a shot editor to produce a shotlist;
producing a mergelist file and a keyframelist from the shotlist using a producer;
editing the mergelist file and the keyframelist using a shot group editor;
determining if closed-caption text of said video is available, wherein if closed-caption text is determined to be available;
processing said closed-caption text to extract data, wherein the closed-caption text comprises lines of text and frame numbers, wherein said lines of text include speaker markers for marking a beginning of different speaker segments; and
segmenting the video into subject segments and speaker segments using the data, wherein the step of segmenting includes the steps of determining whether different subject segments in the data are marked by subject markers, wherein if it is determined that different subject segments in the data are marked by subject markers, further comprising the steps of;
separating the different subject segments into nodes using the subject markers;
separating the different speaker segments using the speaker markers to form separate speaker segments; and
creating an organized tree comprising the nodes and attaching the separate speaker segments to each corresponding node. - View Dependent Claims (20, 21)
determining common elements in the different speaker segments; and
matching the common elements within the different speaker segments to form related segments, wherein the related segments are grouped into corresponding nodes.
-
-
21. The method of claim 20, wherein the common elements comprise proper nouns in the closed-caption text.
-
22. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform the method steps for indexing video, the method comprising the steps of:
-
providing a video directory;
browsing and editing said video directory using a shot editor to produce a shotlist;
producing a mergelist file and a keyframelist from the shotlist using a producer;
editing the mergelist file and the keyframelist using a shot group editor;
determining if closed-caption text of said video is available, wherein if closed-caption text is determined to be available;
processing said closed-caption text to extract data, wherein said data includes change signals; and
segmenting the video into subject segments and speaker segments using the data, wherein segmenting further includes defining shots and stories using said change signals. - View Dependent Claims (23, 24)
aligning the closed-caption text to the video;
generating sentences from the closed-caption text;
extracting noun phrases and proper nouns from the sentences; and
storing said noun phrases and proper nouns in a noun data file.
-
-
25. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform the method steps for indexing video, the method comprising the steps of:
-
providing a video directory;
browsing and editing said video directory using a shot editor to produce a shotlist;
producing a mergelist file and a keyframelist from the shotlist using a producer;
editing the mergelist file and the keyframelist using a shot group editor;
determining if closed-caption text of said video is available, wherein if closed-caption text is determined to be available;
processing said closed-caption text to extract data, wherein the closed-caption text comprises lines of text and frame numbers, wherein said lines of text include speaker markers for marking a beginning of different speaker segments; and
segmenting the video into subject segments and speaker segments using the data, wherein the instructions for performing the step of segmenting includes instructions for;
determining whether different subject segments in the data are marked by subject markers, wherein if it is determined that different subject segments in the data are marked by subject markers, further comprising instructions for;
separating the different subject segments into nodes using the subject markers;
separating the different speaker segments using the speaker markers to form separate speaker segments; and
creating an organized tree comprising the nodes and attaching the separate speaker segments to each corresponding node. - View Dependent Claims (26)
determining common elements in the different speaker segments; and
matching the common elements within the different speaker segments to form related segments, wherein the related segments are grouped into corresponding nodes and the common elements comprise proper nouns in the closed-caption text.
-
Specification