CONTENT ALIGNMENT METHOD AND SYSTEM

• US 20150178369A1
• Filed: 12/24/2013
• Published: 06/25/2015
• Est. Priority Date: 12/24/2013
• Status: Active Grant

• Current Assignee: TCL Research America, Inc. (TCL Technology Group Corp.)
• Original Assignee: TCL Research America, Inc. (TCL Technology Group Corp.)
• Inventors: WANG, Haohong, DING, Hao, HUANG, Jia
• Granted Patent: US 9,323,865 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2246 Trees, e.g. B+trees
• CPC Class Codes: G06F 16/9027 Trees
• CPC Class Codes: H04N 21/4884 for displaying subtitles

Abstract

A method is provided for content alignment. The method includes obtaining a first content sequence and a second content sequence different from the first content sequence. The method also includes representing each of the first content sequence and the second content sequence in a hierarchical structure containing an ordered root element sequence and a sub-tree structure. The ordered root element sequence includes a plurality of root elements and each root element is associated with a sub-tree of elements. The method also includes determining a desired alignment between the first content sequence and the second content sequence using dynamic programming, and outputting results of the desired alignment between the first content sequence and the second content sequence.

Claims

1. A method for content alignment, comprising:

obtaining a first content sequence and a second content sequence different from the first content sequence;

representing each of the first content sequence and the second content sequence in a hierarchical structure containing an ordered root element sequence and a sub-tree structure, wherein the ordered root element sequence includes a plurality of root elements and each root element is associated with a sub-tree of elements;

determining a desired alignment between the first content sequence and the second content sequence using dynamic programming; and

outputting results of the desired alignment between the first content sequence and the second content sequence.

2. The method according to claim 1, wherein:

the hierarchical structure is divided into a plurality of hierarchical levels, and each hierarchical level includes one or more sequences of elements.

3. The method according to claim 1, wherein:

the first content sequence and the second content sequence are movie subtitles; and

the hierarchical structure includes three levels of a “

paragraph-sentence-word”

structure.

4. The method according to claim 1, wherein:

the first content sequence and the second content sequence are video contents; and

the hierarchical structure includes four levels of a “

scene-shot-frame-block”

structure.

5. The method according to claim 2, wherein determining a desired alignment between the first content sequence and the second content sequence further includes:

determining an align pair for each of selected elements from the first content sequence and the second content sequence, wherein the align pair is either a pair of an element of the first content sequence and an element of the second content sequence or a pair of an element of the first content sequence or the second content sequence and null;

determining a distance of each align pair as a dissimilarity between the elements in the align pair; and

determining a total distance of all align pairs of the selected elements as a total distance of an alignment between the selected elements of the first content sequence and the second content sequence.

6. The method according to claim 5, further including:

determining a distance of root elements of the first content sequence and the second content sequence;

determining a distance of sub-tree structure elements of the first content sequence and the second content sequence;

determining a total distance of the first content sequence and the second content sequence, as a total distance of an alignment between the first content sequence and the second content sequence, based on the distance of root elements and the distance of sub-tree structure elements.

7. The method according to claim 6, wherein:

the root elements are represented by vector of features;

the values of the vector is Boolean or normalized real numbers in [0, 1]; and

a Euclidean distance is used as a measurement for a distance between vectors.

8. The method according to claim 6, further including:

recursively determining the total distances between the first content sequence and the second content sequence to find a minimum distance between the first content sequence and the second content sequence; and

determining an alignment with the minimum distance between the first content sequence and the second content sequence as the desired alignment between the first content sequence and the second content sequence.

9. The method according to claim 1, wherein outputting results of the desired alignment further includes:

integrating aligned first content sequence and second content sequence to produce an integrated content sequence based on the desired alignment; and

displaying the integrated content sequence to a user.

10. A content alignment system, comprising:

a memory module;

an output interface; and

a processor coupled to the memory module and the output interface and configured to;

obtain a first content sequence and a second content sequence different from the first content sequence;

represent each of the first content sequence and the second content sequence in a hierarchical structure containing an ordered root element sequence and a sub-tree structure, wherein the ordered root element sequence includes a plurality of root elements and each root element is associated with a sub-tree of elements;

determine a desired alignment between the first content sequence and the second content sequence using dynamic programming; and

output results of the desired alignment between the first content sequence and the second content sequence through the output interface.

11. The content alignment system according to claim 10, wherein:

the hierarchical structure is divided into a plurality of hierarchical levels, and each hierarchical level includes one or more sequences of elements.

12. The content alignment system according to claim 10, wherein:

the first content sequence and the second content sequence are movie subtitles; and

the hierarchical structure includes three levels of a “

paragraph-sentence-word”

structure.

13. The content alignment system according to claim 10, wherein:

the first content sequence and the second content sequence are video contents; and

the hierarchical structure includes four levels of a “

scene-shot-frame-block”

structure.

14. The content alignment system according to claim 11, wherein, to determine a desired alignment between the first content sequence and the second content sequence, the processor is further configured to:

determine an align pair for each of selected elements from the first content sequence and the second content sequence, wherein the align pair is either a pair of an element of the first content sequence and an element of the second content sequence or a pair of an element of the first content sequence or the second content sequence and null;

determine a distance of each align pair as a dissimilarity between the elements in the align pair; and

determine a total distance of all align pairs of the selected elements as a total distance of an alignment between the selected elements of the first content sequence and the second content sequence.

15. The content alignment system according to claim 14, wherein the processor is further configured to:

determine a distance of root elements of the first content sequence and the second content sequence;

determining a distance of sub-tree structure elements of the first content sequence and the second content sequence;

determine a total distance of the first content sequence and the second content sequence, as a total distance of an alignment between the first content sequence and the second content sequence, based on the distance of root elements and the distance of sub-tree structure elements.

16. The content alignment system according to claim 15, wherein:

the root elements are represented by vector of features;

the values of the vector is Boolean or normalized real numbers in [0, 1]; and

a Euclidean distance is used as a measurement for a distance between vectors.

17. The content alignment system according to claim 15, wherein the processor is further configured to:

recursively determine the total distances between the first content sequence and the second content sequence to find a minimum distance between the first content sequence and the second content sequence; and

determine an alignment with the minimum distance between the first content sequence and the second content sequence as the desired alignment between the first content sequence and the second content sequence.

18. The content alignment system according to claim 10, wherein, to output results of the desired alignment, the processor is further configured to:

integrate aligned first content sequence and second content sequence to produce an integrated content sequence based on the desired alignment; and

display the integrated content sequence to a user.