Thematic segmentation of long content using deep learning and contextual cues

• US 10,733,980 B2
• Filed: 04/17/2019
• Issued: 08/04/2020
• Est. Priority Date: 08/23/2017
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP SE
• Inventors: Kotri, Jayananda Appanna, Sharma, Tarun, Kejriwal, Sharad, Dasari, Yashwanth, S, Abinaya
• Primary Examiner(s): Riley, Marcus T
• Application Number:
  US16/386,498
• Publication Number:
  US 20190244605A1
• Time in Patent Office: 475 Days
• Field of Search: 704 1-504
• CPC Class Codes: G06F 40/258 Heading extraction; Automatic titling; Numbering
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G10L 15/04 Segmentation; Word boundary detection
• CPC Class Codes: G10L 15/16 using artificial neural networks
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications

Abstract

A recurrent neural network (RNN) is trained to identify split positions in long content, wherein each split position is a position at which the theme of the long content changes. Each sentence in the long content is converted to a vector that corresponds to the meaning of the sentence. The sentence vectors are used as inputs to the RNN. The high-probability split points determined by the RNN may be combined with contextual cues to determine the actual split point to use. The split points are used to generate thematic segments of the long content. The multiple thematic segments may be presented to a user along with a topic label for each thematic segment. Each topic label may be generated based on the words contained in the corresponding thematic segment.

Claims

1. A system comprising:

a memory that stores instructions;

one or more processors configured by the instructions to perform operations comprising;

accessing a plurality of sentences embedded in a file that includes a paragraph change indicator at a position within the plurality of sentences;

generating a plurality of sentence vectors, each sentence vector of the plurality of sentence vectors corresponding to a respective sentence of the plurality of sentences;

providing a subset of the plurality of sentence vectors as an input to a recurrent neural network (RNN);

based on the position of the paragraph change indicator and an output of the RNN responsive to the input, determining that a subset of the plurality of sentences relate to a first topic; and

providing an output comprising the subset of the plurality of sentences related to the first topic.

2. The system of claim 1, wherein the generating of each sentence vector of the plurality of sentence vectors comprises:

accessing a plurality of word vectors, each word vector of the plurality of word vectors corresponding to a respective word of the sentence corresponding to the respective sentence vector; and

averaging the plurality of word vectors to generate the sentence vector.

3. The system of claim 1, wherein the operations further comprise:

accessing an audio file; and

generating the plurality of sentences from the audio file using speech-to-text conversion.

4. The system of claim 1, wherein:

the determining that the subset of the plurality of sentences relate to the first topic comprises;

comparing each value of a plurality of output values from the RNN to a predetermined threshold, each output value corresponding to a possible split position indicating a split between the first topic and a second topic; and

the determining that the subset of the plurality of sentences relate to the first topic is based on results of the comparisons.

5. The system of claim 1, wherein:

the file includes a header indicator at a second position within the plurality of sentences; and

the determining that the subset of the plurality of sentences relate to the first topic is further based on the second position.

6. The system of claim 1, wherein the operations further comprise:

determining a set of words comprised by the subset of the plurality of sentences; and

generating a name of the first topic based on the set of words.

7. The system of claim 1, wherein:

the operations further comprise;

accessing a uniform resource locator (URL);

accessing a media file using the URL;

generating the plurality of sentences by using speech-to-text conversion on the media file; and

identifying a second subset of the plurality of sentences related to a second topic, using the RNN;

the output comprising the subset of the plurality of sentences related to the first topic is a first media file; and

the operations further comprise;

generating a first name for the first topic;

generating a second media file comprising the second subset of the plurality of sentences related to the second topic;

generating a second name for the second topic; and

providing a user interface that includes the first name, a link to the first media file, the second name, and a link to the second media file.

8. A method comprising:

accessing, by one or more processors, a plurality of sentences embedded in a file that includes a paragraph change indicator at a position within the plurality of sentences;

generating, by the one or more processors, a plurality of sentence vectors, each sentence vector of the plurality of sentence vectors corresponding to a respective sentence of the plurality of sentences;

providing, by the one or more processors, a subset of the plurality of sentence vectors as an input to a recurrent neural network (RNN);

based on the position of the paragraph change indicator and an output of the RNN responsive to the input, determining, by the one or more processors, that a subset of the plurality of sentences relate to a first topic; and

providing, by the one or more processors, an output comprising the subset of the plurality of sentences related to the first topic.

9. The method of claim 8, wherein the generating of each sentence vector of the plurality of sentence vectors comprises:

accessing a plurality of word vectors, each word vector of the plurality of word vectors corresponding to a respective word of the sentence corresponding to the respective sentence vector; and

averaging the plurality of word vectors to generate the sentence vector.

10. The method of claim 8, further comprising:

accessing an audio file; and

generating the plurality of sentences from the audio file using speech-to-text conversion.

11. The method of claim 8, wherein:

the determining that the subset of the plurality of sentences relate to the first topic comprises;

comparing each value of a plurality of output values from the RNN to a predetermined threshold, each output value corresponding to a possible split position indicating a split between the first topic and a second topic; and

the determining that the subset of the plurality of sentences relate to the first topic is based on results of the comparisons.

12. The method of claim 8, wherein:

the file includes a header indicator at a second position within the plurality of sentences; and

the determining that the subset of the plurality of sentences relate to the first topic is further based on the second position.

13. The method of claim 8, further comprising:

determining a set of words comprised by the subset of the plurality of sentences; and

generating a name of the first topic based on the set of words.

14. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

accessing, by one or more processors, a plurality of sentences embedded in a file that includes a paragraph change indicator at a position within the plurality of sentences;

generating, by the one or more processors, a plurality of sentence vectors, each sentence vector of the plurality of sentence vectors corresponding to a respective sentence of the plurality of sentences;

providing, by the one or more processors, a subset of the plurality of sentence vectors as an input to a recurrent neural network (RNN);

based on the position of the paragraph change indicator and an output of the RNN responsive to the input, determining, by the one or more processors, that a subset of the plurality of sentences relate to a first topic; and

providing, by the one or more processors, an output comprising the subset of the plurality of sentences related to the first topic.

15. The non-transitory machine-readable storage medium of claim 14, wherein the generating of each sentence vector of the plurality of sentence vectors comprises:

accessing a plurality of word vectors, each word vector of the plurality of word vectors corresponding to a respective word of the sentence corresponding to the respective sentence vector; and

averaging the plurality of word vectors to generate the sentence vector.

16. The non-transitory machine-readable storage medium of claim 14, wherein the operations further comprise:

accessing an audio file; and

generating the plurality of sentences from the audio file using speech-to-text conversion.

17. The non-transitory machine-readable storage medium of claim 14, wherein:

the determining that the subset of the plurality of sentences relate to the first topic comprises;

comparing each value of a plurality of output values from the RNN to a predetermined threshold, each output value corresponding to a possible split position indicating a split between the first topic and a second topic; and

the determining that the subset of the plurality of sentences relate to the first topic is based on results of the comparisons.

18. The non-transitory machine-readable storage medium of claim 14, wherein:

the file includes a header indicator at a second position within the plurality of sentences; and

the determining that the subset of the plurality of sentences relate to the first topic is further based on the second position.

19. The non-transitory machine-readable storage medium of claim 14, wherein the operations further comprise:

determining a set of words comprised by the subset of the plurality of sentences; and

generating a name of the first topic based on the set of words.

20. The non-transitory machine-readable storage medium of claim 14, wherein:

the operations further comprise;

accessing a uniform resource locator (URL);

accessing a media file using the URL;

generating the plurality of sentences by using speech-to-text conversion on the media file; and

identifying a second subset of the plurality of sentences related to a second topic, using the RNN;

the output comprising the subset of the plurality of sentences related to the first topic is a first media file; and

the operations further comprise;

generating a first name for the first topic;

generating a second media file comprising the second subset of the plurality of sentences related to the second topic;

generating a second name for the second topic; and

providing a user interface that includes the first name, a link to the first media file, the second name, and a link to the second media file.