SUMMARIZATION SYSTEM

• US 20190327103A1
• Filed: 04/17/2019
• Published: 10/24/2019
• Est. Priority Date: 04/19/2018
• Status: Active Grant

• Current Assignee: SRI International, Inc.
• Original Assignee: SRI International, Inc.
• Inventors: Niekrasz, John
• Granted Patent: US 11,018,885 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/345 Summarisation for human users
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 25/51 for comparison or discrimination
• CPC Class Codes: H04L 12/1831 Tracking arrangements for later retrieval, e.g. recording contents, participants activities or behavior, network status

Abstract

In general, the disclosure describes techniques for automatically generating summaries of meetings. A computing system obtains a transcript of a meeting and may produce, based on the transcript of the meeting, a data structure that comprises utterance features. Furthermore, the computing system may determine, based on the transcript of the meeting, temporal bounds of a plurality of activity episodes within the meeting. For each respective activity episode of a plurality of activity episodes, the computing system may determine, based on the utterance features associated with the respective activity episode, a conversational activity type associated with the respective activity episode. Additionally, the computing system may produce an episode summary for the respective activity episode that is dependent on the determined conversational activity type associated with the respective activity episode. The episode summary for the respective activity episode summarizes a conversational activity that occurred during the respective activity episode.

Claims

1. A method for automatically generating summaries of meetings, the method comprising:

obtaining, by a computing system, a transcript of a meeting;

producing, by the computing system, based on the transcript of the meeting, a data structure that comprises utterance features;

determining, by the computing system, based on the transcript of the meeting, temporal bounds of a plurality of activity episodes within the meeting, wherein for each respective activity episode of the plurality of activity episodes, the utterance features include at least one utterance feature associated with the respective activity episode;

generating, by the computing system, a plurality of episode summaries, wherein generating the plurality of episode summaries comprises, for each respective activity episode of the plurality of activity episodes;

determining, by the computing system, based on the utterance features associated with the respective activity episode, a conversational activity type associated with the respective activity episode; and

producing, by the computing system, an episode summary for the respective activity episode that is dependent on the determined conversational activity type associated with the respective activity episode, the episode summary for the respective activity episode summarizing a conversational activity that occurred during the respective activity episode; and

combining, by the computing system, the plurality of episode summaries to generate a summary of the meeting.

2. The method of claim 1, wherein determining the utterance features comprises at least one of:

determining references to tasks;

determining references to conversation participants;

determining references to future dates;

determining features that describe an utterance in terms of grammatical tense-aspect-mood;

or determining agreement-oriented speech acts.

3. The method of claim 1, wherein two or more of the activity episodes are associated with different conversational activity types and overlap temporally.

4. The method of claim 1, wherein:

determining the conversational activity type associated with the respective activity episode comprises selecting, by the computing system, the conversational activity type associated with the respective activity episode from among a predefined plurality of conversational activity types, and

each conversational activity type of the predefined plurality of conversational activity types corresponds to a different type of activity capable of being performed by participants of the meeting by conversing.

5. The method of claim 4, wherein:

each respective conversational activity type of the predefined plurality of conversational activity types is associated with a respective tuple that includes a plurality of elements different from pluralities of elements of tuples associated with other conversational activity types of the predefined plurality of conversational activity types; and

for each respective activity episode of the plurality of activity episodes, producing the episode summary for the respective activity episode comprises;

generating, by the computing system, one or more intermediate extractions for the respective activity episode, wherein each of the one or more intermediate extractions is an instance of the tuple associated with the conversational activity type associated with the respective activity episode,

wherein, for each respective intermediate extraction of the one or more intermediate extractions for the respective activity episode, generating the respective intermediate extraction comprises assembling, by the computing system, utterance features from among the utterance features associated with the respective activity episode into tuple components of the tuple of the respective intermediate extraction; and

using, by the computing system, the tuple of a particular intermediate extraction of the one or more intermediate extractions for the respective activity episode to populate the episode summary for the respective activity episode.

6. The method of claim 5, wherein:

the one or more intermediate extractions includes a plurality of intermediate extractions for the respective activity episode, and

producing the episode summary for the respective activity episode further comprises;

after generating the plurality of intermediate extractions for the respective activity episode, performing, by the computing system, a pairwise similarity comparison of the intermediate extractions for the respective activity episode;

assigning, by the computing system, based on the pairwise similarity comparison, numerical weightings to the intermediate extractions for the respective activity episode; and

selecting, by the computing system, based on the numerical weightings assigned to the intermediate extractions for the respective activity episode, the particular intermediate extraction from among the plurality of intermediate extractions for the respective activity episode.

7. The method of claim 5, wherein:

the one or more intermediate extractions includes a plurality of intermediate extractions for the respective activity episode, and

producing the episode summary for the respective activity episode further comprises using, by the computing system, the tuples of one or more of the intermediate extractions other than the particular intermediate extraction to supplement the episode summary for the respective activity episode.

8. The method of claim 1, further comprising, for each of the activity episodes:

selecting, by the computing system, a set of utterances to characterize content of the activity episode; and

presenting, by the computing system, the selected set of utterances to a user in addition to the episode summary for the activity episode.

9. A computing system for automatically generating abstractive summaries of meetings, the computing system comprising:

a memory configured to store a transcript of a meeting; and

one or more processing circuits configured to;

produce, based on the transcript of the meeting, a data structure that comprises utterance features;

determine, based on the transcript of the meeting, temporal bounds of a plurality of activity episodes within the meeting, wherein for each respective activity episode of the plurality of activity episodes, the utterance features include at least one utterance feature associated with the respective activity episode;

generating a plurality of episode summaries, wherein the one or more processing circuits are configured such that, as part of generating the plurality of episode summaries the one or more processing circuits, for each respective activity episode of the plurality of activity episodes;

determine, based on the utterance features associated with the respective activity episode, a conversational activity type associated with the respective activity episode; and

produce an episode summary for the respective activity episode that is dependent on the determined conversational activity type associated with the respective activity episode, the episode summary for the respective activity episode summarizing a conversational activity that occurred during the respective activity episode; and

combine the plurality of episode summaries to generate a summary of the meeting.

10. The computing system of claim 9, wherein the one or more processors are configured such that, as part of determining the utterance features, the one or more processors determine at least one of:

references to tasks;

references to conversation participants;

references to future dates;

features that describe an utterance in terms of grammatical tense-aspect-mood;

or agreement-oriented speech acts.

11. The computing system of claim 9, wherein two or more of the activity episodes are associated with different conversational activity types and overlap temporally.

12. The computing system of claim 9, wherein:

the one or more processors are configured such that, as part of determining the conversational activity type associated with the respective activity episode, the one or more processors select the conversational activity type associated with the respective activity episode from among a predefined plurality of conversational activity types, and

each conversational activity type of the predefined plurality of conversational activity types corresponds to a different type of activity capable of being performed by participants of the meeting by conversing.

13. The computing system of claim 12, wherein:

each respective conversational activity type of the predefined plurality of conversational activity types is associated with a respective tuple that includes a plurality of elements different from pluralities of elements of tuples associated with other conversational activity types of the predefined plurality of conversational activity types; and

for each respective activity episode of the plurality of activity episodes, the one or more processors are configured such that, as part of producing the episode summary for the respective activity episode, the one or more processing circuits;

generate one or more intermediate extractions for the respective activity episode, wherein each of the one or more intermediate extractions is an instance of the tuple associated with the conversational activity type associated with the respective activity episode,

wherein, for each respective intermediate extraction of the one or more intermediate extractions for the respective activity episode, the one or more processing circuits are configured such that, as part of generating the respective intermediate extraction, the one or more processing circuits assemble utterance features from among the utterance features associated with the respective activity episode into tuple components of the tuple of the respective intermediate extraction; and

use the tuple of a particular intermediate extraction of the one or more intermediate extractions for the respective activity episode to populate the episode summary for the respective activity episode.

14. The computing system of claim 13, wherein:

the one or more intermediate extractions includes a plurality of intermediate extractions for the respective activity episode, and

the one or more processing circuits are configured such that, as part of producing the episode summary for the respective activity episode, the one or more processors;

after generating the plurality of intermediate extractions for the respective activity episode, perform a pairwise similarity comparison of the intermediate extractions for the respective activity episode;

assign, based on the pairwise similarity comparison, numerical weightings to the intermediate extractions for the respective activity episode; and

select, based on the numerical weightings assigned to the intermediate extractions for the respective activity episode, the particular intermediate extraction from among the plurality of intermediate extractions for the respective activity episode.

15. The computing system of claim 13, wherein:

the one or more intermediate extractions includes a plurality of intermediate extractions for the respective activity episode, and

the one or more processing circuits are configured such that, as part of producing the episode summary for the respective activity episode, the one or more processing circuits further use the tuples of one or more of the intermediate extractions other than the particular intermediate extraction to supplement the episode summary for the respective activity episode.

16. The computing system of claim 9, wherein, for each of the activity episodes, the one or more processing circuits are configured to:

select a set of utterances to characterize content of the activity episode; and

present the selected set of utterances to a user in addition to the episode summary for the activity episode.

17. A non-transitory computer-readable medium comprising instructions, wherein execution of the instructions causes one or more processing circuits of a computing system to perform operations comprising:

obtaining a transcript of a meeting;

producing, based on the transcript of the meeting, a data structure that comprises utterance features;

determining, based on the transcript of the meeting, temporal bounds of a plurality of activity episodes within the meeting, wherein for each respective activity episode of the plurality of activity episodes, the utterance features include at least one utterance feature associated with the respective activity episode;

generating a plurality of episode summaries, wherein generating the plurality of episode summaries comprises, for each respective activity episode of the plurality of activity episodes;

determining, based on the utterance features associated with the respective activity episode, a conversational activity type associated with the respective activity episode; and

producing an episode summary for the respective activity episode that is dependent on the determined conversational activity type associated with the respective activity episode, the episode summary for the respective activity episode summarizing a conversational activity that occurred during the respective activity episode; and

combining the plurality of episode summaries to generate a summary of the meeting.

18. The non-transitory computer-readable medium of claim 17, wherein:

determining the conversational activity type associated with the respective activity episode comprises selecting the conversational activity type associated with the respective activity episode from among a predefined plurality of conversational activity types, and

each conversational activity type of the predefined plurality of conversational activity types corresponds to a different type of activity capable of being performed by participants of the meeting by conversing.

19. The non-transitory computer-readable medium of claim 18, wherein:

each respective conversational activity type of the predefined plurality of conversational activity types is associated with a respective tuple that includes a plurality of elements different from pluralities of elements of tuples associated with other conversational activity types of the predefined plurality of conversational activity types; and

for each respective activity episode of the plurality of activity episodes, producing the episode summary for the respective activity episode comprises;

generating one or more intermediate extractions for the respective activity episode, wherein each of the one or more intermediate extractions is an instance of the tuple associated with the conversational activity type associated with the respective activity episode,

wherein, for each respective intermediate extraction of the one or more intermediate extractions for the respective activity episode, generating the respective intermediate extraction comprises assembling utterance features from among the utterance features associated with the respective activity episode into tuple components of the tuple of the respective intermediate extraction; and

using the tuple of a particular intermediate extraction of the one or more intermediate extractions for the respective activity episode to populate the episode summary for the respective activity episode.

20. The non-transitory computer-readable medium of claim 19, wherein:

the one or more intermediate extractions includes a plurality of intermediate extractions for the respective activity episode, and

producing the episode summary for the respective activity episode further comprises;

after generating the plurality of intermediate extractions for the respective activity episode, performing a pairwise similarity comparison of the intermediate extractions for the respective activity episode;

assigning, based on the pairwise similarity comparison, numerical weightings to the intermediate extractions for the respective activity episode; and

selecting, based on the numerical weightings assigned to the intermediate extractions for the respective activity episode, the particular intermediate extraction from among the plurality of intermediate extractions for the respective activity episode.