Audio-based event interaction analytics

• US 10,043,517 B2
• Filed: 12/09/2015
• Issued: 08/07/2018
• Est. Priority Date: 12/09/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Blandin, Sebastien, Nandakumar, Karthik, Wynter, Laura
• Primary Examiner(s): Godbold, Douglas
• Application Number:
  US14/963,295
• Publication Number:
  US 20170169816A1
• Time in Patent Office: 972 Days
• Field of Search: None
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 25/48 specially adapted for particular use

Abstract

In an approach for audio based event analytics, a processor receives a recording of audio from an event. A processor collects information about the event and a list of participants. A processor segments the recording into, at least, a plurality of utterances. A processor analyzes the segmented recording. A processor summarizes the recording based on the segmentation and the analysis. A processor generates insights about interactions patterns of the event based on the segmentation and the analysis.

Claims

1. A method for audio based event analytics, the method comprising:

segmenting, by one or more processors, an audio recording from an event into, at least, a plurality of pauses and a plurality of acoustic utterances based, at least in part, on acoustic segmentation, wherein each acoustic utterance in the plurality of utterances is an individual word;

analyzing, by one or more processors, the plurality of acoustic utterances to identify at least a first keyword based, at least in part, on;

mining, by one or more processors, at least the first keyword from a plurality of information about the event; and

matching a phoneme sequence corresponding to the audio recording with the phoneme sequence corresponding to at least the first keyword;

extracting, by one or more processors, speech prosody characteristics and dialog acts from the plurality of acoustic utterances;

analyzing, by one or more processors, the speech prosody characteristics, the dialog acts, and at least the first keyword to determine a communication efficiency between a first participant and a second participant of the audio recording;

grouping, by one or more processors, at least a first acoustic utterance and a second acoustic utterance of the plurality of acoustic utterances into a snippet based, at least in part, on the first acoustic utterance and the second acoustic utterance being related to a topic;

determining, by one or more processors, that the topic associated with the snippet is relevant based, at least in part, on a frequency of occurrence of at least the first keyword included in the snippet; and

producing, by one or more processors, an audio summary of relevant snippets generated from the audio recording.

2. The method of claim 1, wherein determining, by one or more processors, a relevance of the snippet is further based, at least in part, on:

the plurality of information about the event and a list of participants; and

associating, by one or more processors, the topic with the snippet based on the plurality of information about the event.

3. The method of claim 1, further comprising

generating insights about interactions patterns of the event based, at least in part, on:

mapping, by one or more processors, an acoustic utterance in the plurality of acoustic utterances to the first participant and the second participant; and

identifying, by one or more processors, a type of interaction pattern between the first participant and the second participant based, at least in part, on the acoustic utterance.

4. The method of claim 1, wherein the information about the event is one or more documents containing text to be referenced during the event.

5. A computer program product for audio based event analytics, the computer program product comprising:

one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the program instructions comprising;

program instructions to segment, by one or more processors, an audio recording from an event into, at least, a plurality of pauses and a plurality of acoustic utterances based, at least in part, on acoustic segmentation, wherein each acoustic utterance in the plurality of utterances is an individual word;

program instructions to analyze the plurality of acoustic utterances to identify at least a first keyword based, at least in part, on;

program instructions to mine at least the first keyword from a plurality of information about the event;

program instructions to match a phoneme sequence corresponding to the audio recording with the phoneme sequence corresponding to at least the first keyword;

program instructions to extract speech prosody characteristics and dialog acts from the plurality of acoustic utterances;

program instructions to analyze the speech prosody characteristics, the dialog acts, and at least the first keyword to determine a communication efficiency between a first participant and a second participant of the audio recording;

program instructions to group at least a first acoustic utterance and a second acoustic utterance of the plurality of acoustic utterances into a snippet based, at least in part, on the first acoustic utterance and the second acoustic utterance being related to a topic;

program instructions to determine that the topic associated with the snippet is relevant based, at least in part, on a frequency of occurrence of at least the first keyword included in the snippet; and

program instructions to produce an audio summary of relevant snippets generated from the audio recording.

6. The computer program product of claim 5, wherein the program instructions to determine a relevance of the snippet is further based, at least in part, on:

the plurality of information about the event and a list of participants; and

program instructions to associate the topic with the snippet based on the plurality of information about the event.

7. The computer program product of claim 5, further comprising program instructions to generate insights about interactions patterns of the event based, at least in part, on:

program instructions to map an acoustic utterance in the plurality of acoustic utterances to the first participant and the second participant; and

program instructions to identify a type of interaction pattern between the first participant and the second participant based, at least in part, on the acoustic utterance.

8. The computer program product of claim 5, wherein the information about the event is one or more documents containing text to be referenced during the event.

9. A computer system for audio based event analytics, the computer system comprising:

one or more computer processors, one or more computer readable storage media, and program instructions stored on the one or more computer readable storage media for execution by at least one of the one or more processors, the program instructions comprising;

program instructions to segment, by one or more processors, an audio recording from an event into, at least, a plurality of pauses and a plurality of acoustic utterances based, at least in part, on acoustic segmentation, wherein each acoustic utterance in the plurality of utterances is an individual word;

program instructions to analyze the plurality of acoustic utterances to identify at least a first keyword based, at least in part, on;

program instructions to mine at least the first keyword from a plurality of information about the event;

program instructions to match a phoneme sequence corresponding to the audio recording with the phoneme sequence corresponding to at least the first keyword;

program instructions to extract speech prosody characteristics and dialog acts from the plurality of acoustic utterances; and

program instructions to analyze the speech prosody characteristics, the dialog acts, and at least the first keyword to determine a communication efficiency between a first participant and a second participant of the audio recording;

program instructions to group at least a first acoustic utterance and a second acoustic utterance of the plurality of acoustic utterances into a snippet based, at least in part, on the first acoustic utterance and the second acoustic utterance being related to a topic;

program instructions to determine that the topic associated with the snippet is relevant based, at least in part, on a frequency of occurrence of at least the first keyword included in the snippet; and

program instructions to produce an audio summary of relevant snippets generated from the audio recording.

10. The computer system of claim 9, wherein the program instructions to determine a relevance of the snippet is further based, at least in part, on:

the plurality of information about the event and a list of participants; and

program instructions to associate the topic with the snippet based on the plurality of information about the event.

11. The computer system of claim 9, further comprising program instructions to generate insights about interactions patterns of the event based, at least in part, on:

program instructions to map an acoustic utterance in the plurality of acoustic utterances to the first participant and the second participant; and

program instructions to identify a type of interaction pattern between the first participant and the second participant based, at least in part, on the acoustic utterance.