Automated cognitive recording and organization of speech as structured text

• US 10,535,352 B2
• Filed: 11/16/2017
• Issued: 01/14/2020
• Est. Priority Date: 11/16/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Pankanti, Sharathchandra, Ravizza, Stefan, Rueger, Erik
• Primary Examiner(s): Han, Qi
• Application Number:
  US15/814,677
• Publication Number:
  US 20190147882A1
• Time in Patent Office: 789 Days
• Field of Search: 704231, 704235, 704243, 704246, 704248, 704250, 704253, 704257, 704270, 7042701
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04M 3/567 Multimedia conference systems
• CPC Class Codes: H04M 3/568 audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30

Abstract

A computer-implemented method includes associating, using a processor, one or more words in an electronic agenda template to at least one agenda item indicative of a point for discussion. The processor captures a real-time interaction comprising speech from one or more participants of a plurality of discussion participants into a digital representation. The processor isolates a portion of the real-time interaction from the digital representation. The portion is associated with a single speaker of the plurality of discussion participants. The processor makes at least one match between an isolated portion of the real-time interaction and the at least one agenda item. The processor determines an intent of the single speaker from the isolated portion and matching the determined intent of the single speaker to the at least one agenda item on the electronic agenda template, and generates discussion minutes output based on the matched intent and agenda item.

Claims

1. A computer-implemented method for converting speech into structured text, the speech being generated by a plurality of sources, the computer-implemented method comprising:

associating, using a processor, one or more words in an electronic agenda template to at least one agenda item indicative of a point for discussion;

capturing, using the processor, a real-time interaction comprising speech from one or more participants of a plurality of discussion participants into a digital representation;

isolating, using the processor, a portion of the real-time interaction from the digital representation, wherein the portion is associated with a single speaker of the plurality of discussion participants, and making at least one match between an isolated portion of the real-time interaction and the at least one agenda item;

determining, using the processor, an intent of the single speaker from the isolated portion and matching the determined intent of the single speaker to the at least one agenda item on the electronic agenda template; and

generating, using the processor, a discussion minutes output based on the matched intent and agenda item.

2. The computer-implemented method of claim 1, wherein determining the intent of the single speaker comprises, after capturing the real-time interaction:

receiving, using the processor, a spoken communication indicative of an intent to start or stop speaking; and

granting or terminating, using the processor, discussion control to the single speaker, wherein the discussion control allows or disallows the single speaker to speak during the real-time interaction.

3. The computer-implemented method of claim 1, wherein the structuring is based on a predefined template.

4. The computer-implemented method of claim 1, wherein the structuring receives Natural Language Processing (NLP) as input.

5. The computer-implemented method of claim 1, wherein generating the discussion minutes output comprises:

determining, using the processor, whether there is missing information on the electronic agenda template;

outputting, using the processor, a query for the missing information on the electronic agenda template to the plurality of speakers, in natural language, during the real-time interaction;

receiving, using the processor, an input in natural language from one or more of the plurality of speakers, the input indicative of a response to the query;

matching, using the processor, the input indicative of the response to the missing information;

saving, to a computer memory, a digital representation of the input to the discussion minutes; and

generating, using the processor, the discussion minutes output based on the electronic agenda template.

6. The computer-implemented method of claim 5, wherein the discussion minutes output is NLP output.

7. The computer-implemented method of claim 6, further comprising:

receiving, using the processor, a NLP input to search the digital representation for an agenda item; and

generating, using the processor, an NLP output indicative of the agenda item.

8. A system for converting speech into structured text, comprising:

a processor configured to;

associate one or more words in an electronic agenda template to at least one agenda item indicative of a point for discussion;

capture a real-time interaction comprising speech from one or more participants of the plurality of discussion participants into a digital representation;

isolate a portion of the real-time interaction from the digital representation, wherein the portion is associated with a single speaker of the plurality of discussion participants, and make at least one match between an isolated portion of the real-time interaction and the at least one agenda item;

determine an intent of the single speaker from the isolated portion and matching the determined intent of the single speaker to the at least one agenda item on the electronic agenda template; and

generate a discussion minutes output based on the matched intent and agenda item.

9. The system of claim 8, wherein determining the intent of the single speaker comprises, after capturing the real-time interaction:

receiving, using the processor, a spoken communication indicative of an intent to start or stop speaking; and

granting or terminating, using the processor, discussion control to the single speaker, wherein the discussion control allows or disallows the single speaker to speak during the real-time interaction.

10. The system of claim 8, wherein the structuring is based on a predefined template.

11. The system of claim 8, wherein the structuring receives Natural Language Processing (NLP) as input.

12. The system of claim 8, wherein generating the discussion minutes output comprises:

determining, using the processor, whether there is missing information on the electronic agenda template;

outputting, using the processor, a query for the missing information on the electronic agenda template to the plurality of speakers, in natural language, during the real-time interaction;

receiving, using the processor, an input in natural language from one or more of the plurality of speakers, the input indicative of a response to the query;

matching, using the processor, the input indicative of the response to the missing information;

saving, to a computer memory, a digital representation of the input to the discussion minutes; and

generating, using the processor, the discussion minutes output based on the electronic agenda template.

13. The system of claim 12, wherein the discussion minutes output is NLP output.

14. The system of claim 13, further comprising:

receiving, using the processor, a NLP input to search the digital representation for an agenda item; and

generating, using the processor, an NLP output indicative of the agenda item.

15. A computer program product for structuring and recording discussion minutes of a plurality of discussion participants during a discussion, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform a method comprising:

associating, using a processor, one or more words in an electronic agenda template to at least one agenda item indicative of a point for discussion;

capturing, using the processor, a real-time interaction comprising speech from one or more participants of the plurality of discussion participants into a digital representation;

isolating, using the processor, a portion of the real-time interaction from the digital representation, wherein the portion is associated with a single speaker of the plurality of discussion participants, and making at least one match between an isolated portion of the real-time interaction and the at least one agenda item;

determining, using the processor, an intent of the single speaker from the isolated portion and matching the determined intent of the single speaker to the at least one agenda item on the electronic agenda template; and

generating, using the processor, a discussion minutes output based on the matched intent and agenda item.

16. The computer program product of claim 15, wherein determining the intent of the single speaker comprises, after capturing the real-time interaction:

receiving, using the processor, a spoken communication indicative of an intent to start or stop speaking; and

granting or terminating, using the processor, discussion control to the single speaker, wherein the discussion control allows or disallows the single speaker to speak during the real-time interaction.

17. The computer program product of claim 15, wherein the structuring is based on a predefined template.

18. The computer program product of claim 15, wherein the structuring receives Natural Language Processing (NLP) as input.

19. The computer program product of claim 15, wherein generating the discussion minutes output comprises:

determining, using the processor, whether there is missing information on the electronic agenda template;

outputting, using the processor, a query for the missing information on the electronic agenda template to the plurality of speakers, in natural language, during the real-time interaction;

receiving, using the processor, an input in natural language from one or more of the plurality of speakers, the input indicative of a response to the query;

matching, using the processor, the input indicative of the response to the missing information;

saving, to a computer memory, a digital representation of the input to the discussion minutes; and

generating, using the processor, the discussion minutes output based on the electronic agenda template.

20. The computer program product of claim 19, wherein the discussion minutes output is NLP output.