Speech recognition and summarization

• US 10,679,005 B2
• Filed: 10/30/2019
• Issued: 06/09/2020
• Est. Priority Date: 09/10/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Shires, Glen, Swigart, Sterling, Zolla, Jonathan, Gauci, Jason J.
• Primary Examiner(s): Guerra-Erazo, Edgar X
• Application Number:
  US16/669,125
• Publication Number:
  US 20200065379A1
• Time in Patent Office: 223 Days
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 21/10 Transforming into visible information
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

The subject matter of this specification can be embodied in, among other things, a method that includes receiving two or more data sets each representing speech of a corresponding individual attending an internet-based social networking video conference session, decoding the received data sets to produce corresponding text for each individual attending the internet-based social networking video conference, and detecting characteristics of the session from a coalesced transcript produced from the decoded text of the attending individuals for providing context to the internet-based social networking video conference session.

Claims

1. A method comprising:

receiving, at data processing hardware of a conference system, speech data from a first computing device, the speech data representing an utterance spoken by a first participant of a communication session between the first participant and a second participant;

processing, by the data processing hardware, the speech data to;

transcribe, in real time, the speech data representing the spoken utterance spoken by the first participant of the communication session in real-time; and

identify, in real-time, a topic of the communication session by analyzing one or more words and/or phrases in the text of the speech data;

obtaining, by the data processing hardware, a particular search result associated with the determined topic of the communication session;

generating, by the data processing hardware, a user interface component for the particular search result associated with the determined topic of the communication session; and

outputting, by the data processing hardware, the user interface component for the particular search result to the first computing device in real-time, the user interface component when received by the first computing device causing the first computing device to display the user interface component on a first graphical user interface executing on the first computing device.

2. The method of claim 1, further comprising transmitting, by the data processing hardware, the text of the speech data to a second computing device associated with the second participant, the text of the speech data when received by the second computing device causing the second computing device to display the text of the speech data on a second graphical user interface executing on the second computing device.

3. The method of claim 2, further comprising outputting, by the data processing hardware, the user interface component for the particular search result to the second computing device in real-time, the user interface component when received by the o second computing device causing the second computing device to display the user interface component on the second graphical user interface.

4. The method of claim 1, wherein obtaining the particular search result comprises:

obtaining, from a search engine, one or more search results that are identified as a result of performing a query using one or more relevant terms in the text of the speech data as being potentially relevant to the determined topic; and

selecting the particular search result from among the one or more search results identified as the result of performing the query.

5. The method of claim 1, wherein obtaining the particular search result comprises obtaining a particular phone number as a result of performing a query using one or more relevant terms in the text of the speech data as being potentially relevant to the determined topic.

6. The method of claim 1, wherein obtaining the particular search result comprises obtaining a particular email address as a result of performing a query using one or more relevant terms in the text of the speech data as being potentially relevant to the determined topic.

7. The method of claim 1, wherein generating the user interface component comprises generating a hyperlink for the particular search result associated with the determined topic of the communication session.

8. The method of claim 1, wherein generating the user interface component comprises:

identifying a location associated with the determined topic of the communication session; and

generating a map image associated with the identified location.

9. The method of claim 1, wherein generating the user interface component comprises:

identifying a location associated with the determined topic of the communication session; and

generating a hyperlink to a map image associated with the identified location.

10. The method of claim 1, wherein the data processing hardware resides on a server system, the server system in communication with the first computing device and a second computing device via network, the second computing device associated with the second participant of the communication session.

11. A conference system comprising:

data processing hardware; and

memory hardware in communication with the data processing hardware and storing instructions, that when executed by the data processing hardware, cause the data processing hardware to perform operations comprising;

receiving speech data from a first computing device, the speech data representing an utterance spoken by a first participant of a communication session between the first participant and a second participant;

processing the speech data to;

transcribe, in real time, the speech data representing the spoken utterance spoken by the first participant of the communication session in real-time; and

identify, in real-time, a topic of the communication session by analyzing one or more words and/or phrases in the text of the speech data;

obtaining a particular search result associated with the determined topic of the communication session;

generating a user interface component for the particular search result associated with the determined topic of the communication session; and

outputting the user interface component for the particular search result to the first computing device in real-time, the user interface component when received by the first computing device causing the first computing device to display the user interface component on a first graphical user interface executing on the first computing device.

12. The conference system of claim 11, wherein the operations further comprise transmitting the text of the speech data to a second computing device associated with the second participant, the text of the speech data when received by the second computing device causing the second computing device to display the text of the speech data on a second graphical user interface executing on the second computing device.

13. The conference system of claim 12, wherein the operations further comprise outputting the user interface component for the particular search result to the second computing device in real-time, the user interface component when received by the second computing device causing the second computing device to display the user interface component on the second graphical user interface.

14. The conference system of claim 11, wherein obtaining the particular search result comprises:

obtaining, from a search engine, one or more search results that are identified as a result of performing a query using one or more relevant terms in the text of the speech data as being potentially relevant to the determined topic; and

selecting the particular search result from among the one or more search results identified as the result of performing the query.

15. The conference system of claim 11, wherein obtaining the particular search result comprises obtaining a particular phone number as a result of performing a query using one or more relevant terms in the text of the speech data as being potentially relevant to the determined topic.

16. The conference system of claim 11, wherein obtaining the particular search result comprises obtaining a particular email address as a result of performing a query using one or more relevant terms in the text of the speech data as being potentially relevant to the determined topic.

17. The conference system of claim 11, wherein generating the user interface component comprises generating a hyperlink for the particular search result associated with the determined topic of the communication session.

18. The conference system of claim 11, wherein generating the user interface component comprises:

identifying a location associated with the determined topic of the communication session; and

generating a map image associated with the identified location.

19. The conference system of claim 11, wherein generating the user interface component comprises:

identifying a location associated with the determined topic of the communication session; and

generating a hyperlink to a map image associated with the identified location.

20. The conference system of claim 11, wherein the data processing hardware resides on a server system, the server system in communication with the first computing device and a second computing device via network, the second computing device associated with the second participant of the communication session.