Speech recognition and summarization

• US 9,420,227 B1
• Filed: 11/13/2013
• Issued: 08/16/2016
• Est. Priority Date: 09/10/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Shires, Glen, Swigart, Sterling, Zolla, Jonathan, Gauci, Jason J.
• Primary Examiner(s): Guerra-Erazo, Edgar
• Application Number:
  US14/078,800
• Time in Patent Office: 1,007 Days
• Field of Search: 704/9, 704/10, 704/235, 704/243, 704/246, 704/270, 704/270.1, 704/275
• US Class Current: 1/1
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 21/10 Transforming into visible information
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

The subject matter of this specification can be embodied in, among other things, a method that includes receiving two or more data sets each representing speech of a corresponding individual attending an internet-based social networking video conference session, decoding the received data sets to produce corresponding text for each individual attending the internet-based social networking video conference, and detecting characteristics of the session from a coalesced transcript produced from the decoded text of the attending individuals for providing context to the internet-based social networking video conference session.

Claims

1. A computer-implemented method comprising:

receiving video conference data relating to a video conference;

determining a likely discussion topic associated with the video conference based on the video conference data, comprising generating, by an automated speech recognizer, a transcription of at least a portion of the video conference using at least a portion of the video conference data;

identifying a resource based on the likely discussion topic associated with the video conference; and

providing, by one or more computers, a representation of the resource for output to one or more participants of the video conference.

2. The computer-implemented method of claim 1, wherein determining the likely discussion topic associated with the video conference based on the video conference data comprises:

determining one or more annotations that describe attributes of the video conference data;

annotating the video conference data with the one or more annotations that describe attributes of the video conference data; and

determining the likely discussion topic associated with the video conference based on the annotated video conference data.

3. The computer-implemented method of claim 1, wherein the resource identified based on the likely discussion topic associated with the video conference comprises advertising content that corresponds to the likely discussion topic associated with the video conference.

4. The computer-implemented method of claim 1, wherein identifying the resource based on the likely discussion topic associated with the video conference comprises:

identifying one or more terms associated with the likely discussion topic associated with the video conference;

obtaining, from a search module, one or more search results that are identified as a result of performing a query using one or more of the terms associated with the likely discussion topic associated with the video conference; and

selecting a resource associated with a particular search result that is selected from among the one or more search results identified as a result of performing the query.

5. The computer-implemented method of claim 1, wherein the resource identified based on the likely discussion topic associated with the video conference comprises at least one of a resource identifying an event that is associated with the likely discussion topic associated with the video conference or a resource identifying a location that is associated with the likely discussion topic associated with the video conference.

6. The computer-implemented method of claim 1, wherein providing the representation of the resource for output to the one or more participants of the video conference comprises providing the representation of the resource for output to the one or more participants of the video conference in a context region of an interface associated with the video conference.

7. The computer-implemented method of claim 1, comprising:

identifying a first portion of the video conference data that corresponds to a first segment of the video conference, and (ii) a second portion of the video conference data that corresponds to a second segment of the video conference;

determining (i) a first likely discussion topic that corresponds to the first segment of the video conference based on the first portion of the video conference data, and (ii) a second likely discussion topic that corresponds to the second segment of the video conference based on the second portion of the video conference data, wherein determining the first likely discussion topic that corresponds to the first segment of the video conference comprises generating, by the automated speech recognizer, a transcription of at least the first segment of the video conference using at least the first portion of the video conference data, and determining the second likely discussion topic that corresponds to the second segment of the video conference comprises generating, by the automated speech recognizer, a transcription of at least the segment of the video conference using at least the second portion of the video conference data;

identifying the resource based on at least one of the first likely discussion topic that corresponds to the first segment of the video conference or the second likely discussion topic that corresponds to the second segment of the video conference; and

providing the representation of the resource for output to the one or more participants of the video conference.

8. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving video conference data relating to a video conference;

determining a likely discussion topic associated with the video conference based on the video conference data, comprising generating, by an automated speech recognizer, a transcription of at least a portion of the video conference using at least a portion of the video conference data;

identifying a resource based on the likely discussion topic associated with the video conference; and

providing a representation of the resource for output to one or more participants of the video conference.

9. The system of claim 8, wherein determining the likely discussion topic associated with the video conference based on the video conference data comprises:

determining one or more annotations that describe attributes of the video conference data;

annotating the video conference data with the one or more annotations that describe attributes of the video conference data; and

determining the likely discussion topic associated with the video conference based on the annotated video conference data.

10. The system of claim 8, wherein the resource identified based on the likely discussion topic associated with the video conference comprises advertising content that corresponds to the likely discussion topic associated with the video conference.

11. The system of claim 8, wherein identifying the resource based on the likely discussion topic associated with the video conference comprises:

identifying one or more terms associated with the likely discussion topic associated with the video conference;

obtaining, from a search module, one or more search results that are identified as a result of performing a query using one or more of the terms associated with the likely discussion topic associated with the video conference; and

selecting a resource associated with a particular search result that is selected from among the one or more search results identified as a result of performing the query.

12. The system of claim 8, wherein the resource identified based on the likely discussion topic associated with the video conference comprises at least one of a resource identifying an event that is associated with the likely discussion topic associated with the video conference or a resource identifying a location that is associated with the likely discussion topic associated with the video conference.

13. The system of claim 8, wherein providing the representation of the resource for output to the one or more participants of the video conference comprises providing the representation of the resource for output to the one or more participants of the video conference in a context region of an interface associated with the video conference.

14. The system of claim 8, wherein the operations comprise:

identifying a first portion of the video conference data that corresponds to a first segment of the video conference, and (ii) a second portion of the video conference data that corresponds to a second segment of the video conference;

determining (i) a first likely discussion topic that corresponds to the first segment of the video conference based on the first portion of the video conference data, and (ii) a second likely discussion topic that corresponds to the second segment of the video conference based on the second portion of the video conference data, wherein determining the first likely discussion topic that corresponds to the first segment of the video conference comprises generating, by the automated speech recognizer, a transcription of at least the first segment of the video conference using at least the first portion of the video conference data, and determining the second likely discussion topic that corresponds to the second segment of the video conference comprises generating, by the automated speech recognizer, a transcription of at least the segment of the video conference using at least the second portion of the video conference data;

identifying the resource based on at least one of the first likely discussion topic that corresponds to the first segment of the video conference or the second likely discussion topic that corresponds to the second segment of the video conference; and

providing the representation of the resource for output to the one or more participants of the video conference.

15. A computer-readable storage device storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving video conference data relating to a video conference;

determining a likely discussion topic associated with the video conference based on the video conference data, comprising generating, by an automated speech recognizer, a transcription of at least a portion of the video conference using at least a portion of the video conference data;

identifying a resource based on the likely discussion topic associated with the video conference; and

providing a representation of the resource for output to one or more participants of the video conference.

16. The computer-readable storage device of claim 15 wherein determining the likely discussion topic associated with the video conference based on the video conference data comprises:

determining one or more annotations that describe attributes of the video conference data;

annotating the video conference data with the one or more annotations that describe attributes of the video conference data; and

determining the likely discussion topic associated with the video conference based on the annotated video conference data.

17. The computer-readable storage device of claim 15, wherein the resource identified based on the likely discussion topic associated with the video conference comprises advertising content that corresponds to the likely discussion topic associated with the video conference.

18. The computer-readable storage device of claim 15, wherein identifying the resource based on the likely discussion topic associated with the video conference comprises:

identifying one or more terms associated with the likely discussion topic associated with the video conference;

obtaining, from a search module, one or more search results that are identified as a result of performing a query using one or more of the terms associated with the likely discussion topic associated with the video conference; and

selecting a resource associated with a particular search result that is selected from among the one or more search results identified as a result of performing the query.

19. The computer-readable storage device of claim 15, wherein the resource identified based on the likely discussion topic associated with the video conference comprises at least one of a resource identifying an event that is associated with the likely discussion topic associated with the video conference or a resource identifying a location that is associated with the likely discussion topic associated with the video conference.

20. The computer-readable storage device of claim 15, wherein providing the representation of the resource for output to the one or more participants of the video conference comprises providing the representation of the resource for output to the one or more participants of the video conference in a context region of an interface associated with the video conference.