Audio recall during voice conversations

• US 9,325,809 B1
• Filed: 08/22/2013
• Issued: 04/26/2016
• Est. Priority Date: 09/07/2012
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: MindMeld, Inc. (Cisco Systems, Inc.)
• Inventors: Barros, Brett, Jheeta, Moninder, Kola, George Samuel, Tuttle, Timothy
• Primary Examiner(s): Cheema, Umar
• Assistant Examiner(s): Tolchinsky, Gregory P
• Application Number:
  US13/973,922
• Time in Patent Office: 978 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04L 65/1083 In-session procedures
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04M 2201/40 using speech recognition speech recognition per se G10L15/00
• CPC Class Codes: H04M 2203/305 Recording playback features, e.g. increased speed
• CPC Class Codes: H04M 3/00 Automatic or semi-automatic exchanges
• CPC Class Codes: H04M 3/42221 Conversation recording systems at the subscriber's set H04M1/656
• CPC Class Codes: H04M 3/563 User guidance or feature selection

Abstract

Conversation information is provided to a conversation participant. Audio data is received a client device associated with a conversation participant. The client device maintains a buffer of received audio data, for instance audio data received in a previous interval of time. A recall request is received by the client device for information associated with the buffered audio data. The buffered audio data is analyzed responsive to the recall request to identify the requested information. The identified information is then presented at the client device.

Claims

1. A computer-implemented method for providing conversation information to participants of a conversation, comprising:

buffering, at a client device, audio data received from one or more conversation participants;

receiving, at the client device, a recall request for information associated with the buffered audio data from a conversation participant;

without pausing the conversation;

notifying client devices associated with other conversation participants of the recall request, each client device associated with another conversation participant configured to buffer conversation data received from one or more conversation participants;

receiving, from the notified client devices, information representative of buffered conversation data;

analyzing, by the client device, the buffered audio data and the received information representative of buffered conversation data to identify one or more topics of the conversation associated with the analyzed data;

identifying content associated with the identified one or more topics; and

presenting, at the client device in real time, the identified content.

2. The computer-implemented method of claim 1, further comprising:

capturing, by the client device, audio data from a user of the client device participating in the conversation.

3. The computer-implemented method of claim 1, wherein buffering audio data comprises storing the received audio data in conjunction with time data representing a time at which the audio data was received.

4. The computer-implemented method of claim 1, wherein analyzing the buffered audio data and the received information representative of buffered conversation data comprises:

determining one or more conversation parameters associated with the conversation.

5. The computer-implemented method of claim 4, wherein determining one or more conversation parameters associated with the conversation:

converting the buffered audio data into a text transcript;

identifying keywords within the text transcript relevant to the conversation; and

identifying one or more topics of the conversation based on the identified keywords.

6. The computer-implemented method of claim 5, wherein the identified keywords comprise one or more of:

nouns, noun phrases, verbs, and verb phrases that appear within the text transcript over a threshold frequency or a threshold number of instances;

terms predetermined to be relevant;

terms dynamically determined to be relevant; and

terms associated with brands, products, services, or other commercial interests.

7. The computer-implemented method of claim 5, wherein identifying one or more topics of the conversation based on the identified keywords comprises:

identifying a topic keyword based on the identified keywords, the topic keyword not included in the identified keywords, wherein an identified topic of conversation comprises the identified topic keyword.

8. The computer-implemented method of claim 4, wherein determining one or more conversation parameters associated with the conversation comprises:

accessing one or more of the buffered audio data and characteristics or information associated with conversation participants; and

determining one or more of relationships between conversation participants, measures of affinity between conversation participants, and a context for conversation participants.

9. The computer-implemented method of claim 4, wherein the determined one or more conversation parameters comprise one or more of:

concepts, names, places, entities, and phrases.

10. The computer-implemented method of claim 1, wherein the client device and the one or more notified client devices are configured to provide text transcripts of audio data buffered by each client device to a central server, and further comprising:

receiving, from the central server, one or more conversation parameters, the central server configured to analyze the text transcripts provided by the client device and the notified client devices to identify the one or more conversation parameters.

11. The computer-implemented method of claim 1, wherein presenting the identified content comprises displaying the identified content on a client device display.

12. The computer-implemented method of claim 11, wherein the displayed content includes one or more of:

a time that a portion of audio data associated with the requested information was spoken by a conversation participant or captured by the client device, an identity of a conversation participant associated with the requested information, and information derived from the identified conversation parameters.

13. The computer-implemented method of claim 11, wherein presenting the identified content comprises presenting an interface configured to allow a user of the client device to perform a search based one or more identified conversation parameters.

14. A non-transitory computer-readable storage medium storing executable computer program instructions for providing conversation information to participants of a conversation, the instructions performing steps comprising:

buffering, at a client device, audio data received from one or more conversation participants;

receiving, at the client device, a recall request for information associated with the buffered audio data from a conversation participant;

without pausing the conversation;

notifying client devices associated with other conversation participants of the recall request, each client device associated with another conversation participant configured to buffer conversation data received from one or more conversation participants;

receiving, from the notified client devices, information representative of buffered conversation data;

analyzing, by the client device, the buffered audio data and the received information representative of buffered conversation data to identify one or more topics of the conversation associated with the analyzed data;

identifying content associated with the identified one or more topics; and

presenting, at the client device in real time, the identified content.

15. The computer-readable storage medium of claim 14, wherein the client device and the one or more notified client devices are configured to provide text transcripts of audio data buffered by each client device to a central server, and the instructions performing further steps comprising:

receiving, from the central server, one or more conversation parameters, the central server configured to analyze the text transcripts provided by the client device and the notified client devices to identify the one or more conversation parameters.

16. A computer system for providing conversation information to participants of a conversation, comprising:

a non-transitory computer readable storage medium storing executable computer program instructions comprising instructions for;

buffering, at the computer system, audio data received from one or more conversation participants;

receiving, at the computer system, a recall request for information associated with the buffered audio data from a conversation participant;

without pausing the conversation;

notifying client devices associated with other conversation participants of the recall request, each client device associated with other conversation participants configured to buffer conversation data received from one or more conversation participants;

receiving, from the notified client devices, information representative of buffered conversation data;

analyzing, by the client device, the buffered audio data and the received information representative of buffered conversation data to identify one or more topics of the conversation associated with the analyzed data;

identifying content associated with the identified one or more topics; and

presenting, at the client device in real time, the identified content; and

a processor for executing the computer program instructions.

17. The computer system of claim 16, wherein the computer system and the one or more notified client devices are configured to provide text transcripts of audio data buffered by the computer system and each notified client device to a central server, the instructions further comprising instructions for:

receiving, from the central server, one or more conversation parameters, the central server configured to analyze the text transcripts provided by the computer system and the notified client devices to identify the one or more conversation parameters.