Accelerating agent performance in a natural language processing system

• US 10,621,282 B1
• Filed: 04/26/2018
• Issued: 04/14/2020
• Est. Priority Date: 10/27/2017
• Status: Active Grant

• Current Assignee: Interactions, LLC
• Original Assignee: Interactions, LLC
• Inventors: Selfridge, Ethan, Johnston, Michael, Lifgren, Robert, Dreher, James, Leonard, John
• Primary Examiner(s): He, Jialong
• Application Number:
  US15/963,915
• Time in Patent Office: 719 Days
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/56 Natural language generation
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/221 Announcement of recognition results
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A computer-implemented method for providing agent assisted transcriptions of user utterances. A user utterance is received in response to a prompt provided to the user at a remote client device. An automatic transcription is generated from the utterance using a language model based upon an application or context, and presented to a human agent. The agent reviews the transcription and may replace at least a portion of the transcription with a corrected transcription. As the agent inputs the corrected transcription, accelerants are presented to the user comprising suggested texted to be inputted. The accelerants may be determined based upon an agent input, an application or context of the transcription, the portion of the transcription being replaced, or any combination thereof. In some cases, the user provides textual input, to which the agent transcribes an intent associated with the input with the aid of one or more accelerants.

Claims

1. A computer-implemented method comprising:

receiving at an application at an application server, from a remote client device over a network, an utterance from a user at the remote client device;

generating an automatic transcription of the received utterance using a language model, the language model selected based upon an identified context of the application indicating an interaction state between the user at the remote client device and the application at the application server;

providing the automatic transcription over the network to a client device of an agent for presentation to the agent via a user interface;

receiving, from the agent via the user interface, an indication indicating replacement of at least a portion of the automatic transcription, comprising at least a piece of text for replacing or supplementing the portion of the automatic transcription;

identifying one or more suggestions comprising one or more pieces of suggested text for replacing or supplementing the portion of the automatic transcription, based upon the received indication and the identified context, by;

identifying a plurality of potential suggestions based upon the identified context;

identifying a suffix portion comprising at least a portion of the indication received from the agent;

identifying the one or more suggestions from the plurality of potential suggestions, each of the one or more suggestions having a prefix portion that matches with the suffix portion by at least a threshold amount;

generating and providing for display to the agent via the user interface, the one or more suggestions; and

generating an updated transcription of the utterance based upon the automatic transcription, the indication, and a selected one of the one or more suggestions.

2. The computer-implemented method of claim 1, wherein identifying one or more suggestions to the agent from the plurality of potential suggestions comprises:

identifying a first subset of suggestions having prefix portions that exactly match with the suffix portion, and

responsive to the first subset of suggestions not including a threshold number of suggestions, identifying a second subset of suggestions having prefix portions that do not exactly match with the suffix portion.

3. The computer-implemented method of claim 1, wherein the one or more suggestions are identified from the plurality of one or more suggestions based at least in part upon an auditory similarity to the portion of the automatic transcription to be replaced.

4. The computer-implemented method of claim 1, wherein the prefix portion of a suggestion of the one or more suggestions comprises a portion of the suggestion matching the identified suffix portion in length.

5. The computer-implemented method of claim 1, wherein the plurality of potential suggestions are generated using a language model selected based upon the identified context.

6. The computer-implemented method of claim 1, further comprising presenting a priming string to the agent via the user interface, wherein the priming string is generated based upon a prompt provided to the user at the client device and the identified context.

7. The computer-implemented method of claim 1, further comprising providing, over the network, a prompt to the user at the client device associated with the context, and wherein the utterance from the user is received in response to the provided prompt.

8. A language processing system comprising a processor and a memory in communication with the processor, the memory storing programming instructions executable by the processor to:

receive a user utterance, wherein the user utterance is provided by a user at a remote client device as part of a series of interactions between the user and an application at an application server, and is associated with an identified context of the application;

generate an automatic transcription of the received utterance using a language model selected based upon the context;

provide the automatic transcription over the network to an agent device of an agent for presentation to the agent via a user interface;

generate, in response to receiving from the agent via the user interface an indication comprising at least a piece of text for replacing or supplementing a portion of the automatic transcription, one or more suggestions comprising one or more pieces of suggested text for replacing or supplementing the portion of the automatic transcription, based upon the received indication and the identified context, by;

identifying a plurality of potential suggestions based upon the identified context;

identifying a suffix portion, the suffix portion comprising at least a portion of the indication received from the agent;

identifying the one or more suggestions from the plurality of potential suggestions, each of the one or more suggestions having a prefix portion that matches with the suffix portion by at least a threshold amount;

transmit the one or more suggestions to the agent device for display to the agent via the user interface, and receive an updated transcription of the utterance based upon the automatic transcription, the indication, and a selected one of the one or more suggestions.

9. The language processing system of claim 8, wherein identifying one or more suggestions to the agent from the plurality of potential suggestions comprises:

identifying a first subset of suggestions having prefix portions that exactly match with the suffix portion, and

responsive to the first subset of suggestions not including a threshold number of suggestions, identifying a second subset of suggestions having prefix portions that do not exactly match with the suffix portion.

10. The language processing system of claim 8, wherein the one or more suggestions are identified from the plurality of one or more suggestions based at least in part upon an auditory similarity to the portion of the automatic transcription to be replaced.

11. The language processing system of claim 8, wherein the prefix portion of a suggestion of the one or more suggestions comprises a portion of the suggestion matching the identified suffix portion in length.

12. The language processing system of claim 8, wherein the plurality of potential suggestions are generated using a language model selected based upon the context.

13. The language processing system of claim 8, wherein the programming instructions are further executable by the processor to generate a priming string based upon a prompt provided to the user at the client device and the context, and to provide over the network to the agent device the priming string for display to the agent via the user interface.

14. The language processing system of claim 8, wherein the programming instructions are further executable by the processor to receive the utterance from the user in response to a prompt provided to the user.

15. A non-transitory computer-readable storage medium having instructions stored which, when executed by a computing device, result in the computing device performing operations comprising:

receiving at an application at an application server, from a remote client device over a network, an utterance from a user at the remote client device;

generating an automatic transcription of the received utterance using a language model, the language model selected based upon an identified context of the application indicating an interaction state between the user at the remote client device and the application at the application server;

providing the automatic transcription over the network to a client device of an agent for presentation to the agent via a user interface;

receiving, from the agent via the user interface, an indication indicating replacement or supplement of at least a portion of the automatic transcription, comprising at least a piece of text for replacing or supplementing the portion of the automatic transcription;

identifying one or more suggestions comprising one or more pieces of text for replacing or supplementing the portion of the automatic transcription, based upon the received indication and the identified context, by;

identifying a plurality of potential suggestions based upon the identified context;

identifying a suffix portion, the suffix portion comprising at least a portion of the indication received from the agent;

identifying the one or more suggestions from the plurality of potential suggestions, each of the one or more suggestions having a prefix portion that matches with the suffix portion by at least a threshold amount;

generating and providing for display to the agent via the user interface, the one or more suggestions; and

generating an updated transcription of the utterance based upon the automatic transcription, the indication, and a selected one of the one or more suggestions.

16. The non-transitory computer-readable storage medium of claim 15, wherein identifying one or more suggestions to the agent from the plurality of potential suggestions comprises:

identifying a first subset of suggestions having prefix portions that exactly match with the suffix portion, and

responsive to the first subset of suggestions not including a threshold number of suggestions, identifying a second subset of suggestions having prefix portions that do not exactly match with the suffix portion.

17. The non-transitory computer-readable storage medium of claim 15, wherein the one or more suggestions are identified from the plurality of one or more suggestions based at least in part upon an auditory similarity to the portion of the automatic transcription to be replaced.

18. The non-transitory computer-readable storage medium of claim 15, wherein the prefix portion of a suggestion of the one or more suggestions comprises a portion of the suggestion matching the identified suffix portion in length.

19. The non-transitory computer-readable storage medium of claim 15, further comprising presenting a priming string to the agent via the user interface, wherein the priming string is generated based upon a prompt provided to the user at the client device and the identified context.

20. The non-transitory computer-readable storage medium of claim 15, further comprising providing, over the network, a prompt to the user at the client device associated with the context, and wherein the utterance from the user is received in response to the provided prompt.