Providing customer service agents with customer-personalized result of spoken language intent

• US 10,417,345 B1
• Filed: 12/22/2014
• Issued: 09/17/2019
• Est. Priority Date: 12/22/2014
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Carlson, Ty Loren, Lai, Hsuan-Cheng
• Primary Examiner(s): Washburn, Daniel C
• Assistant Examiner(s): Ogunbiyi, Oluwadamilola M
• Application Number:
  US14/579,214
• Time in Patent Office: 1,730 Days
• Field of Search: 715716, 704 9
• CPC Class Codes: G06F 16/638 Presentation of query results
• CPC Class Codes: G06F 16/90335 Query processing
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 40/237 Lexical tools
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30

Abstract

A system in which a customer service agent (CSA) is able to assist a customer with obtaining a desired response from a speech-controlled appliance while protecting customer data. The customer service agent submits queries to a natural language understanding (NLU) processor that performs entity resolution using personalized library information stored in an entity library based on the customer identity information and/or an device identifier. The CSA is shielded from the entity library itself, as well as data stored on the speech-controlled appliance. The CSA can instruct the NLU processor to deliver results to multiple endpoints, including both the customer'"'"'s appliance and the CSA agent'"'"'s console.

Claims

1. A method for assisting a user with resolving a speech-based request, the method comprising:

receiving, at a first device at a first time, a first utterance, wherein;

the first device is associated with a customer identifier, and

the first utterance includes a first command to contact a separate customer service device operated by a customer service agent;

receiving, at the customer service device, the customer identifier associated with the first device;

based at least in part on input by the customer service agent, sending, from the customer service device to a computer-implemented natural language processing system that is separate from the first device, a text query, the customer identifier, and a designation of at least one network-connected device including the customer service device;

determining, by the natural language processing system, that the customer identifier is associated with a first domain;

determining, by the natural language processing system and based at least in part on the customer identifier, that first user specific lexical information stored in at least one data storage device is associated with the first domain;

determining, by the natural language processing system, that the text query corresponds to a first intent associated with the first domain and a first field associated with the first intent;

determining, by the natural language processing system and based at least in part on the first user specific lexical information and a first model associated with the first domain, a first entity corresponding to the first field;

sending a first interpretation of the text query from the natural language processing system to the customer service device, the first interpretation corresponding to the first intent and the first entity; and

displaying the first interpretation on a screen of the customer service device.

2. The method of claim 1, wherein determining the first entity further comprises:

calculating a plurality of probabilities, where each probability is a probability that a respective interpretation of the text query is correct, wherein the calculating uses respective similarities between words in the respective interpretation of the text query and the user specific lexical information.

3. The method of claim 1, further comprising:

receiving, at the first device at a second time after the first time, a second utterance corresponding to a second command that was recommended by the customer service agent based on the first interpretation;

sending data corresponding to the second utterance to the natural language processing system;

determining, by the natural language processing system and based at least in part on the user specific lexical information, that the second utterance includes the second command; and

causing the first device to perform at least one action corresponding to the second command.

4. A computer-implemented natural language processing system comprising:

at least one data storage device;

at least one processor; and

a computer-readable medium encoded with instructions which, when executed by the at least one processor, cause the natural language processing system to;

receive a text query and an entity identifier from a first network-connected device; and

in response to receiving the text query;

determine that the entity identifier is associated with a first domain,

determine, based at least in part on the entity identifier, that first user specific lexical information stored in the at least one data storage device is associated with the first domain,

determine that the text query corresponds to a first intent associated with the first domain and a first field associated with the first intent,

determine, based at least in part on the first user specific lexical information and a first model associated with the first domain, a first entity corresponding to the first field,

prepare a first result for the text query, the first result comprising data representing the first entity,

prepare a second result for the text query, where the second result does not include the data representing the first entity,

send a first message to the first network-connected device, the first message causing the first network connected device to output a first representation of the first result, and

send a second message to a second network-connected device that is different than the first network-connected device, the second message causing the second network connected device to output a second representation of the second result.

5. The computer-implemented natural language processing system of claim 4, wherein the entity identifier identifies one or more of a customer account, a client device, or a household associated with the client device.

6. The computer-implemented natural language processing system of claim 4, wherein the computer-readable medium is encoded with additional instructions which, when executed by the at least one processor, further cause the system to:

determine that the entity identifier is associated with a second domain;

determine, based at least in part on the entity identifier, that second user specific lexical information stored in at least one data storage device is associated with the second domain;

determine that the text query corresponds to a second intent associated with the second domain and a second field associated with the second intent; and

determine, based at least in part on the second user specific lexical information and a second model associated with the second domain, a second entity corresponding to the second field.

7. The computer-implemented natural language processing system of claim 6, wherein the computer-readable medium is encoded with additional instructions which, when executed by the at least one processor, further cause the system to:

cause the second result to include the second entity.

8. The computer-implemented natural language processing system of claim 4, wherein the computer-readable medium is encoded with additional instructions which, when executed by the at least one processor, further cause the system to:

receive a designation of a plurality of network-connected devices to be transmitted results, the plurality of network-connected devices including the first network-connected device and the second network-connected device; and

cause the natural language processing system to send the first message to the first network-connected device and to send the second message to the second network-connected device based at least in part on the designation.

9. The computer-implemented natural language processing system of claim 4, wherein:

the first network-connected-device is associated with a customer service agent; and

the second network-connected device is associated with the entity identifier.

10. The computer-implemented natural language processing system of claim 4, wherein the first result comprises at least one of:

a second entity corresponding to the first field;

or

a second intent corresponding to the text query.

11. A method performed by a computer-implemented natural language processing system, comprising:

receiving, from a first network-connected device, a text query and an entity identifier; and

in response to receiving the text query;

determining that the entity identifier is associated with a first domain,

determining, based at least in part on the entity identifier, that first user specific lexical information stored in at least one data storage device is associated with the first domain,

determining that the text query corresponds to a first intent associated with the first domain and a first field associated with the first intent,

determining, based at least in part on the first user specific lexical information and a first model associated with the first domain, a first entity corresponding to the first field,

preparing a first result for the text query, the first result comprising data representing the first entity,

preparing a second result for the text query, where the second result does not include the data representing the first entity,

sending a first message to the first network-connected device, the first message causing the first network connected device to output a first representation of the first result, and

sending a second message to a second network-connected device that is different than the first network-connected device, the second message causing the second network connected device to output a second representation of the second result.

12. The method of claim 11, wherein the entity identifier identifies one or more of a customer account, a client device, or a household associated with the client device.

13. The method of claim 11, further comprising:

determining that the entity identifier is associated with a second domain;

determining, based at least in part on the entity identifier, that second user specific lexical information stored in at least one data storage device is associated with the second domain;

determining that the text query corresponds to a second intent associated with the second domain and a second field associated with the second intent; and

determining, based at least in part on the second user specific lexical information and a second model associated with the second domain, a second entity corresponding to the second field.

14. The computer implemented method of claim 13, further comprising:

causing the second result to include data representing the second entity.

15. The method of claim 11, further comprising:

receiving a designation of a plurality of network-connected devices to be transmitted results, the plurality of network-connected devices including the first network-connected device and the second network-connected device;

wherein sending the first message to the first network-connected device and sending the second message to the second network-connected device are based at least in part on the designation.

16. The method of claim 11, wherein:

the first network-connected-device is associated with a customer service agent; and

the second network-connected device is associated with the entity identifier.

17. The method of claim 11, wherein the first result comprises at least one of:

a second entity corresponding to the first field;

or

a second intent corresponding to the text query.