Environmentally aware dialog policies and response generation

• US 10,304,448 B2
• Filed: 02/21/2017
• Issued: 05/28/2019
• Est. Priority Date: 06/21/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Boies, Daniel, Heck, Larry, Anastasakos, Tasos, Sarikaya, Ruhi
• Primary Examiner(s): Abebe, Daniel
• Application Number:
  US15/438,227
• Publication Number:
  US 20170162201A1
• Time in Patent Office: 826 Days
• Field of Search: 704275
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces

Abstract

Environmental conditions, along with other information, are used to adjust a response of a conversational dialog system. The environmental conditions may be used at different times within the conversational dialog system. For example, the environmental conditions can be used to adjust the dialog manager'"'"'s output (e.g., the machine action). The dialog state information that is used by the dialog manager includes environmental conditions for the current turn in the dialog as well as environmental conditions for one or more past turns in the dialog. The environmental conditions can also be used after receiving the machine action to adjust the response that is provided to the user. For example, the environmental conditions may affect the machine action that is determined as well as how the action is provided to the user. The dialog manager and the response generation components in the conversational dialog system each use the available environmental conditions.

Claims

1. A system comprising:

at least one processor; and

a memory storing instructions that when executed by the at least one processor perform a set of operations comprising;

receiving a user dialog act requesting a computer system to perform a task involving a contact;

accessing environmental conditions that are associated with the contact, wherein the environment conditions include at least one of a noise level, a time zone, or a temperature;

based on the environmental conditions and the user dialog act, determining a machine action to display a suggestion to send a text-based message to the user; and

performing the machine action to display the suggestion to the user.

2. The system of claim 1, wherein the task is calling the contact.

3. The system of claim 1, wherein the machine action is displaying a suggested action other than the task.

4. The system of claim 3, wherein the suggested action is messaging the contact.

5. The system of claim 1, wherein the environmental conditions that are associated with the contact include a noise level.

6. The system of claim 1, wherein determining the machine action is further based on dialog state information, wherein the dialog state information includes at least one of:

previous turn language understanding results, speech recognition results and confidences, knowledge results, previous machine actions, session context, or client context.

7. The system of claim 1, wherein the environmental conditions indicate that the contact is in at least one of the following conditions:

in a roaming service area or in a noisy environment.

8. A computer-implemented method, performed by at least one processor, for using environmental conditions of a contact in a conversational dialog system, comprising:

receiving a user dialog act requesting a computer system to perform a task involving the contact;

accessing environmental conditions that are associated with the contact;

determining from the environmental conditions that the contact is in at least one of the following conditions;

a roaming service area or a time zone where the time is night;

based on the determination from the environmental conditions, determining a machine action; and

performing the machine action to provide a response to the user.

9. The method of claim 8, wherein the task is calling the contact.

10. The method of claim 8, wherein the machine action is displaying a suggested action other than the task.

11. The method of claim 10, wherein the suggested action is messaging the contact.

12. The method of claim 8, wherein the environmental conditions that are associated with the contact include at least one of:

a location;

a noise level;

a velocity;

a temperature;

a current activity;

or a current computing device.

13. The method of claim 8, wherein determining the machine action is further based on dialog state information, wherein the dialog state information includes at least one of:

previous turn language understanding results, speech recognition results and confidences, knowledge results, previous machine actions, session context, or client context.

14. The method of claim 8, wherein the environmental conditions further indicate that the contact is in a noisy environment.

15. A storage device storing computer-executable instructions for using environmental conditions of a contact in a conversational dialog system, the instructions comprising:

receiving, from a user, a user dialog act requesting a computer system to call the contact;

accessing environmental conditions that are associated with the contact;

based on the environmental conditions and the user dialog act, determine a suggested action other than calling the contact;

providing the suggested action to the user, wherein the suggested action is sending a text-based message to the contact;

receiving further input from the user; and

based on the further input, sending a text-based message to the contact.

16. The storage device of claim 15, wherein the environmental conditions that are associated with the contact include at least a noise level.

17. The storage device of claim 15, wherein the environmental conditions that are associated with the contact include at least one of:

a location;

a noise level;

a velocity;

a temperature;

a current activity;

or a current computing device.

18. The storage device of claim 15, wherein the environmental conditions indicate that the contact is in at least one of the following conditions:

in a roaming service area, driving, or in a noisy environment.

19. The storage device of claim 15, wherein determining the suggested action is further based on dialog state information, wherein the dialog state information includes at least one of:

previous turn language understanding results, speech recognition results and confidences, knowledge results, previous machine actions, session context, or client context.

20. The storage device of claim 15, wherein the instructions further comprise:

accessing environmental conditions that are associated with the user; and

based on the environmental conditions that are associated with the user, determining an output type for providing the suggested action to the user, wherein the output type is at least one of audible or visual.