Conversation User Interface

• US 20130283168A1
• Filed: 04/18/2012
• Published: 10/24/2013
• Est. Priority Date: 04/18/2012
• Status: Active Grant

• Current Assignee: Verint Americas Incorporated (Verint Systems Incorporated)
• Original Assignee: Next IT Corporation (Verint Systems Incorporated)
• Inventors: Brown, Fred A., Miller, Tanya M., Wooters, Charles C., Culley, Bryan Michael, Snavely, Eli D.
• Granted Patent: US 9,223,537 B2
• Time in Patent Office: Days
• US Class Current: 715/728
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: H04L 51/04 Real-time or near real-time messaging, e.g. instant messaging [IM]

Abstract

A conversation user interface enables users to better understand their interactions with computing devices, particularly when speech input is involved. The conversation user interface conveys a visual representation of a conversation between the computing device, or virtual assistant thereon, and a user. The conversation user interface presents a series of dialog representations that show input from a user (verbal or otherwise) and responses from the device or virtual assistant. Associated with one or more of the dialog representations are one or more graphical elements to convey assumptions made to interpret the user input and derive an associated response. The conversation user interface enables the user to see the assumptions upon which the response was based, and to optionally change the assumption(s). Upon change of an assumption, the conversation GUI is refreshed to present a modified dialog representation of a new response derived from the altered set of assumptions.

Claims

1. One or more computer-readable media storing computer-executable instructions that, when executed on one or more processors, cause the one or more processors to perform acts comprising:

causing display of a conversation user interface in conjunction with a site of a service provider;

receiving input from a user while the user engages in a session on the site of the service provider, the user input comprising one of audio input, keypad input, or touch input;

representing the user input in the conversation user interface;

determining a response to the user input;

representing the response in the conversation user interface;

enabling the user to interact with the conversation user interface to ascertain how the response was determined and to modify assumptions used to determine the response;

determining a revised response based on the modified assumptions; and

representing the modified response in the conversation user interface.

2. One or more computer-readable media as recited in claim 1, wherein the user input is the audio input, and the user input is represented in the conversation user interface in a text format.

3. One or more computer-readable media as recited in claim 1, wherein the user input is the audio input, and the determining a response comprises a combination of interpreting the audio input and ascertaining a suitable response for the audio input as interpreted.

4. One or more computer-readable media as recited in claim 3, wherein the assumptions used to determine the response comprise first assumptions pertaining to interpretation of the audio input and second assumptions pertaining to ascertainment of the suitable response.

5. One or more computer-readable media as recited in claim 4, further comprising representing the first and second assumptions in association with the response in the conversation user interface.

6. One or more computer-readable media as recited in claim 1, further comprising learning, from past user behavior, what assumptions should be used in making the determination of the response.

7. One or more computer-readable media as recited in claim 1, further comprising identifying a context associated with the user or associated with the session of the user on the site, wherein the assumptions used to determine the response are set, at least in part, by the context.

8. One or more computer-readable media as recited in claim 1, further comprising recording modifications made by the user to the assumptions and learning from the user modifications to adjust how the determination of the response to a future user input is made.

9. One or more computer-readable media as recited in claim 1, wherein the representing the user input and representing the response comprises graphically presenting a series of dialog bubbles that visually distinguish between the user input from the user and the response from a virtual assistant.

10. One or more computer-readable media as recited in claim 1, further comprising enabling the user to interact with the conversation user interface to select a prior dialog bubble and start a new conversation from the prior dialog bubble.

11. One or more computer-readable media as recited in claim 1, further comprising, in response to receiving the user input:

parsing the user input to identify one or more concepts expressed therein;

identifying a context associated with the user or associated with the session of the user on the site;

determining an intent of the user input based at least in part on the one or more concepts and the context; and

determining the response to the user input based at least in part on the intent and the context.

12. One or more computer-readable media as recited in claim 11, wherein the query comprises a string of text, and the identifying of the one or more concepts comprises identifying one or more keywords within the string of text and mapping the one or more keywords to the one or more concepts.

13. One or more computer-readable media as recited in claim 11, wherein the query comprises a string of text, and the identifying of the one or more concepts comprises:

identifying one or more tokens within the string of text;

mapping the one or more tokens to one or more vocab items; and

pattern matching the one or more vocab items to the one or more concepts.

14. One or more computer-readable media as recited in claim 11, wherein the determining of the intent comprises mapping the one or more concepts to one of multiple different intents associated with the one or more concepts based at least in part on the context.

15. One or more computer-readable media as recited in claim 11, wherein the determining of the response comprises mapping the intent to one of multiple different responses associated with the intent based at least in part on the context.

16. One or more computer-readable media as recited in claim 11, wherein the intent is identified with reference to at least a first portion of the context and the response is identified with reference to at least a second, different portion of the context.

17. One or more computer-readable media as recited in claim 11, wherein the context comprises:

whether or not the user has signed in with the site of the service provider;

a status of the user with the service provider;

a page of the site from which the user provides the query to the virtual assistant;

how long the user has remained on the page of the site from which the user provides the query to the virtual assistant;

a navigation history of the user during the session prior to the user providing the query to the virtual assistant;

a location of a cursor when the user provides the query to the virtual assistant;

a prior query provided by the user to the virtual assistant during the session or during a prior session with the site;

a time of day at which the user provides the query to the virtual assistant;

a date on which the user provides the query to the virtual assistant;

an age of the user;

a location of the user;

a device type from which the user accesses the site;

a language of the user;

user preferences;

user buying patterns;

past user behavior;

or

a previous interaction between the user and the virtual assistant.

18. One or more computer-readable media as recited in claim 11, further comprising causing an action on behalf of the user, the action comprising one of:

purchasing an item on behalf of the user;

initiating a request on behalf of the user;

initiating a communication on behalf of the user;

communication to others on behalf of the user;

or

altering a reservation or order on behalf of the user.

19. One or more computer-readable media as recited in claim 11, further comprising learning over time that certain words of the user input map to certain vocabulary terms.

20. One or more computer-readable media as recited in claim 11, further comprising learning over time that certain phrases of the user input map to corresponding concepts.

21. A portable computing device comprising:

one or more computer-readable media as recited in claim 1; and

one or more processors to execute the computer-executable instructions stored on the computer-readable media to perform the acts.

22. A computing system comprising:

one or more servers for hosting the site;

the one or more computer-readable media as recited in claim 1; and

one or more processors to execute the computer-executable instructions stored on the computer-readable media to perform the acts.

23. One or more computer-readable media storing computer-executable instructions that, when executed on one or more processors, cause the one or more processors to perform acts comprising:

causing display of a conversation user interface having first dialog representations associated with input from a user and second dialog representations associated with a response from a virtual assistant;

providing, for display as part of the conversation user interface, one or more graphical elements to convey at least one assumption made to interpret the user input and derive an associated response; and

enabling, via the conversation user interface, the user to change the assumption, whereupon change of the assumption causes an update of the conversation user interface with at least one modified first dialog representation or modified second dialog representation.

24. One or more computer-readable media as recited in claim 23, wherein the assumption comprises a speech recognition assumption that pertains to recognizing verbal input from the user.

25. One or more computer-readable media as recited in claim 23, wherein the assumption comprises a logic assumption that pertains to deriving the response to the user input.

26. One or more computer-readable media as recited in claim 23, wherein the assumption comprises one or more assistant-initiated clarifications that ask the user to clarify the user input.

27. One or more computer-readable media as recited in claim 23, wherein one or more assumptions are associated with each of the first and second dialog representations.

28. One or more computer-readable media as recited in claim 23, wherein the one or more graphical elements that convey associated assumptions are initially hidden in the conversation user interface, the first and second dialog representations being independently selectable via the conversation user interface to reveal the one or more graphical elements in association with a corresponding first or second dialog representation.

29. One or more computer-readable media as recited in claim 23, wherein the user has a user profile, and further comprising choosing the assumption made to interpret the user input based at least in part on the user profile.

30. One or more computer-readable media as recited in claim 23, further comprising choosing the assumption made to interpret the user input based at least in part on the user profile.

31. One or more computer-readable media as recited in claim 23, further comprising choosing the assumption made to interpret the user input based at least in part a context associated with the user.

32. One or more computer-readable media as recited in claim 23, further comprising learning characteristics of the user, and choosing the assumption made to interpret the user input based at least in part these learned tendencies.

33. One or more computer-readable media as recited in claim 32, wherein the characteristics of the user comprises one or more of speech features, preferences, behaviors, patterns, and navigation history.

34. One or more computer-readable media as recited in claim 23, further comprising enabling removal of one or more of the first or second dialog representations from the conversation user interface.

35. One or more computer-readable media as recited in claim 23, further comprising audibly outputting the response.

36. One or more computer-readable media as recited in claim 23, further comprising receiving the user input, and in response to receiving the input:

parsing the user input to identify one or more concepts expressed therein;

identifying a context associated with the user or associated with the session of the user on the site;

determining an intent of the user input based at least in part on the one or more concepts and the context; and

determining the response to the user input based at least in part on the intent and the context.

37. A portable computing device comprising:

one or more computer-readable media as recited in claim 23; and

one or more processors to execute the computer-executable instructions stored on the computer-readable media to perform the acts.

38. One or more computer-readable media storing computer-executable instructions that, when executed on one or more processors, cause the one or more processors to perform acts comprising:

causing display of a conversation user interface that represents a conversation between a user and a virtual assistant;

adding, for display in the conversation user interface, first dialog representations associated with input from the user and second dialog representations associated with responses from the virtual assistant to visually convey the conversation; and

enabling, via interaction with the conversation user interface, the user to select a particular one of the first or second dialog representations within the conversation user interface and restart the conversation from the particular one of the first or second dialog representations.

39. One or more computer-readable media as recited in claim 38, wherein the first and second dialog representations in the conversation user interface after the particular one are removed.

40. A portable computing device comprising:

one or more computer-readable media as recited in claim 38; and

one or more processors to execute the computer-executable instructions stored on the computer-readable media to perform the acts.

41. A computing device, comprising:

a memory;

at least one processor;

a display; and

a conversation application stored in the memory and executed by the processor to facilitate a conversation with a user of the communication device, the conversation application providing a conversation graphical user interface (GUI) to be displayed on the display, the conversation GUI comprising;

an entry portion to receive, from the user, an input in a form of audio or text;

an area to visually convey the conversation, the area to present user dialog representations associated with the input from the user and device dialog representations associated with responses to the user input; and

a control associated with at least one of the user and device dialog representations to visually change, upon actuation, the conversation GUI to reveal how a particular response to a particular user input was determined.

42. The computing device of claim 41, embodied as a portable communication device comprising a communication module to enable communication over a wireless network.

43. A method comprising:

under control of one or more computing systems configured to implement a virtual assistant,

representing user input in the conversation user interface;

representing a response to the user input in the conversation user interface;

enabling the user to interact with the conversation user interface to ascertain how the response was determined and to modify assumptions used to determine the response, wherein modification of one or more assumptions results in a modified response; and

representing the modified response in the conversation user interface.

44. A method of claim 43, wherein the representing the user input and representing the response comprises graphically presenting a series of dialog bubbles that visually distinguish between the user input and the response.

45. A method of claim 43, wherein the enabling comprises providing a control to reveal, upon actuation, how the response to the user input was determined.

46. A method of claim 43, further comprising learning characteristics of the user and choosing the assumptions used to determine the response from the characteristics.

47. One or more computer-readable media as recited in claim 46, wherein the characteristics of the user comprises one or more of speech features, preferences, behaviors, patterns, and navigation history.

48. One or more computer-readable media storing computer-executable instructions that, when executed on one or more processors, cause the one or more processors to serve or render a conversation graphical user interface (GUI), the conversation GUI comprising:

an entry portion to receive, from a user, an input in a form of audio or text;

an area to visually convey the conversation, the area to present user dialog representations associated with the input received from the user and device dialog representations associated with responses to the user input; and

a control associated with at least one of the user and device dialog representations to visually change, upon actuation, the conversation GUI to reveal how a particular response to a particular user input was determined.

49. One or more computer-readable media as recited in claim 48, wherein the particular response was determined based on learned characteristics of the user.

50. One or more computer-readable media as recited in claim 49, wherein the characteristics of the user comprises one or more of speech features, preferences, behaviors, patterns, and navigation history.