Method and apparatus for managing user conversations

• US 7,487,095 B2
• Filed: 09/02/2005
• Issued: 02/03/2009
• Est. Priority Date: 02/11/2003
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Zieman, Yuri, Hill, Jeffrey
• Primary Examiner(s): MCFADDEN, SUSAN IRIS
• Application Number:
  US11/218,925
• Publication Number:
  US 20060080107A1
• Time in Patent Office: 1,250 Days
• Field of Search: 704/275
• US Class Current: 704/275
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

In one aspect, an arbitrary natural language communication is received from a user. A concept recognition process is applied to automatically derive a representation of concepts embodied in the communication. The concept representation is used to provide to a human agent information useful in responding to the natural language communication.

Claims

1. A method comprising:

receiving an arbitrary natural language communication from a user;

applying a concept recognition process to automatically derive a representation of concepts embodied in the communication;

using the concept representation to provide to a human agent information useful in responding to the natural language communication, wherein the information provided to the human agent includes a plurality of possible responses to the user'"'"'s communication;

enabling the human agent to select a response from the plurality of possible responses; and

delivering the selected response to the user.

2. The method of claim 1 in which the arbitrary natural language communication is expressed in speech.

3. The method of claim 2 in which the communication is expressed using a telephone or other voice instrument.

4. The method of claim 1 in which the communication is a method stored in a voice mailbox.

5. The method of claim 1 in which the arbitrary natural language communication is expressed in text.

6. The method of claim 5 in which the text is expressed electronically.

7. The method of claim 6 in which the text is expressed in an email.

8. The method of claim 7 in which the text is expressed through instant messaging.

9. The method of claim 5 in which the text is expressed in a manner associated with a web page.

10. The method of claim 1 in which the concept recognition process is universally applicable to any communication in a natural language.

11. The method of claim 1 in which the concept representation is expressed in a mark-up language.

12. The method of claim 1 in which the information provided to the human agent includes an audible playback of a recorded version of the natural language communication.

13. The method of claim 12 in which the playback is compressed in time relative to the communication.

14. The method of claim 1 in which the information provided to the human agent includes a display of a text corresponding to the communication.

15. The method of claim 1 in which the information provided to the human agent includes information about at least one prior communication or response that preceded the natural language communication.

16. The method of claim 15 in which the concept recognition process is used to determine how much information about prior communications to provide to the human agent.

17. The method of claim 1 in which the communication is part of a dialog between the user and a response system, the dialog including communications from the user and responses to the user, and the information provided to the human agent includes information about historical portions of the dialog.

18. The method of claim 17 in which a first mode of expression of the communications from the user is different from a second mode of expression of the responses to the user.

19. The method of claim 18 in which the first mode and second mode of expression comprise at least one of text or speech.

20. The method of claim 1 in which a first mode of expression of the communications from the user is different from a second mode of expression of the responses to the user.

21. The method of claim 1 in which the first mode and second mode of expression comprise at least one of text or speech.

22. The method of claim 1 in which the information about possible responses includes a text of a possible response.

23. The method of claim 1 in which the information about possible responses includes an indication of a level of confidence in the appropriateness of the response.

24. The method of claim 1 in which the communication comprises a question and the response comprises an answer to the question.

25. The method of claim 1 in which the communication comprises a question and the response comprises a request for additional information.

26. The method of claim 1 also including enabling the human agent to determine how the information useful in responding to the communication is selected.

27. The method of claim 26 in which the enabling of the human agent includes permitting the agent to use the communication from the user to control how the responsive information is selected.

28. The method of claim 26 in which the enabling of the human agent includes permitting the agent to enter a substitute communication to control how the responsive information is selected.

29. The method of claim 28 in which the substitute communication is a restatement by the human agent of the communication from the user.

30. The method of claim 1 in which the useful responding information is generated by applying the concept representation to a body of information representing other communications and their relationships to concepts.

31. The method of claim 30 in which applying the concept representation includes a matching process to determine a cluster of similar communications to which the user'"'"'s communication likely belongs.

32. The method of claim 1 in which a state is occupied prior to receipt of the communication, and also including selecting a transition to a next state based on the concept representation and on a set of possible transitions.

33. The method of claim 32 in which the transition includes an action to be taken in response to the communication.

34. The method of claim 33 in which the action to be taken comprises a reply communication.

35. The method of claim 33 in which the set of possible transitions is derived from examples of state-transition-state or stimulus-response sequences.

36. The method of claim 35 in which the examples include pre-run-time examples.

37. The method of claim 36 in which the pre-run-time examples comprise voice or text.

38. The method of claim 35 in which the examples occur at runtime.

39. The method of claim 1 also including providing a response to the communication from the user.

40. The method of claim 1 in which the response delivered to the user automatically without the user knowing that it was a human agent who selected the response.

41. The method of claim 40 in which the selected response is generated by the human agent.

42. The method of claim 41 in which the selected response is spoken or typed by the human agent.

43. The method of claim 1 also including providing a graphical user interface for a workstation of the human agent, the information useful in responding being presented in the interface, the interface being presented as part of a user interface of a third party'"'"'s response system software.

44. The method of claim 43 in which the user interface provides conceptual context for a communication from a user.

45. The method of claim 1, wherein the selected response is delivered to the user in real time relative to the communication.

46. The method of claim 1, wherein the selected response is delivered to the user at a later time relative to the communication.

47. The method of claim 46 in which the communication is provided in speech and the response is provided in text.

48. The method of claim 1 also including selecting the human agent to handle the response to the communication.

49. The method of claim 48 in which the human agent is automatically selected by a work distribution process.

50. The method of claim 49 in which the work distribution process uses information deduced from the concept representation in automatically selecting the human agent.

51. A method comprising:

receiving an arbitrary natural language communication from a user;

automatically deriving a representation of concepts embodied in the communication; and

using the concept representation, automatically providing a response to the communication to the user in a different mode of expression than the mode of expression used for the communication, wherein the response is selected by a human agent from a plurality of possible responses to the communication received from the user.

52. The method of claim 51 in which the response is provided in a manner that prevents the user from knowing that the response was selected by the human agent.

53. The method of claim 51 in which the communication is provided in speech and the response is provided in text.

54. A method comprising:

initiating a dialog with a user by sending a first natural language communication to the user;

in response to the first natural language communication to the user, receiving a second natural language communication from the user;

applying a concept recognition process to automatically derive a representation of concepts embodied in the second communication;

using the concept representation to provide to a human agent information useful in responding to the second communication;

enabling the human agent to select a response to the second communication; and

delivering the selected response to the user while preventing the user from knowing that the response was selected by the human agent.

55. A method comprising:

receiving a set of recordings or transcripts of dialogs between users and human agents;

recognizing the speech in the recordings, separating each of the dialogs into communications each of which is made by either a user or a human agent;

applying a concept recognition process to derive a representation of concepts embodied in each of the communications;

automatically creating a body of state-transition-state or stimulus-response information from the concept representations that enables automated determination of appropriate responses to natural language communications received from users;

enabling a first human agent to select a response that is one of the appropriate responses; and

delivering the selected response to a first user while preventing the first user from knowing that the response was selected by the first human agent.

56. A method comprising:

receiving example dialogs each comprising a sequence of natural language communications between two parties;

applying a concept recognition process to automatically derive a representation of concepts embodied in each of the communications;

using the sequences of communications to form a body of state-transition-state or stimulus-response information that enables a determination of an appropriate transition for any arbitrary communication that is received when in a particular one of the states;

enabling a human agent to select a response after the appropriate transition is determined; and

delivering the selected response to user.

57. The method of claim 56 also including using the concept representations to form clusters of communications that are related in the concepts that are embodied in them.

58. The method of claim 56 in which the selected response is delivered in a manner that prevents the user from knowing that the response was selected by the human agent.

59. The method of claim 56 in which the dialogs relate to contact center operation.

60. The method of claim 56 in which the dialogs comprise requests and responses to the requests.

61. The method of claim 56 in which the dialogs comprise real-time dialogs.

62. The method of claim 61 in which the dialogs comprise a string of voice messages.

63. The method of claim 56 in which the representations of concepts are expressed in a mark-up language.

64. The method of claim 57 in which the communications in the cluster comprise communications that represent different ways of expressing similar sets of concepts.

65. A method comprising:

receiving an arbitrary natural language communication from a user;

determining a degree of involvement of a human agent in responding to the communication from the user; and

performing at least one mode of a plurality of conversation management modes based on the determined degree involvement, wherein;

a first mode of the plurality of conversation management modes comprises producing a response to the user in an automated manner without interaction by the human agent;

a second mode of the plurality of conversation management modes comprises;

enabling the human agent to select a response from a plurality of possible responses to the communication from the user; and

delivering the selected response to the user; and

a third mode of the plurality of conversation management modes comprises initiating direct communication between the human agent and the user.

66. A The method of claim 65, wherein the response selected by the human agent is delivered to the user in a manner that prevents the user from knowing that the response was selected by the human agent.

67. The method of claim 66 in which the determining of the degree of involvement is based in part on an estimate of how long it would take the human agent to respond if the communication is referred to the human agent for response.

68. The method of claim 66 in which the at least one performed conversation management mode is selected based on a level of confidence in the appropriateness of an automatically generated response.

69. The method of claim 66 in which the at least one performed conversation management mode is selected based on business rules.

70. The method of claim 66, wherein the first mode is the at least one performed mode, and wherein the method further comprises switching between the first mode and the second mode.

71. The method of claim 66, wherein the second mode is the at least one performed mode, and wherein the method further comprises switching between the second mode and the third mode.

72. The method of claim 71 in which the response is entered as text or spoken.

73. The method of claim 65 in which the selected response is delivered automatically to the user.

74. A method comprising:

enabling a user to access a contact service facility;

receiving communications from the user at the contact service facility;

providing responses to the user'"'"'s communications; and

causing at least one of the responses to be selected by a human agent based on the results of an automated concept matching process applied to the communications, the user being unaware that the human agent selected the response.

75. The method of claim 74 in which a first mode of expression of the communications from the user is different from a second mode of expression of the responses to the user.

76. The method of claim 75 in which the first mode and second mode of expression comprise at least one of text or speech.

77. A method comprising:

maintaining a body of state-transition-state or stimulus-response information that represents possible sequences of natural language communications between a user and a response system, the information being generated automatically from historical sequences of communications; and

using selected ones of the sequences of communications to manage human agents who provide responses to user communications, wherein at least a portion of the provided responses include automatically generated responses that are selected by the human agents and delivered to the users.

78. The method of claim 77 in which the selected ones are used to train the human agents.

79. The method of claim 77 in which the selected ones are used to evaluate the human agents.

80. The method of claim 77 in which the sequences are used to manage the human agents by providing the agents with communications that are part of the sequences and evaluating responses of the human agents against known appropriate responses.

81. A The method of claim 77, wherein selected responses are delivered while preventing the users from knowing that the responses were selected by the human agents.

82. The method of claim 81 in which the body of information is used in connection with testing of the response system.

83. The method of claim 81 in which the body of information is used in connection with software processes used in the response system.

84. A method comprising:

maintaining a body of state-transition-state or stimulus-response information that enables automated determination of appropriate responses to natural language communications received from users;

receiving other natural language communications from users for which appropriate responses cannot be determined;

tracking actions taken by a human agent in connection with responding to the other natural language communications, wherein the actions taken by the human agent include selecting responses for use in responding to the other natural language communications, wherein the selected responses are delivered to the users; and

automatically inferring from the other natural language communications and the selected responses, information for inclusion in the body of state-transition-state or stimulus-response information.

85. The method of claim 84 in which the selected responses are delivered to the users while preventing the users from knowing that the responses were selected by the human agent.

86. The method of claim 85 also including enabling an administrator to review the inferred information prior to including it in the body of state-transition-state or stimulus-response information.

87. The method of claim 84 in which the actions taken by the human agent include keystrokes or mouse actions.

88. The method of claim 84 also including providing the human agent with possible responses to the natural language communications, and in which the tracking of actions includes tracking which of the possible responses the human agent chooses and inferring that the chosen response is a correct response to one of the communications.

89. The method of claim 84 also including providing the human agent with possible responses to the natural language communications, and, if the human agent responds to the communication without choosing one of the possible responses, inferring that the possible responses are incorrect.

90. The method of claim 89 also including enabling the human user to indicate that one of the possible answers was correct, even though the human user is respond to the communication without making a choice among the possible responses.

91. A method comprising:

maintaining a body of state-transition-state or stimulus-response information that enables automated determination of appropriate responses to natural language communications received from users, the state-transition-state or stimulus-response information being associated with a contact center of an enterprise;

updating the body of information based on communications received from users and responses selected by human agents of the contact center, wherein the responses selected by human agents are delivered to the user while preventing the users from knowing that the responses were provided by the human agents; and

analyzing the body of information to infer knowledge about the operation of the enterprise.

92. A method comprising:

maintaining a body of state-transition-state or stimulus-response information that enables automated determination of appropriate responses to natural language communications received from users, the state-transition-state or stimulus-response information being based on concept representations derived from example natural language communications;

the example natural language communications being predominantly in one language;

using the state-transition-state or stimulus-response information to provide appropriate responses to natural language communications received from users in a second language different from the one language;

enabling a human agent to select a response from the provided appropriate responses; and

delivering the selected response to a user.

93. A method comprising:

displaying to a human agent a user interface containing concept representation-based information useful in responding to natural language communications from users;

the information including automatically generated possible natural language responses and indications of relative confidence levels associated with the responses;

enabling the human agent to select a response from the automatically generated possible natural language responses; and

delivering the selected response to a user.

94. The method of claim 93 also including preventing the user from knowing that the response was selected by the human agent.

95. The method of claim 93 also including enabling the human agent to enter a substitute of the user'"'"'s communication, and generating the possible natural language responses from the substitute communication.

96. The method of claim 93 also including providing controls in the interface that enable the human agent to choose a level of response with respect to the degree of involvement of the human agent.

97. The method of claim 96 in which the level of response includes direct conversation with the user.

98. The method of claim 96 in which the level of response includes providing the response automatically.

99. A method comprising:

maintaining a body of state-transition-state or stimulus-response information that enables automated determination of appropriate responses to natural language communications received from users, the state-transition-state or stimulus-response information being based on concept representations derived from example natural language communications, each of the states having possibly multiple transitions leading to a later state;

when in a predetermined one of the states, using information about the multiple transitions to improve the accuracy of recognition of a speech recognizer that is processing a spoken communication from a user;

generating a plurality of possible responses based at least in part on the spoken communication from the user;

enabling a human agent to select a response from the automatically generated possible natural language responses; and

delivering the selected response to the user.

100. The method of claim 99 also including using the information about multiple transitions to improve the accuracy of discriminate matching of the concept representation of the spoken communication with clusters of concept representations in the body of information.

101. A method comprising:

receiving natural language communications from users;

automatically considering possible responses to the communications and confidence levels with respect to the responses, providing automated responses to a first portion of the users based on the confidence levels;

refraining from providing automated responses to a second portion of the users;

enabling the human agent to select responses from the possible responses; and

delivering the selected response to the second portion of the users while preventing the second portion of the users from knowing that the response was selected by the human agent.

102. A method comprising:

receiving natural language communications from users;

automatically recognizing concepts contained in the communications;

distributing the communications to human agents for responding to the users, the distribution being based on the concepts recognized in the communications;

enabling at least a portion of the human agents to select responses based on the distribution; and

delivering the selected responses to the users while preventing the users from knowing that the responses were selected by the portion of the human agents.

103. An apparatus comprising:

a user interface for a human agent at a contact service facility, the user interface including;

a window containing information provided by a contact service process, the information including;

information about a user of the facility; and

window elements embedded in the window provided by the contact service process, the elements including a list of possible natural language responses based on concept representations for an active communication of a user, and indications of relative confidence that the respective responses are appropriate for the communication of the user; and

a conversation management mode configured to enable the human agent to select a response from the list of possible natural language responses, and to deliver the selected response to the user.

104. The apparatus of claim 103 in which the window elements include a place for a human agent to view text corresponding to the communication of the user, and a place for the human-agent to enter a substitute text for the communication of the user.

105. The apparatus of claim 103, wherein the conversation management mode is further configured to prevent the user from knowing that the response was selected by the human agent.

106. The apparatus of claim 103, wherein the conversation management mode is a first conversation management mode, and the apparatus further comprises a second conversation management mode configured to provide a response to the user in an automated manner.

107. The apparatus of claim 106, wherein the apparatus is configured to switch between the first conversation management mode and the second conversation management mode.

108. The apparatus of claim 106, wherein apparatus further comprises a third conversation management mode configured to enable direct conversation management between the human agent and the user.

109. The apparatus of claim 108, wherein the apparatus is configured to switch between the first conversation management mode, the second conversation management mode, and the third conversation management mode.