CALL STEERING DATA TAGGING INTERFACE WITH AUTOMATIC SEMANTIC CLUSTERING

• US 20150081290A1
• Filed: 11/26/2014
• Published: 03/19/2015
• Est. Priority Date: 09/27/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Tremblay, Real, Tremblay, Jerome, Ulug, Amy E., Fortier, Jean-Francois, Berleur, Francois, Marcus, Jeffrey N., Mauro, David Andrew
• Granted Patent: US 9,251,785 B2
• Time in Patent Office: Days
• US Class Current: 704/231
• CPC Class Codes: G06F 3/00 Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 2015/0638 Interactive procedures
• CPC Class Codes: H04M 11/00 Telephonic communication systems specially adapted for combination with other electrical systems
• CPC Class Codes: H04M 3/493 Interactive information services, e.g. directory enquiries ; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals

Abstract

A system and method for providing an easy-to-use interface for verifying semantic tags in a steering application in order to generate a natural language grammar. The method includes obtaining user responses to open-ended steering questions, automatically grouping the user responses into groups based on their semantic meaning, and automatically assigning preliminary semantic tags to each of the groups. The user interface enables the user to validate the content of the groups to ensure that all responses within a group have the same semantic meaning and to add or edit semantic tags associated with the groups. The system and method may be applied to interactive voice response (IVR) systems, as well as customer service systems that can communicate with a user via a text or written interface.

Claims

1. A method for enabling a developer of a steering application to associate semantic tags with user responses, the method comprising:

obtaining user responses to an open-ended steering question posed by an interactive response system;

automatically grouping the user responses into groups, wherein each group is a set of sentences that are semantically related;

automatically assigning preliminary semantic tags to each of the groups; and

providing a user interface that enables a user to validate the content of the groups to ensure that all sentences within a group have the same semantic meaning and to view and edit the preliminary semantic tags associated with the groups, wherein the interface includes;

a groups view that displays a list of the groups and corresponding semantic tags for each group, wherein the groups view enables a user to edit the preliminary semantic tags associated with each of the groups,

a sentence view that displays, for a selected group in the groups view, a list of unique sentences associated with the selected group, wherein in the sentence view a user is able to verify whether or not a sentence belongs to the group selected in the groups view, and

a related-groups view that displays, for a selected group in the groups view or a selected sentence in the sentence view, a plurality of groups most closely-related to the selected group or sentence.

2. The method of claim 1, wherein the user responses are spoken utterances.

3. The method of claim 2, wherein, prior to grouping the user responses into groups, transcriptions are obtained for the spoken utterances.

4. The method of claim 1, wherein the user responses are in the form of written or textual inputs.

5. The method of claim 4, wherein the written or textual inputs include one or more of text messaging, emails, online chat system communications, and Internet or web-based communications.

6. The method of claim 1, wherein the semantic tags comprise a value for an action and a value for an object.

7. The method of claim 1, wherein, from the sentence view, a user is able to move a displayed sentence to a different group.

8. The method of claim 1, wherein, in the related groups view, groups are displayed in a star diagram with the selected group or sentence in the middle of the star diagram and the plurality of most closely-related groups surrounding the selected group or sentence.

9. The method of claim 1, wherein, in the related groups view, the plurality of most closely-related groups are displayed in a list in order of semantic distance from the selected group or sentence.

10. The method of claim 1, wherein a sentence in the sentence view can be moved to a different group by dragging the sentence to one of the plurality of most closely-related groups in the related-groups view.

11. The method of claim 1, wherein a sentence in the sentence view can be moved to a different group by dragging the sentence to one of the plurality of groups in the groups view.

12. The method of claim 1, wherein the user interface enables the user to merge groups.

13. The method of claim 1, wherein the user interface enables the user to create a new group.

14. A non-transitory, computer-readable medium comprising a computer program that, when executed by a computer system, enables the computer system to perform a method for enabling a developer of a steering application to associate semantic tags with user responses, the method comprising:

obtaining user responses to an open-ended steering question posed by an interactive response system;

selecting a subset of user responses;

automatically grouping the user responses within the subset into groups, wherein each group is a set of sentences that are semantically related;

automatically assigning preliminary semantic tags to each of the groups;

providing a user interface that enables a user to validate the content of the groups to ensure that all sentences within a group have the same semantic meaning and to view and edit the preliminary semantic tags associated with the groups, wherein the interface includes;

a groups view that displays a list of the groups and corresponding semantic tags for each group, wherein the groups view enables a user to edit the preliminary semantic tags associated with each of the groups,

a sentence view that displays, for a selected group in the groups view, a list of unique sentences associated with the selected group, wherein in the sentence view a user is able to verify whether or not a sentence belongs to the group selected in the groups view, and

a related-groups view that displays, for a selected group in the groups view or a selected sentence in the sentence view, a plurality of groups most closely-related to the selected group or sentence; and

iteratively repeating the selecting, grouping, assigning, and providing steps with different subsets of user responses until all user responses have been processed, wherein each iteration uses data from previously validated and tagged groups to increase the accuracy of the clustering and assigning steps.

15. The non-transitory, computer-readable medium of claim 14, wherein the subsets are selected and processed such that the most common user responses are processed first.

16. The non-transitory, computer-readable medium of claim 14, wherein the subsets are selected and processed such that user responses with key words or phrases are processed first.

17. The non-transitory, computer-readable medium of claim 14, wherein the user responses are spoken utterances and, prior to the selecting step, transcriptions are obtained for the utterances.

18. The non-transitory, computer-readable medium of claim 14, wherein the user responses are written responses.

19. The non-transitory, computer-readable medium of claim 14, wherein the selecting, grouping, assigning, and providing steps are repeated in response to the user pressing a button to initiate semantic clustering for a next subset of responses.

20. A non-transitory, computer-readable medium comprising a computer program that, when executed by a computer system, enables the computer system to perform a method for enabling a developer of a steering application to associate semantic tags with user responses, the method comprising:

obtaining user responses to an open-ended steering question;

automatically grouping the user responses into groups, wherein each group is a set of sentences that are semantically related;

automatically assigning preliminary semantic tags to each of the groups; and

providing a user interface that enables a user to validate the content of the groups to ensure that all sentences within a group have the same semantic meaning and to view and edit the preliminary semantic tags associated with the groups, wherein the interface includes;

a groups view that displays a list of the groups and corresponding semantic tags for each group, wherein the groups view enables a user to edit the preliminary semantic tags associated with each of the groups,

a sentence view that displays, for a selected group in the groups view, a list of unique sentences associated with the selected group, wherein in the sentence view a user is able to verify whether or not a sentence belongs to the group selected in the groups view, and

a related-groups view that displays, for a selected group in the groups view or a selected sentence in the sentence view, a plurality of groups most closely-related to the selected group or sentence.