Methods and apparatus for automatic training using natural language techniques for analysis of queries presented to a trainee and responses from the trainee

• US 7,249,011 B2
• Filed: 08/12/2002
• Issued: 07/24/2007
• Est. Priority Date: 08/12/2002
• Status: Active Grant

• Current Assignee: Avaya Incorporated
• Original Assignee: Avaya Technology Corporation Miami Lakes FLA US
• Inventors: Chou, Wu, Matula, Valentine C., Erhart, George W., Sinder, Daniel J.
• Primary Examiner(s): HAN, QI
• Application Number:
  US10/217,177
• Publication Number:
  US 20040030541A1
• Time in Patent Office: 1,807 Days
• Field of Search: 704/2, 704/270, 704231-257, 704/258
• US Class Current: 704/2
• CPC Class Codes: G09B 19/00 Teaching not covered by other main groups of this subclass teaching or practice apparatus for gun-aiming or gun-laying F41G3/26
• CPC Class Codes: G09B 7/02 of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student

Abstract

Techniques for training a call center operator trainee are described. A database contains queries including caller speech providing information to be used in routing a call. Each query is associated with an operator response as well as numerical values produced by analyzing the query and the operator response. A training module presents a query to a trainee and receives the trainee response. The training module compares the trainee response to one or more of the numerical values in the database to determine if the trainee response is consistent with the correct destination for the call. If the trainee response is not consistent with the correct destination, the training module provides guidance to assist the trainee in associating the query with the correct destination.

Claims

1. A system for training a call center operator, comprising:

a database containing queries for presentation to a trainee, each query comprising speech typical of that spoken by a caller to a call center, the speech providing information intended to guide an operator in routing the call, each query being further associated with stored destination information indicating the destination specified by the query; and

a training module for selecting a query from the database and presenting it to the trainee, the training module being operative to receive a speech response from the trainee, the training module being further operative to interpret the trainee response to determine whether the trainee response indicates the trainee has selected the destination indicated by the destination information and indicate to the trainee whether or not the trainee response is correct.

2. The system of claim 1 wherein the training module provides guidance to the trainee if the trainee has given an incorrect response, the guidance highlighting elements of the query which indicate the destination.

3. The system of claim 2 further comprising a training file storing results of training sessions and wherein the training module includes a query selector operative to examine the training file to identify elements requiring additional training and to select queries based on elements requiring additional training and other predetermined criteria.

4. The system of claim 3 wherein the training module further comprises a query and response interface for presenting queries to the trainee and receiving trainee responses.

5. The system of claim 4 wherein the database includes with each query a stored operator response, a stored response vector, a query destination tag and a query vector, the stored response vector comprising numerical information produced by analyzing an operator response and the query vector comprising numerical information produced by analyzing the query.

6. The system of claim 5 wherein the database further includes a destination document for each possible destination, the destination document for a destination containing numerical information obtained by processing all calls to the destination.

7. The system of claim 6 wherein the training module includes a response analysis module operative to analyze the trainee response to produce a trainee response vector, the trainee response vector comprising numerical information produced by analyzing the trainee response, the response analysis module being also operative to compare the trainee response vector with the stored response vector to determine if the trainee response vector is consistent with the stored response vector, the response analysis module being operative to identify the trainee response as correct if the trainee response vector is consistent with the stored response vector.

8. The system of claim 7 wherein the response analysis module is also operative to compare the trainee response with the destination documents, choose the destination document most closely matching the trainee response, retrieve the examine the query destination tag to determine if the trainee response is most closely matches the destination document associated with the correct destination and identify the trainee response as correct if the trainee response most closely matches the correct destination document.

9. The system of claim 8 wherein the training module further comprises a trainee guidance module operative to provide guidance to the trainee if the trainee response module identifies the trainee response as incorrect, the trainee guidance module being operative to select a list of keywords appearing in the query and present the keywords to the trainee, the keywords being selected depending on their ability to distinguish the destination indicated by the query from other destinations.

10. The system of claim 9 wherein the trainee guidance module is operative to examine the database for a stored response vector similar to the trainee response vector computed for the trainee response, to retrieve the query associated with the stored response vector, to identify keywords distinguishing the query actually presented from the query associated with the stored response vector which has been retrieved and to present the keywords to the trainee.

11. A method of training a call center operator trainee, comprising:

(a) constructing a database comprising a plurality of entries, each entry comprising speech from a caller to an operator and from an operator to the caller, the speech representing information relevant to a destination to which a call should be routed, the speech from the caller to the operator including a query made by the caller in seeking to inform the operator of the destination to which the call should be routed, the speech from the operator to the caller including a response by the operator informing the caller of the destination to which the operator is routing the call, each entry further comprising numerical values generated by analyzing the speech of the caller and the speech of the operator, the numerical values representing information indicating a destination to which the call should be routed, the numerical values including a query vector produced by analyzing the query and a stored response vector produced by analyzing the operator response as well as a query destination tag identifying the destination indicated by the query, the database further including a destination document for each possible destination to which a call may be routed, the destination document for each destination being produced by analyzing calls routed to that destination;

(b) selecting an entry including a query for presentation to the trainee;

(c) presenting the query to the trainee;

(d) receiving a response from the trainee;

(e) analyzing the response from the trainee to produce a trainee response vector;

(f) comparing the trainee response vector to the stored response vector to determine if the trainee response vector is consistent with the stored response vector;

(g) if the trainee response vector is consistent with the stored response vector, proceeding to step (m);

(h) if the trainee response vector is not consistent with the stored response vector, comparing the trainee response vector with each destination document to determine which destination document most closely matches the trainee response vector;

(i) examining the query destination tag to determine if the trainee response vector matches the correct destination document;

(j) if the trainee response vector matches the correct destination document, proceeding to step (m);

(k) if the trainee response vector does not match the correct destination document, notifying the trainee that his or her response is incorrect;

(l) if training is not complete, returning to step b;

(m) notifying the trainee that his or her response is correct; and

(n) if training is not complete, returning to step b.

12. The method of claim 11, wherein step (k) is followed by a step of providing guidance to the trainee in order to assist him or her in identifying the destination indicated by the query.

13. The method of claim 12, where each of steps (k) and (m) is followed by a step of storing details of the query presented, the trainee response and the correctness or incorrectness of the trainee response in a training file.

14. The method of claim 13, wherein the step of selecting an entry includes examining the training file to identify areas needing further training and wherein criteria used to select an entry include consideration of which areas need further training.

15. The method of claim 14, where entries stored in the training file during a training session with a particular trainee are associated with that trainee.

16. The method of claim 15, wherein the step of providing guidance to the trainee includes presenting key terms included in the query.

17. The method of claim 15, wherein the step of providing guidance to the trainee includes selecting an entry associated with the destination indicated by the trainee'"'"'s response, identifying key terms which differ between the selected query and the query actually presented to the trainee and presenting the differing key terms to the trainee.