Topic discriminator using posterior probability or confidence scores

• US 5,625,748 A
• Filed: 04/18/1994
• Issued: 04/29/1997
• Est. Priority Date: 04/18/1994
• Status: Expired due to Term

• Current Assignee: Ramp Holdings Incorporated (Clean Harbors Incorporated)
• Original Assignee: BBN Corporation (Verizon Communications Inc.)
• Inventors: Ng, Kenney, Gish, Herbert, Rohlicek, Jan R., McDonough, John W., Jeanrenaud, Phillippe
• Primary Examiner(s): MacDonald, Allen R.
• Assistant Examiner(s): CHOWDHURY, INDRINAL
• Application Number:
  US08/228,755
• Time in Patent Office: 1,107 Days
• Field of Search: 395/2.45, 395/2.47, 395/2.48, 395/2.49, 395/2.6
• US Class Current: 704/251
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/10 using distance or distortion measures between unknown speech and reference templates
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding

Abstract

An improved topic discriminator includes an integrated speech recognizer or word and phrase spotter as part of a speech event detector, and a topic classifier trained on topic-dependent event frequencies. The event frequencies are determined from either or both transcribed data with a text event detector and untranscribed speech data with a speech event detector. In accordance with another aspect of the present invention, the phrase spotter is used to detect the presence of phrases without the need of parsing the output of a speech recognizer'"'"'s hypothesized transcription. In accordance with another aspect of the invention, an improved technique is used to select a subset of the potential speech events on whose event frequencies the topic discrimination decision is made. Finally, in accordance with another aspect of the invention, an improved method of topic modeling is used to improve the performance of the topic discriminator.

Claims

1. A method of selecting a vocabulary of important keywords or speech events, the presence or absence of which in a hypothesis string associated with a speech message serves to indicate the topic membership of the message, said method comprising the steps of:

generating hypothesis strings in response to and as a function of a plurality of training messages produced from the output of an event detector, each of said training messages being known to be about one of a finite set of topics;

deriving word or event frequency statistics from said hypothesis strings;

assigning topic-conditional significance or association scores to all possible words or events as a function of the word or event frequency statistics; and

selecting those words or events having topic-conditional significance or association scores above a predetermined threshold.

2. The method according to claim 1, wherein said step of generating said hypothesis strings in response to and as a function of the plurality of training messages produced from the output of the event detector includes the step of generating said hypothesis strings so that each indicates only the binary-valued presence or absence of a word or event.

3. The method according to claim 2, wherein said step of generating said hypothesis strings in response to and as a function of the plurality of training messages includes the step of deriving said hypothesis strings from a text transcription of the speech message produced from the output of the event detector.

4. The method according to claim 2, wherein said step of generating said hypothesis strings in response to and as a function of the plurality of training messages includes the steps of (a) obtaining said hypothesis strings by applying the speech messages to the input of the event detector of the type providing a hypothesis string output in response to a speech message;

and (b) detecting the output of the event detector.

5. The method according to claim 4, wherein the event detector is a speech recognizer and said step of detecting the output of the event detector includes the step of detecting the output of said speech recognizer.

6. The method according to claim 1, wherein said step of generating hypothesis strings in response to and as a function of the plurality of training messages includes the step of generating each hypothesis string so that said hypothesis string includes a probability or confidence score for every word or event in the string, said score indicating the overall likelihood the word or event occurred in the actual speech message associated with the string.

7. The method according to claim 6, wherein said step of generating hypothesis strings in response to and as a function of the plurality of training messages includes the steps of (a) generating said hypothesis strings by applying the speech messages to the input of an event detector of the type providing hypothesis strings and associated word or events scores in response to speech messages;

and (b) detecting the output of the event detector.

8. The method according to claim 1, wherein the event detector is a word or event spotter and said step of detecting the output of the event detector includes the step of detecting the output of said word or event spotter.

9. The method according to claim 1, wherein said step of deriving word or event frequency statistics from said hypothesis strings includes the step of counting the number of keyword or event occurrences in the hypothesis strings.

10. The method according to claim 1, wherein said step of deriving word or event frequency statistics from said hypothesis strings includes the step of obtaining said word-frequency statistics by summing the confidence scores for all words or events in the hypothesis strings.

11. The method according to claim 1, wherein said step of assigning topic-conditional significance or association scores to all possible words or events includes the step of obtaining topic-conditional significance or association scores by a hypothesis test.

12. The method according to claim 11, wherein said hypothesis test is a chi-squared test, and the step of obtaining topic-conditional significance or association scores by said chi-squared test includes the step of calculating the chi-squared (X²) values as follows:

##EQU16## and storing said chi-squared values in a table having cells defined by i rows and j columns;

wherein ##EQU17## is the expected number of counts for each cell;

##EQU18## N_ij equals the number of conversations assigned to cell (i,j) found for each cell in the table.

13. The method according to claim 12, wherein said step of obtaining topic-conditional significance or association scores by a chi-squared test includes the step of generating at least one contingency table from the word-frequency statistics derived from the message hypothesis strings and basing said chi-squared test on said contingency tables.

14. The method according to claim 13, wherein each topic is allotted its own row in the contingency table and said step of obtaining topic-conditional significance or association scores by a chi-squared test includes the step of conducting a hypothesis test across all topics simultaneously.

15. The method according to claim 13, wherein a single topic is allotted its own row in the contingency table while all other topics are grouped together in a second row of said table, and said step of obtaining topic-conditional significance or association scores by a chi-squared test includes the step conducting said chi-squared test for only a single word and topic at a time.

16. The method according to claim 1, wherein said step of deriving word or event frequency statistics from said hypothesis strings includes the steps of (a) using said word or event frequency statistics to construct topic-conditional mixture densities for each possible word or event, and (b) using said topic-conditional mixture densities to calculate an association score for each word.

17. The method according to claim 16, wherein the mixture have the form of:

##EQU19## wherein Tⁱ is one of i topics;

q_im is the prior probability of the selection of mixture m conditioned on topic T_i ;

f_im (θ

) is the conditional multi-variate probability density function of θ

; and

M_i is the total number of mixtures associated with topic T_i.

18. The method according to claim 17, wherein said mixture densities are comprised of the weighted sum of delta-dirac functions.

19. The method according to claim 17, wherein said mixture densities are comprised of the weighted sum of beta-density functions of the form:

##EQU20## wherein C(n_im.sup.(0),n_im.sup.(1), . . . ,n_im.sup.(k))is a normalization constant chosen to ensure f_im (θ

) is a proper density function.

20. The method according to claim 19, wherein further including the steps of estimating the parameters of said mixture densities with an Estimation-Maximization algorithm.

21. The method according to claim 19, wherein said association score is a mutual information measure occurrences and that of topic membership, wherein the desired mutual information measure I(T,W) can be expressed as:

##EQU21## where p(w_k .sup.(n) |T.sup.(n)) is the likelihood of the occurrence pattern for word or event v_k observed in training sequence w.sup.(n) conditioned on its true topic T.sup.(n), and ##EQU22## is the likelihood for the same occurrence pattern independent of topic.

22. The method according to claim 19, wherein said association score is a Kullback-Liebler distance between the probability density of the word conditioned on a given topic, and the density of the word conditioned on all other topics, said distance measure having the form:

##EQU23## where N(T) is the total number of conversations or messages relating to the topic of interest, and N(T) is the number relating to all other topics.