System and method for searching, analyzing and displaying text transcripts of speech after imperfect speech recognition

• US 6,973,428 B2
• Filed: 05/24/2001
• Issued: 12/06/2005
• Est. Priority Date: 05/24/2001
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Boguraev, Branimir Konstantinov, Cooper, James W.
• Primary Examiner(s): Young, W. R.
• Assistant Examiner(s): Opsasnick, Michael N.
• Application Number:
  US09/864,982
• Publication Number:
  US 20020178002A1
• Time in Patent Office: 1,657 Days
• Field of Search: 704/231, 704/251, 704/276, 704/277
• US Class Current: 704/251
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A speech conversation is changed to a text transcript, which is then pre-processed and subjected to text mining to determine salient terms. Salient terms are those terms that meet a predetermined level of selectivity in a collection. The text transcript of the speech conversation is displayed by emphasizing the salient terms and minimizing non-salient terms. An interface is provided that allows a user to select a salient term, whereupon the speech conversation is played beginning at the location, in the speech file, of the selected salient term.

Claims

1. A method for processing text transcripts of speech after imperfect speech recognition, the method comprising the steps of:

converting a speech document to text;

processing the text to determine salient terms; and

displaying the text by emphasizing the salient terms and minimizing non-salient terms, wherein said salient terms are determined by;

determining high selectivity terms in the at least one text transcript;

determining how many high selectivity terms there are;

determining how many multiword and high selectivity terms there are that are above a first predetermined selectivity;

when there are ten or more high selectivity and multiword terms that have confidences greater than a first predetermined selectivity, displaying the ten or more high selectivity and multiword terms; and

when there are less than ten high selectivity and multiword terms that are above a first predetermined selectivity;

determining how many single word and high selectivity terms there are;

determining if there are ten or more single word and multiword terms that are high selectivity terms that have selectivities greater than a first predetermined selectivity;

when there are ten or more single word and multiword terms that are high selectivity terms that have selectivity greater than a first predetermined selectivity, displaying all of the ten or more single word and multiword terms; and

when there are not ten or more single word and multiword terms that are high selectivity terms that have confidences greater than a first predetermined selectivity, displaying all single word and multiword terms that are high selectivity terms and that have confidences greater than a second predetermined selectivity.

2. The method of claim 1, wherein the step of processing the text to determine salient terms further comprises the steps of:

increasing sentence and paragraph structure in the text;

removing non-word utterances from the text;

determining high selectivity terms in the text; and

selecting terms from the high selectivity terms that are above a predetermined selectivity.

3. The method of claim 1, further comprising the steps of:

selecting a salient term;

determining a time of the salient term relative to a beginning time of the speech document; and

playing the speech document from the time of the salient term.

4. The method of claim 1, further comprising the step of creating the speech document by recording a conversation, and wherein the step of converting comprises performing speech recognition on the speech document, wherein any speakers of the speech have not trained the speech recognition system with their own speech.

5. A method for processing text transcripts of speech after imperfect speech recognition, the method comprising the steps of:

converting a speech document to at least one text transcript;

increasing sentence and paragraph structure in the at least one text transcript;

removing non-word utterances from the at least one text transcript;

determining salient terms in the at least one text transcript; and

displaying text of the at least one text transcript, the step of displaying performed to emphasize display of the salient terms relative to non-salient terms, wherein said salient terms are determined by;

determining high selectivity terms in the at least one text transcript;

determining how many high selectivity terms there are;

determining how many multiword and high selectivity terms there are that are above a first predetermined selectivity;

when there are ten or more high selectivity and multiword terms that have confidences greater than a first predetermined selectivity, displaying the ten or more high selectivity and multiword terms; and

when there are less than ten high selectivity and multiword terms that are above a first predetermined selectivity;

determining how many single word and high selectivity terms there are;

determining if there are ten or more single word and multiword terms that are high selectivity terms that have selectivities greater than a first predetermined selectivity;

when there are ten or more single word and multiword terms that are high selectivity terms that have selectivity greater than a first predetermined selectivity, displaying all of the ten or more single word and multiword terms; and

when there are not ten or more single word and multiword terms that are high selectivity terms that have confidences greater than a first predetermined selectivity, displaying all single word and multiword terms that are high selectivity terms and that have confidences greater than a second predetermined selectivity.

6. The method of claim 5, wherein the step of displaying text further comprises the steps of:

determining a position of a cursor when a click occurs;

determining if the position is within a predetermined distance from one of a plurality of salient terms that are being displayed;

when the position is within the predetermined distance from one of the salient terms, performing the following steps;

determining which of the salient terms is near the position;

determining a time of the salient term; and

playing the speech document, beginning at the time.

7. The method of claim 5, wherein the steps of increasing and removing are performed by accessing the at least one transcript and creating processed text comprising the at least one transcript with the increased sentence and paragraph structure and with fewer non-word utterances.

8. The method of claim 5, wherein the step of displaying text in the at least one text transcript further comprises the step of making font size of the non-salient terms at least 10 points smaller than font size of the salient terms.

9. The method of claim 5, wherein the step of displaying text in the at least one text transcript further comprises the step of displaying the non-salient terms with non-text symbols.

10. The method of claim 5, wherein the step of displaying text in the at least one text transcript further comprises the step of displaying the non-salient terms wherein they are unreadable.

11. The method of claim 5, wherein the step of determining salient terms in the at least one text transcript further comprises the steps of:

determining high selectivity terms in the at least one text transcript; and

selecting terms from the high selectivity terms that are above a predetermined selectivity.

12. The method of claim 11, wherein the step of determining salient terms in the at least one text transcript further comprises the step of:

removing terms whose category is uncertain from the high selectivity terms.

13. The method of claim 11, wherein the step of determining high selectivity terms further comprises the step of determining selectivity of terms in the at least one text transcript and determining terms having a predetermined selectivity, and wherein the step of selecting terms further comprises the step of selecting terms having the predetermined selectivity that are above a predetermined threshold.

14. A system for processing text transcripts of speech after imperfect speech recognition, the system comprising:

a memory that stores computer-readable code; and

a processor operatively coupled to the memory, the processor configured to implement the computer-readable code, the computer-readable code configured to;

convert a speech document to text;

process the text to determine salient terms; and

display the text by emphasizing the salient terms and minimizing non-salient terms, wherein said salient terms are determined by;

determining high selectivity terms in the at least one text transcript;

determining how many high selectivity terms there are;

determining how many multiword and high selectivity terms there are that are above a first predetermined selectivity;

when there are ten or more high selectivity and multiword terms that have confidences greater than a first predetermined selectivity, displaying the ten or more high selectivity and multiword terms; and

when there are less than ten high selectivity and multiword terms that are above a first predetermined selectivity;

determining how many single word and high selectivity terms there are;

determining if there are ten or more single word and multiword terms that are high selectivity terms that have selectivities greater than a first predetermined selectivity;

when there are ten or more single word and multiword terms that are high selectivity terms that have selectivity greater than a first predetermined selectivity, displaying all of the ten or more single word and multiword terms; and

when there are not ten or more single word and multiword terms that are high selectivity terms that have confidences greater than a first predetermined selectivity, displaying all single word and multiword terms that are high selectivity terms and that have confidences greater than a second predetermined selectivity.

15. The system of claim 14, wherein the computer-readable code is further configured, when processing the text to determine salient terms, to:

increase sentence and paragraph structure in the text;

remove non-word utterances from the text;

determine high selectivity terms in the text; and

select terms from the high selectivity terms that are above a predetermined selectivity.

16. The system of claim 14, wherein the computer-readable code is further configured to:

select a salient term;

determine a time of the salient term relative to a beginning time of the speech document; and

play the speech document from the time of the salient term.

17. The system of claim 14, wherein the computer-readable code is further configured to create the speech document by recording a conversation, and wherein the computer-readable code is further configured, when converting a speech document to text, to perform speech recognition on the speech document, wherein any speakers of the speech have not trained the speech recognition system with their own speech.

18. A system for processing text transcripts of speech after imperfect speech recognition, the system comprising:

a memory that stores computer-readable code; and

a processor operatively coupled to the memory, the processor configured to implement the computer-readable code, the computer-readable code configured to;

convert a speech document to at least one text transcript;

increase sentence and paragraph structure in the at least one text transcript;

remove non-word utterances from the at least one text transcript;

determine salient terms in the at least one text transcript; and

display text of the at least one text transcript, the step of displaying performed to emphasize display of the salient terms relative to non-salient terms, wherein said salient terms are determined by;

determining high selectivity terms in the at least one text transcript;

determining how many high selectivity terms there are;

determining how many multiword and high selectivity terms there are that are above a first predetermined selectivity;

when there are ten or more high selectivity and multiword terms that have confidences greater than a first predetermined selectivity, displaying the ten or more high selectivity and multiword terms; and

when there are less than ten high selectivity and multiword terms that are above a first predetermined selectivity;

determining how many single word and high selectivity terms there are;

determining if there are ten or more single word and multiword terms that are high selectivity terms that have selectivities greater than a first predetermined selectivity;

when there are ten or more single word and multiword terms that are high selectivity terms that have selectivity greater than a first predetermined selectivity, displaying all of the ten or more single word and multiword terms; and

when there are not ten or more single word and multiword terms that are high selectivity terms that have confidences greater than a first predetermined selectivity, displaying all single word and multiword terms that are high selectivity terms and that have confidences greater than a second predetermined selectivity.

19. The system of claim 18, wherein computer-readable code is further configured, when displaying text, to:

determine a position of a cursor when a click occurs;

determine if the position is within a predetermined distance from one of a plurality of salient terms that are being displayed; and

when the position is within the predetermined distance from one of the salient terms, perform the following steps;

determine which of the salient terms is near the position;

determine a time of the salient term; and

play the speech document, beginning at the time.

20. The system of claim 18, wherein computer-readable code is further configured, when determining salient terms in the at least one text transcript, to:

determine high selectivity terms in the at least one text transcript; and

select terms from the high selectivity terms that are above a predetermined selectivity.

21. The system of claim 20, wherein computer-readable code is further configured, when determining salient terms in the at least one text transcript, to:

remove terms whose category is uncertain from the high selectivity terms.

22. The system of claim 20, wherein computer-readable code is further configured, when determining high selectivity terms, to determine selectivity of terms in the at least one text transcript and determine terms having a predetermined selectivity, and wherein computer-readable code is further configured, when selecting terms, to select terms having the predetermined selectivity that are above a predetermined threshold.

23. An article of manufacture for processing text transcripts of speech after imperfect speech recognition, the article of manufacture comprising:

a step to convert a speech document to text;

a step to process the text to determine salient terms; and

a step to display the text by emphasizing the salient terms and minimizing non-salient terms, wherein said salient terms are determined by;

determining high selectivity terms in the at least one text transcript;

determining how many high selectivity terms there are;

determining how many multiword and high selectivity terms there are that are above a first predetermined selectivity;

when there are ten or more high selectivity and multiword terms that have confidences greater than a first predetermined selectivity, displaying the ten or more high selectivity and multiword terms; and

when there are less than ten high selectivity and multiword terms that are above a first predetermined selectivity;

determining how many single word and high selectivity terms there are;

determining if there are ten or more single word and multiword terms that are high selectivity terms that have selectivities greater than a first predetermined selectivity;

when there are ten or more single word and multiword terms that are high selectivity terms that have selectivity greater than a first predetermined selectivity, displaying all of the ten or more single word and multiword terms; and

when there are not ten or more single word and multiword terms that are high selectivity terms that have confidences greater than a first predetermined selectivity, displaying all single word and multiword terms that are high selectivity terms and that have confidences greater than a second predetermined selectivity.

24. The article of manufacture of claim 23, wherein the computer-readable code means further comprises, when processing the text to determine salient terms:

a step to increase sentence and paragraph structure in the text;

a step to remove non-word utterances from the text;

a step to determine high selectivity terms in the text; and

a step to select terms from the high selectivity terms that are above a predetermined selectivity.

25. The article of manufacture of claim 23, wherein the computer-readable code means further comprises:

a step to select a salient term;

a step to determine a time of the salient term relative to a beginning time of the speech document; and

a step to play the speech document from the time of the salient term.

26. The article of manufacture of claim 23, wherein the computer-readable code is further configured to create the speech document by recording a conversation, and wherein the computer-readable code is further configured, when converting a speech document to text, to perform speech recognition on the speech document, wherein any speakers of the speech have not trained the speech recognition article of manufacture with their own speech.

27. An article of manufacture for processing text transcripts of speech after imperfect speech recognition, the article of manufacture comprising:

a step to convert a speech document to at least one text transcript;

a step to increase sentence and paragraph structure in the at least one text transcript;

a step to remove non-word utterances from the at least one text transcript;

a step to determine salient terms in the at least one text transcript; and

a step to display text of the at least one text transcript, the step of displaying performed to emphasize display of the salient terms relative to non-salient terms, wherein said salient terms are determined by;

determining high selectivity terms in the at least one text transcript;

determining how many high selectivity terms there are;

determining how many multiword and high selectivity terms there are that are above a first predetermined selectivity;

when there are ten or more high selectivity and multiword terms that have confidences greater than a first predetermined selectivity, displaying the ten or more high selectivity and multiword terms; and

when there are less than ten high selectivity and multiword terms that are above a first predetermined selectivity;

determining how many single word and high selectivity terms there are;

determining if there are ten or more single word and multiword terms that are high selectivity terms that have selectivities greater than a first predetermined selectivity;

when there are ten or more single word and multiword terms that are high selectivity terms that have selectivity greater than a first predetermined selectivity, displaying all of the ten or more single word and multiword terms; and

when there are not ten or more single word and multiword terms that are high selectivity terms that have confidences greater than a first predetermined selectivity, displaying all single word and multiword terms that are high selectivity terms and that have confidences greater than a second predetermined selectivity.

28. The article of manufacture of claim 27, wherein computer-readable code means further comprises, when displaying text:

a step to determine a position of a cursor when a click occurs;

a step to determine if the position is within a predetermined distance from one of a plurality of salient terms that are being displayed; and

when the position is within the predetermined distance from one of the salient terms, the following steps;

a step to determine which of the salient terms is near the position;

a step to determine a time of the salient term; and

a step to play the speech document, beginning at the time.

29. The article of manufacture of claim 27, wherein computer-readable code means further comprises, when determining salient terms in the at least one text transcript:

a step to determine high selectivity terms in the at least one text transcript; and

a step to select terms from the high selectivity terms that are above a predetermined selectivity.

30. The article of manufacture of claim 29, wherein computer-readable code means further comprises, when determining salient terms in the at least one text transcript:

a step to remove terms whose category is uncertain from the high selectivity terms.

31. The article of manufacture of claim 29, wherein computer-readable code i means further comprises, when determining high selectivity terms, a step to determine selectivity of terms in the at least one text transcript and a step to determine terms having a predetermined selectivity, and wherein computer-readable code means further comprises, when selecting terms, a step to select terms having the predetermined selectivity that are above a predetermined threshold.