Automatic accuracy estimation for audio transcriptions

• US 10,002,606 B2
• Filed: 11/16/2017
• Issued: 06/19/2018
• Est. Priority Date: 05/03/2012
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bostick, James E., Ganci, John M., Kaemmerer, John P., Trim, Craig M.
• Primary Examiner(s): Desir, Pierre-Louis
• Assistant Examiner(s): Kim, Jonathan
• Application Number:
  US15/815,066
• Publication Number:
  US 20180075840A1
• Time in Patent Office: 215 Days
• Field of Search: None
• CPC Class Codes: G06F 40/157 using dictionaries or tables
• CPC Class Codes: G06F 40/189 Automatic justification
• CPC Class Codes: G06F 40/226 Validation
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: G06F 40/268 Morphological analysis
• CPC Class Codes: G06F 40/51 Translation evaluation
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 17/20 Pattern transformations or operations aimed at increasing system robustness, e.g. against channel noise or different working conditions

Abstract

Embodiments of the present invention provide an approach for estimating the accuracy of a transcription of a voice recording. Specifically, in a typical embodiment, each word of a transcription of a voice recording is checked against a customer-specific dictionary and/or a common language dictionary. The number of words not found in either dictionary is determined. An accuracy number for the transcription is calculated from the number of said words not found and the total number of words in the transcription.

Claims

1. A method of assigning a confidence level to at least one axiom extracted from a text, comprising:

determining a number of accurate words in the text based on a spell check function;

dividing the number of accurate words from the text by a total number of words in the text data;

automatically extracting, using natural language processing, from a knowledge base stored in a computer infrastructure, the at least one axiom, wherein the at least one axiom is associated with at least one word from the text, and wherein the axiom comprises a computer-parsable definition of a relationship of data to at least one of the words in the text;

assigning a confidence level to the at least one axiom based on a result of the dividing, wherein the confidence level is assigned based on an output of a Gaussian function applied to the result of the dividing;

receiving a query from a user device, wherein the query includes at least one word;

matching the at least one word with the at least one axiom using the knowledge base; and

providing the at least one axiom from the knowledge base to the user device based on the matching and the assigned confidence level.

2. The method of claim 1, further comprising:

comparing the text to a dictionary.

3. The method of claim 2, further comprising:

determining a number of inaccurate words in the text based on the comparing.

4. The method of claim 2, wherein the dictionary comprises a customer specific dictionary.

5. The method of claim 2, wherein the dictionary comprises a dictionary of common language words.

6. The method of claim 2, wherein the text is extracted from multiple different sources.

7. The method of claim 1, wherein the at least one axiom comprises a computer-parsable definition of a relationship of data to at least one word in a transcription of an audio recording.

8. The method of claim 1, wherein a solution service provider provides a computer infrastructure operable to perform the method.

9. A system for assigning a confidence level to at least one axiom extracted from a text, comprising:

a memory medium comprising instructions;

a bus coupled to the memory medium; and

an audio transcription tool coupled to the bus that when executing the instructions causes the system to;

determine a number of accurate words in the text based on a spell check function;

divide the number of accurate words from the text by a total number of words in the text;

automatically extract, using natural language processing, from a knowledge base stored in a computer infrastructure, the at least one axiom, wherein the at least one axiom is associated with at least one word from the text, and wherein the axiom comprises a computer-parsable definition of a relationship of data to at least one of the words in the text;

assign a confidence level to the at least one axiom based on a result of the dividing, wherein the confidence level is assigned based on an output of a Gaussian function applied to the result of the dividing;

receive a query from a user device, wherein the query includes at least one word;

match the at least one word with the at least one axiom using the knowledge base; and

provide the at least one axiom from the knowledge base to the user device based on the matching and the assigned confidence level.

10. The system of claim 9, the an audio transcription tool coupled to the bus that when executing the instructions further causes the system to:

compare the text to a dictionary.

11. The system of claim 10, wherein the dictionary comprises a customer specific dictionary.

12. The system of claim 10, wherein the dictionary comprises a dictionary of common language words.

13. The system of claim 10, wherein the text is extracted from multiple different sources.

14. The system of claim 9, wherein the at least one axiom comprises a computer-parsable definition of a relationship of data to at least one word in a transcription of an audio recording.

15. A computer program product for assigning a confidence level to at least one axiom extracted from a text, the computer program product comprising a computer readable hardware storage device, and program instructions stored on the computer readable hardware storage device, to:

determine a number of accurate words in the text based on a spell check function;

divide the number of accurate words from the text by a total number of words in the text;

automatically extract, using natural language processing, from a knowledge base stored in a computer infrastructure, the at least one axiom, wherein the at least one axiom is associated with at least one word from the text, and wherein the axiom comprises a computer-parsable definition of a relationship of data to at least one of the words in the text;

assign a confidence level to the at least one axiom based on a result of the dividing, wherein the confidence level is assigned based on an output of a Gaussian function applied to the result of the dividing;

receive a query from a user device, wherein the query includes at least one word;

match the at least one word with the at least one axiom using the knowledge base; and

provide the at least one axiom from the knowledge base to the user device based on the matching and the assigned confidence level.

16. The computer program product of claim 15, the computer program product comprising a computer readable hardware storage device, and program instructions stored on the computer readable storage device, to:

compare the text to a dictionary.

17. The computer program product of claim 16, wherein the dictionary comprises a customer specific dictionary.

18. The computer program product of claim 16, wherein the dictionary comprises a dictionary of common language words.

19. The computer program product of claim 15, wherein the text is extracted from multiple different sources.

20. The computer program product of claim 15, wherein the at least one axiom comprises a computer-parsable definition of a relationship of data to at least one word in a transcription of an audio recording.