Determining voice recognition accuracy in a voice recognition system
First Claim
1. A method for determining voice recognition accuracy of a voice recognition system that recognizes a voice command in a user utterance, comprising:
- analyzing voice recognition information produced by a voice recognition system, including comparing a manually transcribed version of the user utterance to the recognized voice command;
detecting, in a computing device, at least one recognition error based on the analysis, comprising detecting at least one of;
an in-grammar false accept (IGFA) type error,an in-grammar false reject (IGFR) type error,an out-of-grammar false accept (OGFA) type error, oran out-of-grammar correct reject (OGCR) type error;
determining at least one source of error for the detected recognition error; and
implementing at least one solution to eliminate the source of error.
4 Assignments
0 Petitions
Accused Products
Abstract
Systems, methods, and computer program products for determining voice recognition accuracy of a voice recognition system are provided. In one embodiment, voice recognition information produced by a voice recognition system in response to recognizing a user utterance is analyzed. The voice recognition information comprises a recognized voice command associated with the user utterance and a reference to an audio file that includes the user utterance. Based on the analysis, a recognition error may be identified and the source of the error determined. A solution is then automatically implemented to eliminate the source of the error. As part of the analysis, the user utterance may be transcribed to create a transcribed utterance, if the recognized voice command does not match the user utterance. The transcribed utterance may then be compared to the recognized voice command to identify an error.
184 Citations
31 Claims
-
1. A method for determining voice recognition accuracy of a voice recognition system that recognizes a voice command in a user utterance, comprising:
-
analyzing voice recognition information produced by a voice recognition system, including comparing a manually transcribed version of the user utterance to the recognized voice command; detecting, in a computing device, at least one recognition error based on the analysis, comprising detecting at least one of; an in-grammar false accept (IGFA) type error, an in-grammar false reject (IGFR) type error, an out-of-grammar false accept (OGFA) type error, or an out-of-grammar correct reject (OGCR) type error; determining at least one source of error for the detected recognition error; and implementing at least one solution to eliminate the source of error. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18)
-
-
19. A method for determining voice recognition accuracy of a voice recognition system that recognizes a voice command in a user utterance, comprising:
-
analyzing voice recognition information produced by a voice recognition system, including comparing a manually transcribed version of the user utterance to the recognized voice command, comprising analyzing a response time for the voice recognition system to recognize the user utterance as the recognized voice command; detecting, in a computing device, at least one recognition error based on the analysis; determining at least one source of error for the detected recognition error; implementing at least one solution to eliminate the source of error; and reducing a perplexity of the grammar if the response time exceeds a threshold value, the perplexity based on a quantity of decision levels and a quantity of voice commands.
-
-
20. A method for determining voice recognition accuracy of a voice recognition system that recognizes a voice command in a user utterance, comprising:
-
analyzing voice recognition information produced by a voice recognition system, including comparing a manually transcribed version of the user utterance to the recognized voice command, comprising analyzing a quantity of timeout errors associated with a timeout period, the timeout period defining a period of time for a user to provide the user utterance; detecting, in a computing device, at least one recognition error based on the analysis; determining at least one source of error for the detected recognition error; implementing at least one solution to eliminate the source of error; and increasing the timeout period if the quantity of timeout errors exceeds a threshold value.
-
-
21. A tangible computer-readable medium having computer program logic stored thereon that enables a computing device to provide an automated method for determining voice recognition accuracy of a voice recognition system that recognizes a voice command in a user utterance, the computer program logic comprising:
-
an analyzing module configured to analyze voice recognition information produced by the voice recognition system and to compare a manually transcribed version of the user utterance to the recognized voice command; a detecting module configured to detect at least one recognition error based on the analysis, comprising detecting at least one of; an in-grammar false accept (IGFA) type error, an in-grammar false reject (IGFR) type error, an out-of-grammar false accept (OGFA) type error, or an out-of-grammar correct reject (OGCR) type error; a determining module configured to determine at least one source of error for the detected recognition error; and a generating module configured to generate at least one solution to eliminate the source of error. - View Dependent Claims (22, 23)
-
-
24. A system for determining voice recognition accuracy of a voice recognition system that recognizes a voice command in a user utterance, comprising:
-
an analysis module that analyzes voice recognition information produced by the voice recognition system and comparing a manually transcribed version of the user utterance to the recognized voice command; an error detection module that detects at least one recognition error based on the analysis, comprising detecting at least one of; an in-grammar false accept (IGFA) type error, an in-grammar false reject (IGFR) type error, an out-of-grammar false accept (OGFA) type error, or an out-of-grammar correct reject (OGCR) type error; an error source identification module that identifies at least one source of error for the detected recognition error; and a solution generation module that generates at least one solution to eliminate the source of error, wherein at least one of the analysis module, the error detection module, the error source identification module, or the solution generation module is executed by a computing device. - View Dependent Claims (25, 26)
-
-
27. A device for determining voice recognition accuracy of a voice recognition system that recognizes a voice command in a user utterance, comprising:
-
an analysis module that analyzes voice recognition information produced by the voice recognition system and compares a manually transcribed version of the user utterance to the recognized voice command; an error detection module that detects at least one recognition error based on the analysis, comprising detecting at least one of; an in-grammar false accept (IGFA) type error, an in-grammar false reject (IGFR) type error, an out-of-grammar false accept (OGFA) type error, or an out-of-grammar correct reject (OGCR) type error; an error source identification module that identifies at least one source of error for the detected recognition error; and a solution generation module that generates at least one solution to eliminate the source of error. - View Dependent Claims (28, 29)
-
-
30. A computer system for determining voice recognition accuracy of a voice recognition system that recognizes a voice command in a user utterance, comprising:
-
an interface that receives voice recognition information produced by a voice recognition system; a memory; and at least one processor that executes computer program logic residing in the memory, wherein the at least one processor executes; analysis logic to analyze the voice recognition information and to compare a manually transcribed version of the user utterance to the recognized voice command, error detection logic to detect at least one recognition error based on the analysis, comprising detecting at least one of; an in-grammar false accept (IGFA) type error, an in-grammar false reject (IGFR) type error, an out-of-grammar false accept (OGFA) type error, or an out-of-grammar correct reject (OGCR) type error, error source identification logic to identify at least one source of error for the detected recognition error, and solution generation logic to generate at least one solution to eliminate the source of error. - View Dependent Claims (31)
-
Specification