METHOD AND APPARATUS FOR DETERMINING A RECOGNITION RATE

• US 20170133008A1
• Filed: 08/02/2016
• Published: 05/11/2017
• Est. Priority Date: 11/05/2015
• Status: Abandoned Application

• Current Assignee: Le Holdings (Beijing) Co., Ltd., Le Shi Zhi Xin Electronic Technology (Tianjin) Limited
• Original Assignee: Le Holdings (Beijing) Co., Ltd., Le Shi Zhi Xin Electronic Technology (Tianjin) Limited
• Inventors: WANG, Yujun
• Application Number:
  US15/226,169
• Publication Number:
  US 20170133008A1
• Time in Patent Office: Days
• CPC Class Codes: G10L 15/005 Language recognition
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems
• CPC Class Codes: G10L 15/10 using distance or distortion measures between unknown speech and reference templates
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems

Abstract

Disclosed are a method and apparatus for determining a recognition rate, and the method can obtain a string of characters obtained by recognizing voice, and a standard recognition result corresponding to the string of characters, where the standard recognition result includes phonetic character type, and Chinese characters; divide the string of characters according to a character type in the string of characters to generate a sequence of characters, and divide the standard recognition result to generate a standard recognition result sequence: calculate the shortest edition distance between the sequence of characters, and the standard recognition result sequence; and determine a recognition rate of a voice recognition apparatus according to the calculated shortest edition distance.

Claims

1. A method for determining a recognition rate, applicable to a terminal, comprising:

obtaining a string of characters obtained by recognizing voice, and a standard recognition result corresponding to the string of characters, wherein the standard recognition result comprises characters of phonetic character type, and characters of Chinese character type;

dividing the string of characters according to a character type in the string of characters to generate a sequence of characters, wherein when the string of characters comprises phonetic character, a number of phonetic characters representing one complete meaning is divided into a recognition element;

calculating a shortest edition distance between the sequence of characters, and a standard recognition result sequence generated by dividing the standard recognition result sequence;

obtaining an optimum alignment result between the sequence of characters and the standard recognition result sequence according to a calculated shortest edition distance; and

determining a recognition rate of the sequence of characters with respect to the standard recognition result sequence according to the optimum alignment result between the sequence of charactes and the standard recognition result sequence, wherein the recognition rate comprises a recognition error rate of phonetic characters, and a recognition error rate of Chinese characters.

2. The method according to claim 1, wherein dividing the string of characters to according to the character type in the string of characters generate the sequence of characters comprises:

for any one character in the string of characters, when the character type of the any one character is the Chinese character type, determining the any one character as a recognition element; and

when the any one character is a phonetic character, if the any one character is not a first character in the string of characters, and the any one character is located between two Space symbols, or the any one character is the first character in the string of characters, and a next position to the any one character is the Space symbol, then determining the any one character as a recognition element, otherwise, locating the closest two Space symbols to the any one character respectively, and determining all the characters between the located two Space symbols as a recognition element;

sorting respective determined recognition elements according to the positions of the determined recognition elements in the string of characters; and

determining sorted recognition elements as the sequence of characters.

3. The method according to claim 2, wherein calculating the shortest edition distance between the sequence of characters, and the standard recognition result sequence comprises:

creating a two-dimension grid, wherein a first dimension of the two-dimension grid represents the recognition elements in the sequence of characters, and a second dimension of the two-dimension grid represents the standard elements in the standard recognition result sequence;

counting the number of instances of each error type corresponding to each grid element in the two-dimension grid respectively in the left to right direction and the top to bottom direction in the two-dimension grid, wherein the number of instances of the each error type is a sum of the number of instances of the error type in a preceding grid element corresponding to the error type, and the number of instances of the error type of the recognition element corresponding to the grid element with respect to the standard element, and the preceding grid element is a grid element, adjacent to a current grid element, to which a backtracking pointer corresponding to the error type points;

adding counted number of instances of each error type corresponding to each grid element in the two-dimension grid to the corresponding grid element;

selecting a grid element in last row and last column in the two-dimension network, and determining such one of respective error types corresponding to selected grid element that has the smallest number of instances; and

determining the number of instances of the determined error type as the shortest edition distance between the sequence of characters and the standard recognition result sequence.

4. The method according to claim 3, wherein obtaining the optimum alignment result between the sequence of characters and the standard recognition result sequence comprises:

for each grid element in the two-dimension grid, performing the operations of;

determining such one of the respective error types corresponding to the grid element that has the smallest number of instances;

determining the number of instances of the determined error type as the smallest number of error instances corresponding to the grid element; and

obtaining the backtracking pointer corresponding to the determined error type;

determining a set of alignment relationships between the respective recognition elements corresponding to the sequence of characters, and the respective standard elements corresponding to the standard recognition result according to the pointing direction of the backtracking pointer obtained in each grid element starting from the grid element corresponding to the shortest edition distance in the two-dimension grid; and

determining the determined set of alignment relationships between the respective recognition elements corresponding to the sequence of characters, and the respective standard elements corresponding to the standard recognition result as the optimum alignment result between the sequence of characters and the standard recognition result sequence.

5. The method according to claim 4, wherein determining the recognition rate of the sequence of characters with respect to the standard recognition result sequence according to the optimum alignment result between the sequence of characters and the standard recognition result sequence comprises:

obtaining an error type corresponding to each alignment relationship in the set of alignment relationships, and the number of instances of the error type; and

determining the recognition rate of the sequence of characters with respect to the standard recognition result sequence according to the number of instances of the error type corresponding to each alignment relationship in the set of alignment relationships.

6. The method according to claim 5, wherein determining the recognition rate of the sequence of characters with respect to the standard recognition result sequence according to the number of instances of the error type corresponding to each alignment relationship in the set of alignment relationships comprises:

selecting a correspondence relationship of Chinese characters in the set of alignment relationships, wherein the correspondence relationship of Chinese characters comprises standard elements of Chinese characters; and

calculating a rate of the number of correspondence relationships of all the recognition errors in the selected correspondence relationship to the total number of standard elements of Chinese characters as the recognition error rate of Chinese characters of the sequence of characters with respect to the standard recognition result sequence; and

selecting a correspondence relatioship of phonetic characters in the set of alignment relationships, wherein the correspondence relationship of phonetic characters comprises standard elements of phonetc characters; and

calculating a rate of the number of error types of correspondence relationships of all the recognition errors in the selected correspondence relationship to the total number of standard elements of phonetic characters as the recognition error rate of phonetic characters of the sequence of characters with respect to the standard recognition result sequence.

8. The electronic device according to claim 1, wherein the divide the string of characters according to a character type in the string of characters to generate a sequence of characters comprises:

for any one character in the string of characters, when the character type of the any one character is the Chinese character type, determine the any one character as a recognition element; and

when the any one character is a phonetic character, if the any one character is not a first character in the string of characters, and the any one character is located between two Space symbols, or the any one character is the first character in the string of characters, and a next position to the any one character is the Space symbol, determine the any one character as a recognition element, otherwise, locate the closest two Space symbols to the any one character respectively, and determine all the characters between the located two Space symbols as a recognition element;

sort the respective determined recognition elements according to the positions of the determined recognition elements in the string of characters; and

determine the sorted recognition elements as the sequence of characters.

9. The electronic device according to claim 8, wherein the calculate a shortest edition distance between the sequence of characters, and a standard recognition result sequence generated by dividing the standard recognition result comprises:

create a two-dimension grid, wherein a first dimension of the two-dimension grid represents the recognition elements in the sequence of characters, and a second dimension of the two-dimension grid represents the standard elements in the standard recognition result sequence;

count the number of instances of each error type corresponding to each grid element in the two-dimension grid respectively in the left to right direction and the top to bottom direction in the two-dimension grid, wherein the number of instances of the each error type is a sum of the number of instances of the error type in a preceding grid element corresponding to the error type, and the number of instances of the error type of the recognition element corresponding to the grid element with respect to the standard element, and the preceding grid element is a grid element, adjacent to a current grid element to which a backtracking pointer corresponding to the error type points;

add counted number of instances of each error type corresponding to each grid element in the two-dimension grid to the corresponding grid element;

select a grid element in a last row and a last column in the two-dimension network, and determine such one of the respective error types corresponding to the selected grid element that has the smallest number of instances; and

determine the number of instances of the determined error type as the shortest edition distance between the sequence of characters and the standard recognition result sequence.

10. The electronic device according to claim 9, wherein the obtain an optimum alignment result between the sequence of characters and the standard recognition result sequence according to the calculated shortest edition distance comprises:

for each grid element in the two-dimension grid, perform the operations of;

determining such one of the respective error types corresponding to the grid element that has the smallest number of instances;

determining the number of instances of the determined error type as the smallest number of error instances corresponding to the grid element; and

obtaining the backtracking pointer corresponding to the determined error type;

determining a set of alignment, relationships between the respective recognition elements corresponding to the sequence of characters, and the respective standard elements corresponding to the standard recognition result according to the pointing direction of the backtracking pointer obtained in each grid element starting from the grid element corresponding to the shortest edition distance in the two-dimension grid; and

determining the determined set of alignment relationships between the respective recognition elements corresponding to the sequence of characters, and the respective standard elements corresponding to the standard recognition result as the optimum alignment result between the sequence of characters and the standard recognition result sequence.

11. The electronic device according to claim 10, wherein the determine a recognition, rate of the sequence of characters with respect to the standard recognition result sequence according to the optimum alignment result between the sequence of characters and the standard recognition result sequence comprises:

obtain an error type corresponding to each alignment relationship in the set of alignment relationships, and the number of instances of the error type; and

determine the recognition rate of the sequence of characters with respect to the standard recognition result sequence according to the number of instances of the error type corresponding to each alignment relationship in the set of alignment relationships.

12. The electronic device according to claim 11, wherein the determine the recognition rate of the sequence of characters with respect to the standard recognition result sequence according to the number of instances of the error type corresponding to each alignment relationship in the set of alignment relationships comprises:

select a correspondence relationship of Chinese characters in the set of alignment relationships, wherein the correspondence relationship of Chinese characters comprises standard elements of Chinese characters; and

calculate a rate of the number of correspondence relationships of all the recognition errors in the selected correspondence relationship to the total number of standard elements of Chinese characters as the recognition error rate of Chinese characters of the sequence of characters with respect to the standard recognition result sequence; and

select a correspondence relationship of phonetic characters in the set of alignment relationships, wherein the correspondence relationship of phonetic characters comprises standard elements of phonetic characters; and

calculate a rate of the number of error types of correspondence relationships of all the recognition, errors in the selected correspondence relationship to the total number of standard elements of phonetic characters as the recognition error rate of phonetic characters of the sequence of characters with respect to the standard recognition result sequence.

7. An electronic device, comprising:

at least one processor; and

a memory communicably connected with the at least one processor for storing instructions executable by the at least one processor, wherein execution of the instructions by the at least one processor causes the at least one processor to;

obtain a string of characters obtained by recognizing voice, and a standard recognition result corresponding to the string of characters, wherein the standard recognition result comprises characters of phonetic character type, and characters of Chinese character type;

divide the string of characters according to a character type in the string of characters to generate a sequence of characters, wherein when the string of Characters comprises phonetic character, a number of phonetic characters representing one complete meaning is divided into a recognition element;

calculate a shortest edition distance between the sequence of characters, and a standard recognition result sequence generated by dividing the standard recognition result;

obtain an optimum alignment result between the sequence of characters and the standard recognition result sequence according to the calculated shortest edition distance; and

determine a recognition rate of the sequence of characters with respect to the standard recognition result sequence according to the optimum alignment result between the sequence of characters and the standard recognition result sequence, wherein the recognition rate comprises a recognition error rate of phonetic characters, and a recognition error rate of Chinese characters.

13. A non-transitory computer-readable storage medium storing executable instructions that, when executed by an electronic device, cause the electronic device to:

obtain a string of characters obtained by recognizing voice, and a standard recognition result corresponding to the string of characters, wherein the standard recognition result comprises characters of phonetic character type, and characters of Chinese character type;

divide the string of characters according to a character type in the string of characters to generate a sequence of characters, wherein when the string of characters comprises phonetic character, a number of phonetic characters representing one complete meaning is divided into a recognition element;

calculate a shortest edition distance between the sequence of characters, and a standard recognition result sequence generated by dividing the standard recognition result;

obtain an optimum alignment result between the sequence of characters and the standard recognition result sequence according to the calculated shortest edition distance; and

determine a recognition rate of the sequence of characters with respect to the standard recognition result sequence according to the optimum alignment result between the sequence of characters and the standard recognition result sequence, wherein the recognition rate comprises a recognition error rate of phonetic characters, and a recognition error rate of Chinese characters.

14. The non-transitory computer-readable storage medium according to claim 13, wherein the divide the string of characters according to a. character type in the string of characters to generate a sequence of characters comprises:

for any one character In the string of characters, when the character type of the any one character is the Chinese character type, determine the any one character as a recognition element; and

when the any one character is a phonetic character, if the any one character is not a first character In the string of characters, and the any one character is located between two Space symbols, or the any one character is the first character in the string of characters, and a next position, to the any one character is the Space symbol, determine the any one character as a recognition element, otherwise, locate the closest two Space symbols to the any one character respectively, and determine all the characters between the located two Space symbols as a recognition demerit;

sort the respective determined recognition elements according to the positions of the determined recognition elements in the string of characters; and

determine the sorted recognition elements as the sequence of characters.

15. The non-transitory computer-readable storage medium according to claim 14, wherein the calculate a shortest edition distance between the sequence of characters, and a standard recognition result sequence generated by dividing the standard recognition result comprises:

create a two-dimension grid, wherein a first dimension of the two-dimension grid represents the recognition elements in the sequence of characters, and a second dimension of the two-dimension grid represents the standard elements in the standard recognition result sequence;

count the number of instances of each error type corresponding to each grid element in the two-dimension grid respectively in the left to right direction and the top to bottom direction in the two-dimension grid, wherein the number of instances of the each error type is a sum of the number of instances of the error type in a preceding grid element corresponding to the error type, and the number of instances of the error type of the recognition element corresponding to the grid element with respect to the standard element, and the preceding grid element is a grid element, adjacent to a current grid element, to which a backtracking pointer corresponding to the error type points;

add counted number of instances of each error type corresponding to each grid element in the two-dimension grid to the corresponding grid element;

select a grid element in a last row and a last column in the two-dimension network, and determine such one of the respective error types corresponding to the selected grid element that has the smallest number of instances; and

determine the number of instances of the determined error type as the shortest edition distance between the sequence of characters and the standard recognition result sequence.

16. The non-transitory computer-readable storage medium according to claim IS, wherein the obtain an optimum alignment result between the sequence of characters and the standard recognition result sequence according to the calculated shortest edition distance comprises:

for each grid element in the two-dimension grid, perform the operations of

determining such one of the respective error types corresponding to the grid element that, has the smallest number of instances;

determining the number of instances of the determined

determining a set of alignment relationships between the respective recognition elements corresponding to the sequence of characters, and the respective standard elements corresponding to the standard recognition result according to the pointing direction of the backtracking pointer obtained in each grid element starting from the grid element corresponding to the shortest edition distance in the two-dimension grid; and

determining the determined set of alignment relationships between the respective recognition elements corresponding to the sequence of characters, and the respective standard elements corresponding to the standard recognition result as the optimum alignment result between the sequence of characters and the standard recognition result sequence,

17. The non-transitory computer-readable storage medium according to claim 16, wherein the determine a recognition rate of the sequence of characters with respect to the standard recognition result sequence according to the optimum alignment result between the sequence of characters and the standard recognition result sequence comprises:

obtain an error type corresponding to each alignment relationship in the set of alignment relationships, and the number of instances of the error type; and

determine the recognition rate of the sequence of characters with respect to the standard recognition result sequence according to the number of instances of the error type corresponding to each alignment relationship in the set of alignment relationships,

18. The non-transitory computer-readable storage medium according to claim 17, wherein the determine the recognition rate of the sequence of characters with respect to the standard recognition result sequence according to the number of instances of the error type corresponding to each alignment relationship in the set of alignment relationships comprises:

select a correspondence relationship of Chinese characters in the set of alignment relationships, wherein the correspondence relationship of Chinese characters comprises standard elements of Chinese characters; and

calculate a rate of the number of correspondence relationships of all the recognition errors in the selected correspondence relationship to the total number of standard elements of Chinese characters as the recognition error rate of Chinese characters of the sequence, of characters with respect to the standard recognition result sequence; and

select a standard elements of phonetic characters, and calculate a rate of the number of error types of correspondence relationship of all the recognition errors in the select correspondence relationship to the total number of standard elements of phonetic characters as the recognition error rate of phonetic characters of the sequence of characters with respect to the standard recognition result sequence.