Testing speech recognition systems using test data generated by text-to-speech conversion

• US 6,622,121 B1
• Filed: 08/09/2000
• Issued: 09/16/2003
• Est. Priority Date: 08/20/1999
• Status: Expired due to Term

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Kusnitz, Jeffrey A., Lewis, Burn, Crepy, Hubert
• Primary Examiner(s): Ŝmits, Ta̅livaldis Ivars
• Application Number:
  US09/634,179
• Time in Patent Office: 1,133 Days
• Field of Search: 704/251, 704/243, 704/260
• US Class Current: 704/243
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems

Abstract

Methods and systems for testing speech recognition systems are disclosed in which the speech recognition device to be tested is directly monitored in accordance with a text-to-speech device. The collection of reference texts to be used by the speech recognition device is provided by a text-to-speech device preferably, in one embodiment, implemented within the same computer system. In such an embodiment, a digital audio file stored within a storage area of a computer system is generated from a reference text using a text-to-speech device. The digital audio file is later read using a speech recognition device to generate a decoded (or recognized) text representative of the reference text. The reference text and the decoded text are compared in an alignment operation and an error report representative of the recognition rate of the speech recognition device is finally generated.

Claims

1. A method for testing speech recognition systems, the method comprising the steps of:

a) generating a digital audio file from a reference text using a text-to-speech device, the digital audio file being stored within a storage area of a computer system; and

b) reading the digital audio file using a speech recognition device to generate a decoded text representative of the reference text.

2. The method of claim 1, wherein the text-to-speech device and the speech recognition device are implemented within a single computer system.

3. The method of claim 1, wherein the step of generating the digital audio file further comprises the steps of:

a1) tokenizing an initial text stored on a storage area of the computer system to generate a tokenized text;

a2) marking-up of the tokenized text to generate a marked text; and

a3) synthesizing the marked text to generate the digital audio file.

4. The method of claim 1, further comprising the steps of:

c) processing an alignment operation between the reference text and the decoded text; and

d) generating an error report representative of a recognition rate of the speech recognition device.

5. Apparatus for testing speech recognition systems, the apparatus comprising:

at least one processor operative to;

(a) generate and store a digital audio file from a reference text using a text-to-speech process; and

(b) read the digital audio file using a speech recognition process to generate a decoded text representative of the reference text.

6. An article of manufacture for testing speech recognition systems, comprising a machine readable medium containing one or more programs which when executed implement the steps of:

a) generating a digital audio file from a reference text using a text-to-speech device, the digital audio file being stored within a storage area of a computer system; and

b) reading the digital audio file using a speech recognition device to generate a decoded text representative of the reference text.

7. A method for testing speech recognition systems, the method comprising the steps of:

a) generating a synthetic speech from a reference text using a text-to-speech device; and

b) processing the synthetic speech using a speech recognition device to generate a decoded text representative of the reference text.

8. The method of claim 7, wherein the text-to-speech device is implemented within a first computer system and the speech recognition device is implemented within a second computer system.

9. The method of claim 8, wherein the step of processing the synthetic speech further comprises the steps of:

b0) synthesizing the speech into audible audio using an audio output device connected to the first computer system;

b1) recording the synthetic speech using an audio input device connected to the second computer system to generate an audio file; and

b2) recognizing the recorded audio file to output the decoded text.

10. The method of claim 9, wherein the step of generating an audio file further comprises the steps of:

a1) tokenizing an initial text stored on a storage area of a computer system to generate a tokenized text;

a2) marking-up of the tokenized text to generate a marked text; and

a3) synthesizing the marked text to generate the audio file.

11. The method of claim 7, further comprising the steps of:

c) processing an alignment operation between the reference text and the decoded text; and

d) generating an error report representative of a recognition rate of the speech recognition device.

12. Apparatus for testing speech recognition systems, the apparatus comprising:

at least one processor operative to generate a synthetic speech from a reference text using a text-to-speech process; and

at least a second processor operative to process the synthetic speech using a speech recognition process to generate a decoded text representative of the reference text.

13. An article of manufacture for testing speech recognition systems, comprising a machine readable medium containing one or more programs which when executed implement the steps of:

a) generating a synthetic speech from a reference text using a text-to-speech device; and

b) processing the synthetic speech using a speech recognition device to generate a decoded text representative of the reference text.