System and method for targeted tuning module of a speech recognition system

• US 20060036437A1
• Filed: 08/12/2004
• Published: 02/16/2006
• Est. Priority Date: 08/12/2004
• Status: Active Grant

• Current Assignee: Interactions, LLC
• Original Assignee: Sbc Knowledge Ventures, L.P.
• Inventors: Knott, Benjamin Anthony, Bushey, Robert R., Martin, John Mills
• Granted Patent: US 7,580,837 B2
• Time in Patent Office: Days
• US Class Current: 704/231
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/065 Adaptation
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

A system and method are disclosed for targeted tuning of a speech recognition system. A method incorporating teachings of the present disclosure may include deploying a speech recognition module to apply an appropriate interpretation to a plurality of utterance types. The method may also include accessing information representing a collection of recorded utterances and assigned interpretation for each of the plurality of recorded utterances. The assigned interpretation for each of the plurality of recorded utterances may then be compared to an accurate interpretation for each of the plurality of utterance, and a separate accuracy value may be determined for each of the plurality of utterance types. With some implementations, if the separate accuracy value for a given type of utterance is too low, a selection of utterances having the given type may be used to tune the speech recognition module.

Claims

1. A method of tuning a speech system comprising:

accessing information representing a plurality of utterances for at least one speech-enabled application, the plurality of utterances comprising at least a first type of utterance and a second type of utterance;

accessing interpretive information representing an assigned interpretation for at least a portion of the plurality of utterances;

determining an appropriate interpretation for the portion of the plurality of utterances;

comparing the assigned interpretation for the portion of the plurality of utterances to the appropriate interpretation for the portion of the plurality of utterances;

determining that the speech-enabled application more accurately responds to the first type of utterance; and

electing to apply a targeted tuning to the speech-enabled application to improve recognition of the second type of utterance.

2. The method of claim 1, further comprising tuning the speech-enabled application to improve recognition of the second type of utterance by feeding a collection of the second type of utterances into a learning module of the speech-enabled application.

3. The method of claim 2, wherein the tuning step comprises avoiding a feeding of the first type of utterances into the learning module.

4. The method of claim 2, wherein feeding the collection of the second type of utterances into the learning module comprises:

playing a file representing a second type of utterance recording; and

inputting the appropriate interpretation for the recording.

5. The method of claim 1, further comprising improving recognition of the second type of utterance without degrading recognition of the first type of utterance.

6. The method of claim 1, wherein the speech-enabled application executes at an automated call router.

7. The method of claim 1, wherein the speech-enabled application executes at a voice activated services platform.

8. The method of claim 1, wherein the speech-enabled application executes in connection with a call center.

9. The method of claim 1, wherein the assigned interpretation comprises an action to be performed.

10. The method of claim 9, wherein the action-object to be performed is selected from a group consisting of a pay bill action, a transfer to agent action, an inquire about balance action, a change service action, an acquire service action, a cancel service action, an inquire about a bill action, an acquire about an account action, a schedule payment action, a reconnect service action, and another business-related combination of an action and an object to be acted upon in accordance with the action.

11. The method of claim 1, wherein the plurality of utterances comprises an accumulation of utterances received via a deployed speech-enabled application, further wherein the portion of the plurality of utterances comprises all of the accumulation of utterances.

12. The method of claim 1, further comprising storing information representing the plurality of utterances as discrete audio files.

13. The method of claim 12, wherein the discrete audio files have a format selected from a group consisting of a WAV format, an MP3 format, an AU format, and a MIDI format.

14. The method of claim 1, wherein the at least one speech-enabled application comprises an application deployed in an operational environment.

15. The method of claim 1, wherein determining that the speech-enabled application more accurately responds to the first type of utterance comprises:

calculating a system hit rate for the first type of utterance, wherein the system hit rate for the first type of utterance reflects how often the at least one speech-enabled application applied a first type interpretation to a received first type utterance; and

calculating a system hit rate for the second type of utterance.

16. The method of claim 1, further comprising calculating a system error rate for the first type of utterance, wherein the system error rate for the first type of utterance reflects how often the at least one speech-enabled application misapplies a first type interpretation to a received utterance of a type other than the first type of utterance.

17. The method of claim 1, further comprising setting an utterance type-specific hit rate design threshold for each of a collection of expected utterance types, wherein the targeted tuning comprises exclusively tuning the speech-enabled application to utterance types having an actual utterance type specific hit rate that fails to reach a respective utterance type-specific hit rate design threshold.

18. A speech tuning system, comprising:

a repository storing a sample of captured utterances from an implemented speech-enabled application and an assigned utterance type for each of the captured utterances;

an accuracy engine communicatively coupled to the repository and operable to determine if an assigned utterance type for a given captured utterance represents an accurate interpretation of the given captured utterance; and

a targeting engine communicatively coupled to the accuracy engine and operable to determine a first accuracy level of the speech-enabled application in identifying a first type of utterance and a second accuracy level of the speech-enabled application in identifying a second type of utterance.

19. The system of claim 18, further comprising a tuning engine operable to feed the speech-enabled application with a collection of utterances having the first type if the first accuracy level is lower than the second accuracy level.

20. The system of claim 19, wherein the sample of captured utterances comprises the collection of utterances.

21. The system of claim 18, further comprising a call center that comprises the implemented speech-enabled application.

22. The system of claim 18, further comprising a computer readable medium, wherein a set of instructions embodying the accuracy engine and the tuning engine are stored on the computer readable medium.

23. The system of claim 18, further comprising an automated call router that comprises the implemented speech-enabled application.

24. The system of claim 18, further comprising a voice activated services platform that comprises the implemented speech-enabled application.

25. A method of tuning a speech-enabled application comprising:

deploying a speech-recognition module to apply an appropriate interpretation to a plurality of utterance types;

accessing information representing a collection of recorded utterances and assigned interpretation for each of the plurality of recorded utterances;

comparing the assigned interpretation for each of the plurality of recorded utterances to an accurate interpretation for each of the plurality of utterances; and

determining a separate accuracy value for each of the plurality of utterance types.

26. The method of claim 25, further comprising recording the collection of recorded utterances as discrete audio files.

27. The method of claim 25, further comprising feeding the speech-recognition module with a selection of utterances having a given type if the separate accuracy value for the given type is lower than an accuracy threshold value.

28. The method of claim 27, further comprising ensuring that the selection of utterances does not include a different utterance type if the separate accuracy value for the different utterance type is at or above the accuracy threshold value.

29. The method of claim 28, further comprising determining a new accuracy value for the given type of utterance.