Detecting speech input phrase confusion risk

• US 9,384,731 B2
• Filed: 11/06/2013
• Issued: 07/05/2016
• Est. Priority Date: 11/06/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Tjalve, Michael, Karnam, Pavan, Mooney, Dennis
• Primary Examiner(s): Desir, Pierre-Louis
• Assistant Examiner(s): KOVACEK, DAVID M
• Application Number:
  US14/073,631
• Publication Number:
  US 20150127347A1
• Time in Patent Office: 972 Days
• Field of Search: 704 1- 10, 704/200.1, 704231-232, 704236-240, 704246-255, 704/257, 704/270, 704E15001-E1505, 704E11001-E11007
• US Class Current: 1/1
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: G10L 2015/0635 updating or merging of old and new templates; Mean values; Weighting
• CPC Class Codes: G10L 2025/786 Adaptive threshold
• CPC Class Codes: G10L 25/48 specially adapted for particular use
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10

Abstract

Embodiments are disclosed that relate to identifying phonetically similar speech grammar terms during computer program development. For example, one disclosed embodiment provides a method including providing a speech grammar development tool configured to receive input of a text representation of each of a plurality of proposed speech grammar terms, convert each text representation to a phonetic representation of the speech grammar term, compare the phonetic representation of the speech grammar term to the phonetic representations of other speech grammar terms using a weighted similarity matrix, and provide an output regarding risk of confusion between two proposed speech grammar terms based upon a comparison of the phonetic representations of the two proposed speech grammar terms. The method further includes receiving data regarding incorrect speech grammar term identification, and modifying one or more weights in the weighted similarity matrix based upon the data.

Claims

1. On a computing system comprising a processor, a method of detecting phrase confusion risk in a proposed speech grammar for a computer program, the method comprising:

providing, by downloading to a remote computing device, via the processor a speech grammar development tool executable by the remote computing device to

receive input of a text representation of each of a plurality of proposed speech grammar terms,

for each proposed speech grammar term, convert the text representation to a phonetic representation of the speech grammar term,

determine whether a portion of the proposed speech grammar term has a spoken duration below a threshold duration, and if the portion of the proposed speech grammar term has a spoken duration below the threshold duration, then omit the portion from the phonetic representation of the proposed speech grammar term,

compare via a speech recognition engine the phonetic representation of the speech grammar term to the phonetic representations of other speech grammar terms using a weighted similarity matrix, and

provide an output regarding risk of confusion between two proposed speech grammar terms based upon a comparison by the speech recognition engine of the phonetic representations of the two proposed speech grammar terms;

receiving via the processor data regarding incorrect speech grammar term identification; and

modifying via the processor speech grammar used by the speech recognition engine, wherein modifying the speech grammar comprises modifying one or more weights in the weighted similarity matrix based upon the data.

2. The method of claim 1, wherein the weighted similarity matrix comprises, for each phoneme of a plurality of phonemes, a weight for each phonetic feature of a plurality of phonetic features.

3. The method of claim 2, wherein modifying the one or more weights in the weighted similarity matrix comprises manually adjusting the one or more weights based upon the data.

4. The method of claim 2, wherein modifying the one or more weights in the weighted similarity matrix comprises algorithmically adjusting the one or more weights based upon the data.

5. The method of claim 1, wherein determining that the portion of the selected speech grammar term has a spoken duration below a threshold duration comprises force aligning one or more audio samples of the selected speech grammar term to the text representation of the selected speech grammar term and determining that a time stamp of the portion has a value below the threshold duration.

6. The method of claim 5, further comprising receiving the one or more end user audio samples.

7. The method of claim 1, wherein providing the output comprises providing one of three or more possible outputs regarding a relative risk of confusion between the two proposed speech grammar terms.

8. The method of claim 1, wherein the data regarding incorrect speech grammar term identification comprises data regarding an actual pronunciation for a selected speech grammar term that is different than the expected pronunciation of the selected speech grammar term;

and

further comprising storing a phonetic representation of the actual pronunciation of the selected speech grammar term.

9. On a computing device comprising a processor, a method of detecting phrase confusion risk in proposed speech grammar for a computer program, the method comprising:

obtaining via the processor a speech grammar development tool by downloading the speech grammar development tool from a server computing system;

receiving via the processor an input of a text representation of a speech grammar term;

determining via the processor a phonetic representation of the speech grammar term from the text representation;

if a portion of the speech grammar term comprises a duration below a threshold duration, then omitting via the processor the portion of the phonetic representation of the speech grammar term;

comparing via a speech recognition engine executed by the processor the phonetic representation of the speech grammar term to a phonetic representation of another speech grammar term, wherein comparing comprises using a weighted similarity matrix;

outputting to a display a representation of a risk of confusion between the speech grammar term and the other speech grammar term based upon a comparison by the speech recognition engine of the speech grammar term and the other speech grammar term;

determining that the speech grammar term was incorrectly identified;

sending data regarding the incorrect determination to the server computing system;

receiving a modified weighted similarity matrix from the server computing system; and

modifying via the processor the speech grammar used by the speech recognition engine based upon the modified weighted similarity matrix.

10. The method of claim 9, further comprising determining that the portion of the selected speech grammar term has a spoken duration below a threshold duration by force aligning one or more audio samples of the selected speech grammar term to the text representation of the selected speech grammar term and determining that a time stamp of the portion has a value lower than the threshold duration.

11. The method of claim 10, further comprising receiving the one or more audio samples from end users via telemetry.

12. The method of claim 9, wherein comparing the phonetic representation of the speech grammar term to a phonetic representation of another speech grammar term comprises using a weighted similarity matrix.

13. The method of claim 12, further comprising receiving data regarding incorrect speech grammar term identification, and modifying one or more weights in the weighted similarity matrix based upon the data.

14. The method of claim 9, wherein outputting the representation of a risk of confusion comprises providing one of three or more possible outputs to indicate a relative risk of confusion between the two proposed speech grammar terms.

15. A computing system, comprising:

a processor; and

a memory storing instructions executable by the processor to;

provide, by downloading to a remote computing device, a speech grammar development tool configured to

receive input of a text representation of each of a plurality of proposed speech grammar terms;

for each proposed speech grammar term, convert the text representation to a phonetic representation of the speech grammar term,

determine whether the proposed speech grammar term comprises a portion that has a spoken duration below a threshold duration, and when the portion of the proposed speech grammar term has a spoken duration below the threshold duration, then omit the portion from the phonetic representation of the proposed speech grammar term,

and compare via a speech recognition engine the phonetic representation of the speech grammar term to the phonetic representations of other speech grammar terms using a weighted similarity matrix, the weighted similarity matrix comprising, for each phoneme of a plurality of phonemes, a weight for each phonetic feature of a plurality of phonetic features, and

provide an output regarding risk of confusion between two proposed speech grammar terms based upon a comparison via a speech recognition engine of the phonetic representations of the two proposed speech grammar terms;

receive data regarding one or more incorrect speech grammar term identifications, the data comprising an actual pronunciation for a selected speech grammar term that is different from the expected pronunciation of the selected speech grammar term;

store a phonetic representation of the actual pronunciation of the selected speech grammar term; and

modify speech grammar to be used by the speech recognition engine, wherein modifying the speech grammar comprises modifying one or more weights in the weighted similarity matrix based upon the data.

16. The computing system of claim 15, wherein the instructions are executable to determine that the portion of the selected speech grammar term has a spoken duration below a threshold duration by force aligning one or more audio samples of the selected speech grammar term to the text representation of the selected speech grammar term and determining that a time stamp of the portion has a value lower than the threshold duration.

17. The computing system of claim 16, wherein the instructions are executable to receive the one or more audio samples from end users.

18. The computing system of claim 15, wherein the instructions are executable to output three or more possible outputs to indicate of a relative risk of confusion between the two proposed speech grammar terms.