Method and system for recognizing speech using wildcards in an expected response

• US 10,269,342 B2
• Filed: 10/29/2014
• Issued: 04/23/2019
• Est. Priority Date: 10/29/2014
• Status: Active Grant

• Current Assignee: Hand Held Products Incorporated (Honeywell International Inc.)
• Original Assignee: Hand Held Products Incorporated (Honeywell International Inc.)
• Inventors: Braho, Keith, Makay, Jason M
• Primary Examiner(s): Jackson, Jakieda R
• Application Number:
  US14/527,191
• Publication Number:
  US 20160125873A1
• Time in Patent Office: 1,637 Days
• Field of Search: 704239, 704240, 704270
• CPC Class Codes: G10L 15/065 Adaptation
• CPC Class Codes: G10L 15/07 to the speaker
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 25/51 for comparison or discrimination

Abstract

A speech recognition system used in a workflow receives and analyzes speech input to recognize and accept a user'"'"'s response to a task. Under certain conditions, a user'"'"'s response might be expected. In these situations, the expected response may modify the behavior of the speech recognition system to improve recognition accuracy. For example, if the hypothesis of a user'"'"'s response matches the expected response then there is a high probability that the user'"'"'s response was recognized correctly. An expected response may include expected words and wildcard words. Wildcard words represent any recognized word in a user'"'"'s response. By including wildcard words in the expected response, the speech recognition system may make modifications based on a wide range of user responses.

Claims

1. A method for recognizing speech in a speech recognition system, the method comprising the steps of:

sensing, using a microphone, speech input and converting the sensed speech input to an electrical signal;

converting, using a signal processor comprising an analog-to-digital converter, the electrical signal to digital data;

receiving, using a computing device, the digital data, the computing device having at least one processor and a memory;

processing the digital data, using the processor, to produce acoustic features and acoustic data;

processing the acoustic features and the acoustic data using the processor and a library of models corresponding to hypothesis words and stored on the memory, to derive a hypothesis, the hypothesis comprising a sequence of hypothesis words;

assigning each hypothesis word a confidence score;

retrieving from the memory an expected response comprising a sequence of at least one expected word and at least one wildcard word;

comparing the hypothesis word-by-word to the expected response;

adjusting an acceptance threshold for each hypothesis word based on the results of the comparison;

comparing the confidence score assigned to a hypothesis word to its adjusted acceptance threshold and accepting or rejecting the hypothesis word based on the results of the comparison; and

if the hypothesis word is accepted, updating acoustic features and acoustic data of a model, in the library of models, corresponding to the hypothesis word using the acoustic features and acoustic data corresponding to the hypothesis word.

2. The method of claim 1, wherein one confidence score is assigned to more than one hypothesis words.

3. The method according to claim 1, comprising adjusting the acceptance threshold for a hypothesis word by an exact-match adjustment amount if the hypothesis word matches a corresponding expected word in the expected response.

4. The method according to claim 1, comprising adjusting the acceptance threshold for a hypothesis word by a wildcard-match adjustment amount if the hypothesis word corresponds to a wildcard word in the expected response.

5. The method according to claim 1, wherein the acceptance thresholds for hypothesis words corresponding to wildcard words in the expected response are adjusted differently from acceptance thresholds for hypothesis words corresponding to expected words in the expected response.

6. The method according to claim 1, comprising not adjusting the acceptance threshold for any hypothesis words in the hypothesis if a hypothesis word in the hypothesis does not match its corresponding expected word in the expected response.

7. The method according to claim 1, comprising comparing the confidence score assigned to each hypothesis word in the hypothesis to its acceptance threshold and accepting or rejecting the hypothesis word based on the results of the comparison.

8. The method according to claim 7, comprising accepting the hypothesis word if its confidence score exceeds its acceptance threshold or rejecting the hypothesis word otherwise.

9. A method for recognizing speech in a speech recognition system, the method comprising the steps of:

sensing, using a microphone, speech input and converting the sensed speech input to an electrical signal;

converting, using a signal processor comprising an analog-to-digital converter, the electrical signal to digital data;

receiving, using a computing device, the digital data, the computing device having at least one processor and a memory;

processing the digital data, using the processor, to produce acoustic features and acoustic data;

deriving a hypothesis, using the processor running speech recognition algorithms, the acoustic features, the acoustic data, and a library of models corresponding to hypothesis words stored in the memory, the hypothesis comprising a sequence of hypothesis words;

retrieving from the memory an expected response comprising a sequence of at least one expected word and at least one wildcard word;

comparing, in sequence, each hypothesis word in the hypothesis to its corresponding expected word or wildcard word in the expected response;

if the hypothesis word matches the corresponding expected word, then marking the hypothesis word as suitable for use in adaptation;

adapting acoustic features and acoustic data of the models corresponding to hypothesis words marked suitable for adaptation using the acoustic features and the acoustic data corresponding to those hypothesis words;

deriving a hypothesis word, using the processor running speech recognition algorithms and an adapted model of the adapted models stored in the memory; and

comparing a confidence score assigned to the hypothesis word derived from the adapted model to an acceptance threshold and accepting or rejecting the hypothesis word based on the results of the comparison.

10. The method according to claim 9, comprising marking a hypothesis word as not suitable for adaptation if the hypothesis word corresponds to a wildcard word in the expected response, and not using the acoustic data corresponding to the hypothesis words marked as not suitable for adaptation to adapt the models corresponding to those hypothesis words.

11. The method according to claim 9, comprising marking all words in the hypothesis as not suitable for adaptation if a hypothesis word in the hypothesis does not match its corresponding expected word in the expected response, and not adapting the models corresponding to hypothesis words marked not suitable for adaptation.

12. A system for recognizing speech, comprising:

a microphone configured to sense speech input and convert the sensed speech input to an electrical signal;

a signal processor configured to convert the electrical signal to digital data, the signal processor comprising an analog-to-digital converter;

a computing device comprising a processor and a memory configured to execute (i) a recognition algorithm, (ii) a threshold-adjustment algorithm, and (iii) an acceptance algorithm, wherein;

the recognition algorithm processes the digital data to produce acoustic features and acoustic data and assesses the acoustic features and the acoustic data using a library of models corresponding to hypothesis words stored in the memory to generate (i) a hypothesis comprising hypothesis words and (ii) a confidence score associated with one or more hypothesis words;

the threshold-adjustment algorithm adjusts an acceptance threshold corresponding to a hypothesis word if the hypothesis matches an expected response stored in the memory, wherein the expected response comprises at least one expected word and at least one wildcard word; and

the acceptance algorithm accepts a hypothesis word and updates acoustic features and acoustic data of a model, in the library of models, corresponding to the hypothesis word when the hypothesis word'"'"'s confidence score exceeds the hypothesis word'"'"'s acceptance threshold.

13. The system according to claim 12, wherein the threshold-adjustment algorithm comprises (i) reducing the acceptance threshold for hypothesis words that match corresponding expected words by an exact-match adjustment amount and (ii) reducing the acceptance threshold for hypothesis words that match corresponding wildcard words by a wildcard-match adjustment amount.

14. The system according to claim 13, wherein the exact-match adjustment amount is greater than the wildcard-match adjustment amount.

15. The system according to claim 12, wherein no hypothesis-word acceptance thresholds are adjusted if at least one hypothesis word does not match its corresponding expected word in the expected response.

16. The system according to claim 15, wherein the threshold-adjustment algorithm comprises reducing the acceptance threshold for hypothesis words that match corresponding wildcard words by an amount that depends on the matching condition between other hypothesis words and expected words.

17. A system for recognizing speech, comprising:

a microphone configured to sense speech input and convert the sensed speech input to an electrical signal;

a signal processor configured to convert the electrical signal to digital data, the signal processor comprising an analog-to-digital converter;

a computing device comprising a processor and a memory configured to execute (i) a recognition algorithm, (ii) a model-update algorithm, and (iii) an acceptance algorithm, wherein;

the recognition algorithm processes the digital data to produce acoustic features and acoustic data and assesses the acoustic features and the acoustic data using a library of models corresponding to hypothesis words stored in the memory to generate a hypothesis comprising hypothesis words;

the model-update algorithm (i) compares the sequence of words of the hypothesis to an expected response stored in the memory, the expected response comprising expected words and at least one wildcard word, (ii) marks each hypothesis word that matches a corresponding expected word in the expected response as suitable for adaptation, and (iii) adapts acoustic features and acoustic data of a model for a hypothesis word marked suitable for adaptation using the acoustic features and the acoustic data corresponding to that hypothesis word; and

the acceptance algorithm accepts a hypothesis word when the hypothesis word'"'"'s confidence score exceeds the hypothesis word'"'"'s acceptance threshold.

18. The system according to claim 17, wherein the model-update algorithm does not use the acoustic data corresponding to hypothesis words marked not suitable for adaptation to adapt the models, and the hypothesis words marked not suitable for adaptation comprise hypothesis words that correspond to wildcard words in the expected response.

19. The system according to claim 17, wherein the model-update algorithm does not use the acoustic data corresponding to hypothesis words marked not suitable for adaptation to adapt the models, and the hypothesis words marked not suitable for adaptation comprise hypothesis words that do not match corresponding expected words in the expected response.

20. The system according to claim 17, wherein the model-update algorithm does not use the acoustic data corresponding to hypothesis words marked not suitable for adaptation to adapt the library of models, and all hypothesis words are marked as not suitable for adaptation if at least one hypothesis word does not match its corresponding expected word in the expected response.