Apparatus and method to classify sound to detect speech

• US 9,076,459 B2
• Filed: 03/12/2013
• Issued: 07/07/2015
• Est. Priority Date: 03/12/2013
• Status: Active Grant

• Current Assignee: Intermec IP Corporation (Honeywell International Inc.)
• Original Assignee: Intermec IP Corporation (Honeywell International Inc.)
• Inventors: Braho, Keith P., Hardek, David D.
• Primary Examiner(s): Baker, Charlotte M
• Application Number:
  US13/797,250
• Publication Number:
  US 20140278391A1
• Time in Patent Office: 847 Days
• Field of Search: 704/226, 704/233, 375/240.01, 375/E7.076, 379/390.01, 379/392.01, 379/395
• US Class Current: 1/1
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10

Abstract

Audio frames are classified as either speech, non-transient background noise, or transient noise events. Probabilities of speech or transient noise event, or other metrics may be calculated to indicate confidence in classification. Frames classified as speech or noise events are not used in updating models (e.g., spectral subtraction noise estimates, silence model, background energy estimates, signal-to-noise ratio) of non-transient background noise. Frame classification affects acceptance/rejection of recognition hypothesis. Classifications and other audio related information may be determined by circuitry in a headset, and sent (e.g., wirelessly) to a separate processor-based recognition device.

Claims

1. A method of operation in a speech recognition system, the method comprising:

analyzing each of a plurality of frames of audio by a sound classifier;

classifying a first number of the frames of audio as speech by the sound classifier;

classifying a second number of the frames of audio as non-transient background noise by the sound classifier;

classifying a third number of the frames of audio as transient noise events by the sound classifier; and

providing signals indicative at least of the classifications of the frames of audio to a speech recognizer.

2. The method of operation of claim 1, further comprising:

providing input to the sound classifier from at least two microphones.

3. The method of operation of claim 1 wherein the sound classifier is part of a headset with at least two microphones, the speech recognizer is implemented by a processor-based device separate from the headset, and providing the signals indicative at least of the classifications of the frames of audio to a speech recognizer includes wirelessly providing the signals to the processor-based device that implements the speech recognizer.

4. The method of operation of claim 1 wherein providing the signals indicative at least of the classifications of the frames of audio to a speech recognizer includes wirelessly providing the signals with at least a logical relationship to respective data that represents audio of at least some of the frames of audio.

5. The method of operation of claim 4 wherein the respective data that represents audio of at least some of the frames of audio includes at least one of autocorrelation coefficients or digitized audio fragments.

6. The method of operation of claim 4 wherein classifying a second number of the frames of audio as non-transient background noise by the sound classifier includes, for each frame of audio determining a metric for the respective frame, comparing the determined metric for the respective frame to an average metric for a plurality of frames of audio, classifying the respective frame as a transient noise if the determined metric for the respective frame exceeds the average metric for the plurality of frames of audio by at least a first threshold, and otherwise classifying the respective frame as a non-transient background noise.

7. The method of operation of claim 1, further comprising:

employing the classifications in detecting at least one of a start or a stop of speech by a speech detector of the speech recognizer based at least in part on the received classifications of each of the plurality of frames of audio.

8. The method of operation of claim 7 wherein employing the classifications in detecting at least one of a start or a stop of speech by the speech detector of the speech recognizer based at least in part on the received classifications of each of the plurality of frames of audio includes, for each of a set of two or more fragments, determining how many of the fragments in the set are classified as a first one of the classifications, and treating the entire set as either speech or non-speech based on the how many of the fragments in the set are classified as the first one of the classifications.

9. The method of operation of claim 8 wherein determining how many of the fragments in the set are classified as a first one of the classifications includes determining how many of the fragments in the set are classified as speech.

10. The method of operation of claim 8 wherein employing the classifications in detecting a start of speech by a speech detector of the speech recognizer is based at least in part on the received classifications of each of the plurality of frames of audio further includes identifying a set of fragments in which the number of fragments individually classified as speech exceeds a threshold as constituting speech.

11. The method of operation of claim 10, further comprising:

for sets of fragments identified as speech at least one of prepending or postpending additional fragments of audio to the respective set for processing, where the additional fragments of audio occurred immediately before or immediately after the audio fragments of the respective set of fragments.

12. The method of operation of claim 8 wherein employing the classifications in detecting a stop of speech by a speech detector of the speech recognizer is based at least in part on the received classifications of each of the plurality of frames of audio further includes identifying a set of fragments in which the number of fragments individually classified as speech is below a threshold as not constituting speech.

13. The method of operation of claim 10, further comprising:

for a set of fragments identified as constituting speech, at least one of prepending or postpending additional fragments of audio to the set for processing, where the additional fragments of audio occurred immediately before or immediately after the audio fragments of the set of fragments as constituting speech.

14. The method of operation of claim 1, further comprising:

generating features based at least in part on the received classifications of the frames of audio.

15. The method of operation of claim 14 wherein generating features based at least in part on the received classifications of the frames of audio includes generating features based on fragments classified as speech and not on fragments classified as either non-transient background noise or transient noise events.

16. The method of operation of claim 1, further comprising:

updating a background noise estimate based on non-transient background noise and not based on transient noise events or speech events.

17. The method of operation of claim 1, further comprising:

determining confidence values in the classification of the frames by the sound classifier based at least in part on distinguishing between speech and non-transient background noise and transient noise events.

18. The method of operation of claim 1, further comprising:

adjusting a threshold at which a recognized hypothesis based on the audio is either rejected or accepted based at least in part on distinguishing among speech events, non-transient background noise and transient noise events.

19. The method of operation of claim 1, further comprising:

adjusting a confidence value of a hypothesis or portion thereof based at least in part on distinguishing among speech events, non-transient background noise and transient noise events.

20. A method of operation in a speech recognition system, the method comprising:

analyzing each of a plurality of frames of audio by a sound classifier;

classifying a first number of the frames of audio as speech by the sound classifier;

classifying a second number of the frames of audio as non-speech;

forming a hypothesis based on the audio;

adjusting a threshold at which a recognized hypothesis is either rejected or accepted based at least in part on the first and the second numbers of frames.

21. The method of operation of claim 20, further comprising:

providing input to the sound classifier from at least two microphones.

22. A method of operation in a speech recognition system, the method comprising:

analyzing a first segment of audio by a sound classifier;

determining at least two confidences among the following three confidences;

a first confidence that the first segment of audio is speech;

a second confidence that the first segment of audio is non-transient background noise;

or

a third confidence that the first segment of audio is transient background noise;

generating a hypothesis for a second segment of audio that includes the first segment of audio; and

adjusting a threshold at which the hypothesis is either rejected or accepted based at least in part on the at least two confidences.

23. The method of operation of claim 22, further comprising:

providing input to the sound classifier from at least two microphones.

24. A speech recognition system, comprising:

a sound classifier that includes at least one non-transitory processor-readable medium and at least one processor communicatively coupled to the at least one non-transitory processor-readable medium, and that analyzes each of a plurality of frames of audio, classifies a first number of the frames of audio as speech by the sound classifier, classifies a second number of the frames of audio as non-transient background noise by the sound classifier, classifies a third number of the frames of audio as transient noise events by the sound classifier, and provides signals indicative at least of the classifications of the frames of audio.

25. The speech recognition system of claim 24, further comprising:

providing input to the sound classifier from at least two microphones.

26. The speech recognition system of claim 24, further comprising:

a speech recognizer communicatively coupled to receive the signals from the sound classifier and operable to distinguish sets of fragments containing speech from sets of fragments not containing speech based at least in part on the classifications indicated in the signals received from the sound classifier.

27. The speech recognition system of claim 26 wherein the sound classifier is part of a headset with at least two microphones, the speech recognizer is implemented by a processor-based device separate from the head set, and the sound classifier wirelessly provides the signals to the processor-based device that implements the speech recognizer.

28. The speech recognition system of claim 26 wherein the sound classifier wirelessly provides the signals with at least a logical relationship to respective data that represents audio of at least some of the frames of audio.

29. The speech recognition system of claim 28 wherein the respective data that represents audio of at least some of the frames of audio includes at least one of autocorrelation coefficients or digitized audio fragments.

30. The speech recognition system of claim 28 wherein, classifying a second number of the frames of audio as non-transient background noise includes, for each frame of audio not classified as speech, the sound classifier determines a metric for the respective frame, compares the determined metric for the respective frame to an average metric for a plurality of frames of audio, classifies the respective frame as a transient noise if the determined metric for the respective frame exceeds the average metric for the plurality of frames of audio by at least a first threshold, and otherwise classifies the respective frame as non-transient background noise.

31. The speech recognition system of claim 26 wherein a speech detector of the speech recognizer employs the classifications to detect at least one of a start or a stop of speech by based at least in part on the received classifications of each of the plurality of frames of audio.

32. The speech recognition system of claim 31 wherein for each of a set of two or more fragments, the speech detector determines how many of the fragments in the set are classified as a first one of the classifications, and treats the entire set as either speech or non-speech based on the how many of the fragments in the set are classified as the first one of the classifications in order to employ the classifications in the detection at least one of the start or the stop of speech.

33. The speech recognition system of claim 32 wherein the speech detector determines how many of the fragments in the set are classified as speech.

34. The speech recognition system of claim 32 wherein the speech detector a) identifies a first set of fragments in which the number of fragments individually classified as speech exceeds a first threshold as constituting speech and b) identifies a second set of fragments in which the number of fragments individually classified as speech is below a second threshold as not constituting speech.

35. The speech recognition system of claim 34 wherein, for sets of fragments identified as speech, at least one component of the voice recognition system at least one of prepends or postpends additional fragments of audio to the respective set for processing, where the additional fragments of audio occurred immediately before or immediately after the audio fragments of the respective set of fragments.

36. The speech recognition system of claim 24 wherein at least one component of the voice recognition system generates features based at least in part on the received classifications of the frames of audio.

37. The speech recognition system of claim 36 wherein the at least one component of the voice recognition system generates features based on fragments classified as speech and not on fragments classified as either non-transient background noise or transient noise events.

38. The speech recognition system of claim 24 wherein at least one component of the voice recognition system updates a background noise estimate based on non-transient background noise and not based on transient noise events or speech events.

39. The speech recognition system of claim 24 wherein at least one component of the voice recognition system determines confidence values in the classification of the frames as speech and non-transient background noise and transient noise events by the sound classifier based at least in part on how far above or below a threshold a metric for the frame is, where the threshold is based on a baseline for the respective metric.

40. The speech recognition system of claim 24 wherein the threshold adjust module adjusts a threshold at which a recognized hypothesis based on the audio is either rejected or accepted as speech based at least in part on distinguishing among speech events and non-transient background noise and transient noise events.

41. The speech recognition system of claim 24 wherein the threshold adjust module adjusts a confidence value of a hypothesis or portion thereof based at least in part on distinguishing among speech events, non-transient background noise and transient noise events.

42. A speech recognition system, comprising:

a sound classifier that includes at least one non-transitory processor-readable medium and at least one processor communicatively coupled to the at least one non-transitory processor-readable medium, and that

analyzes each of a plurality of frames of audio,

classifies a first number of the frames of audio as speech,

classifies a second number of the frames of audio as non-speech,

forms a hypothesis based on the audio; and

adjusts a threshold at which a recognized hypothesis is either rejected or accepted based at least in part on the first and the second numbers of frames.

43. The speech recognition system of claim 42 wherein,

the sound classifier receives input from at least two microphones.

44. A speech recognition system, comprising:

a sound classifier that includes at least one non-transitory processor-readable medium and at least one processor communicatively coupled to the at least one non-transitory processor-readable medium, and that

analyzes a first segment of audio,

determines at least two confidences among the following three confidences;

a first confidence that the first segment of audio is speech;

a second confidence that the first segment of audio is non-transient background noise;

or

a third confidence that the first segment of audio is transient background noise;

generates a hypothesis for a second segment of audio that includes the first segment of audio; and

adjusts a threshold at which the hypothesis is either rejected or accepted based at least in part on the at least two confidences.

45. The speech recognition system of claim 44 wherein,

the sound classifier receives input from at least two microphones.