Speech end-pointer

• US 8,165,880 B2
• Filed: 05/18/2007
• Issued: 04/24/2012
• Est. Priority Date: 06/15/2005
• Status: Active Grant

• Current Assignee: Blackberry Limited
• Original Assignee: QNX Software Systems Limited (Canada) (Blackberry Limited)
• Inventors: Hetherington, Phillip A., Fallat, Mark
• Primary Examiner(s): Smits, Talivaldis Ivars
• Assistant Examiner(s): PULLIAS, JESSE SCOTT
• Application Number:
  US11/804,633
• Publication Number:
  US 20070288238A1
• Time in Patent Office: 1,803 Days
• Field of Search: 704/253, 704/248
• US Class Current: 704/253
• CPC Class Codes: G10L 25/87 Detection of discrete points within a voice signal

Abstract

An end-pointer determines a beginning and an end of a speech segment. The end-pointer includes a voice triggering module that identifies a portion of an audio stream that has an audio speech segment. A rule module communicates with the voice triggering module. The rule module includes a plurality of rules used to analyze a part of the audio stream to detect a beginning and an end of the audio speech segment. A consonant detector detects occurrences of a high frequency consonant in the portion of the audio stream.

Claims

1. An end-pointer that determines a beginning and an end of a speech segment comprising:

a voice triggering module that identifies a portion of an audio stream comprising an audio speech segment;

a rule module in communication with the voice triggering module, the rule module comprising a plurality of rules used by a processor to analyze a part of the audio stream to detect a beginning and an end of the audio speech segment; and

a consonant detector that calculates a difference between a signal-to-noise ratio in a high frequency band and a signal-to-noise ratio in a low frequency band, where the consonant detector converts the difference between the signal-to-noise ratio in the high frequency band and the signal-to-noise ratio in the low frequency band into a probability value that predicts a likelihood of a high frequency consonant in the portion of the audio stream;

where the beginning of the audio speech segment and the end of the audio speech segment represent boundaries between speech and non-speech portions of the audio stream, and where the rule module identifies the beginning of the audio speech segment or the end of the audio speech segment based on an output of the consonant detector.

2. The end-pointer of claim 1, where the voice triggering module identifies a vowel.

3. The end-pointer of claim 1, where the consonant detector comprises an /s/ detector.

4. The end-pointer of claim 1, where the portion of the audio stream comprises a frame.

5. The end-pointer of claim 1, where the rule module analyzes an energy level in the portion of the audio stream.

6. The end-pointer of claim 1, where the rule module analyzes an elapsed time in the portion of the audio stream.

7. The end-pointer of claim 1, where the rule module analyzes a predetermined number of plosives in the portion of the audio stream.

8. The end-pointer of claim 1, where the rule module identifies the beginning of the audio speech segment or the end of the audio speech segment based on the probability value that predicts the likelihood of the high frequency consonant in the portion of the audio stream.

9. The end-pointer of claim 1, further comprising an energy detector.

10. The end-pointer of claim 1, further comprising a controller in communication with a memory, where the rule module resides within the memory.

11. The end-pointer of claim 1, where the probability value indicates a likelihood that a consonant exists in a frame of the audio stream, where the consonant detector compares the probability value to consonant likelihood values associated with previous frames and identifies a consonant based on detection of an increasing trend.

12. The end-pointer of claim 1, where the probability value indicates a likelihood that a consonant exists in a frame of the audio stream, where the consonant detector compares the probability value to consonant likelihood values associated with previous frames and sets an endpoint based on detection of a decreasing trend.

13. The end-pointer of claim 1, where the probability value comprises a current probability value associated with a current frame of the audio stream, where the consonant detector modifies the current probability value when the current probability value deviates from consonant probability values associated with previous frames.

14. The end-pointer of claim 13, where the consonant detector adds to the current probability value a temporally smoothed difference between the current probability value and a probability value associated with a previous frame, upon determination that the current probability value exceeds the probability value associated with the previous frame, where the consonant detector generates the temporally smoothed difference by multiplying a smoothing factor with the difference between the current probability value and the probability value associated with the previous frame;

where the consonant detector adds to the current probability value a portion of the difference between the current probability value and the probability value associated with the previous frame, upon determination that the current probability value is less than the probability value associated with the previous frame, where the consonant detector generates the portion of the difference by multiplying the difference by a percentage; and

where the smoothing factor is different than the percentage.

15. The end-pointer of claim 1, where the consonant detector comprises a non-transitory computer-readable medium or circuit.

16. A method that identifies a beginning and an end of a speech segment using an end-pointer comprising:

receiving a portion of an audio stream;

determining whether the portion of the audio stream includes a triggering characteristic;

calculating a difference between a signal-to-noise ratio in a high frequency band of the portion of the audio stream and a signal-to-noise ratio in a low frequency band of the portion of the audio stream;

converting, by a consonant detector implemented in hardware or embodied in a computer-readable storage medium, the difference between the signal-to-noise ratio in the high frequency band and the signal-to-noise ratio in the low frequency band into a probability value that predicts a likelihood of a high frequency consonant in the portion of the audio stream; and

applying a rule that passes only a portion of the audio stream to a device when the triggering characteristic identifies a beginning of a voiced segment and an end of a voiced segment;

where the identification of the end of the voiced segment is based on an output of the consonant detector, where the end of the voiced segment represents a boundary between speech and non-speech portions of the audio stream.

17. The method of claim 16, where the rule identifies the portion of the audio stream to be sent to the device.

18. The method of claim 16, where the rule is applied to a portion of the audio stream that does not include the triggering characteristic.

19. The method of claim 16, where the triggering characteristic comprises a vowel.

20. The method of claim 16, where the triggering characteristic comprises an /s/ or an /x/.

21. The method of claim 16, further comprising raising a voice threshold in response to a detection of a high frequency consonant.

22. The method of claim 21, where the voice threshold is raised across a plurality of audio frames.

23. The method of claim 16, where the rule module analyzes an energy in the portion of the audio stream.

24. The method of claim 16, where the rule module analyzes an elapsed time in the portion of the audio stream.

25. The method of claim 16, where the rule module analyzes a predetermined number of plosives in the portion of the audio stream.

26. The method of claim 16, further comprising marking the beginning and the end of a potential speech segment.

27. A system that identifies a beginning and an end of a speech segment comprising:

an end-pointer comprising a processor that analyzes a dynamic aspect of an audio stream to determine the beginning and the end of the speech segment; and

a high frequency consonant detector that marks the end of the speech segment, where the high frequency consonant detector calculates a difference between a signal-to-noise ratio in a high frequency band of the audio stream and a signal-to-noise ratio in a low frequency band of the audio stream, and where the high frequency consonant detector converts the difference between the signal-to-noise ratio in the high frequency band and the signal-to-noise ratio in the low frequency band into a probability value that predicts a likelihood that a high frequency consonant exists in a frame of the audio stream;

where the beginning of the speech segment and the end of the speech segment represent boundaries between speech and non-speech portions of the audio stream, and where the end-pointer identifies the beginning of the audio speech segment or the end of the audio speech segment based on an output of the high frequency consonant detector.

28. The system of claim 27, where the dynamic aspect of the audio stream comprises a characteristic of a speaker.

29. The end system of claim 28, where the characteristic of the speaker comprises a rate of speech.

30. The system of claim 27, where the dynamic aspect of the audio stream comprises a level of background noise in the audio stream.

31. The system of claim 27, where the dynamic aspect of the audio stream comprises an expected sound in the audio stream.

32. The system of claim 31, where the expected sound comprises an expected answer to a question.

33. The system of claim 27, where the high frequency consonant detector comprises a non-transitory computer-readable medium or circuit.

34. A system that determines a beginning and an end of an audio speech segment in an audio stream, comprising:

an /s/ detector that converts a difference between a signal-to-noise ratio in a high frequency band of the audio stream and a signal-to-noise ratio in a low frequency band of the audio stream into a probability value that predicts a likelihood of an /s/ sound in the audio stream; and

an end-pointer comprising a processor that varies an amount of an audio input sent to a recognition device based on a plurality of rules and an output of the /s/ detector;

where the end-pointer identifies a beginning of the audio input or an end of the audio input based on the output of the /s/ detector, and where the beginning of the audio input and the end of the audio input represent boundaries between speech and non-speech portions of the audio stream.

35. The end system of claim 34, where the recognition device comprises an automatic speech recognition device, and where the end-pointer adapts an endpoint of the audio input based on the output of the /s/ detector.

36. A non-transitory computer readable medium that stores software that determines at least one of a beginning and end of an audio speech segment comprising:

a detector that converts sound waves into operational signals;

a triggering logic that analyzes a periodicity of the operational signals;

a signal analysis logic that analyzes a variable portion of the sound waves that are associated with the audio speech segment to determine a beginning and end of the audio speech segment, and

a consonant detector that calculates a difference between a signal-to-noise ratio in a high frequency band and a signal-to-noise ratio in a low frequency band, where the consonant detector converts the difference between the signal-to-noise ratio in the high frequency band and the signal-to-noise ratio in the low frequency band into a probability value that predicts a likelihood of an /s/ sound in the sound waves, where the consonant detector provides an input to the signal analysis logic when the /s/ is detected;

where the beginning of the audio speech segment and the end of the audio speech segment represent boundaries between speech and non-speech portions of the sound waves, and where the signal analysis module identifies the beginning of the audio speech segment or the end of the audio speech segment based on an output of the consonant detector.

37. The non-transitory computer readable medium of claim 36, where the signal analysis logic analyzes a time duration before a voiced speech sound.

38. The non-transitory computer readable medium of claim 36, where the signal analysis logic analyzes a time duration after a voiced speech sound.

39. The non-transitory computer readable medium of claim 36, where the signal analysis logic analyzes a number of transitions before or after a voiced speech sound.

40. The non-transitory computer readable medium of claim 36, where the signal analysis logic analyzes a duration of continuous silence before a voiced speech sound.

41. The non-transitory computer readable medium of claim 36, where the signal analysis logic analyzes a duration of continuous silence after a voiced speech sound.

42. The non-transitory computer readable medium of claim 36, where the signal analysis logic is coupled to a vehicle.

43. The non-transitory computer readable medium of claim 36, where the signal analysis logic is coupled to an audio system.