Speech signal pre-processing system and method of extracting characteristic information of speech signal

• US 20070288236A1
• Filed: 03/27/2007
• Published: 12/13/2007
• Est. Priority Date: 04/05/2006
• Status: Abandoned Application

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Hyun-Soo
• Application Number:
  US11/728,715
• Publication Number:
  US 20070288236A1
• Time in Patent Office: Days
• US Class Current: 704/231
• CPC Class Codes: G10L 25/30 using neural networks
• CPC Class Codes: G10L 25/93 Discriminating between voiced and unvoiced parts of speech signals G10L25/90 takes precedence

Abstract

A speech signal pre-processing system and a method of extracting characteristic information of a speech signal. To do this, it is determined whether characteristic information of an input speech signal is extracted using harmonic peaks. According to the determination result, a speech signal frame or characteristic frequency regions derived according to a morphological analysis result is (are) input to a speech signal characteristic information extractor for extracting speech signal characteristic information requested by a speech signal processing system in a next stage. The speech signal characteristic information extractor selected by a controller receives the speech signal frame or the characteristic frequency regions derived according to a morphological analysis result and extracts the speech signal characteristic information requested by the speech signal processing system.

Claims

1. A speech signal pre-processing system comprising:

a speech signal recognition unit for recognizing speech from an input signal and outputting the input signal as a speech signal;

a speech signal converter for generating a speech signal frame by receiving the speech signal and converting the received speech signal of a time domain to a speech signal of a frequency domain;

a morphological analyzer for receiving the speech signal frame and generating characteristic frequency regions having a morphological analysis-based signal waveform through a morphological operation;

a speech signal characteristic information extractor for receiving the speech signal frame or the morphological analysis-based characteristic frequency regions and extracting speech signal characteristic information requested by a speech signal processing system in a next stage; and

a controller for determining according to a pre-set determination condition whether the characteristic information of the speech signal is extracted using harmonic peaks of the speech signal frame, and extracting the speech signal characteristic information requested by the speech signal processing system by outputting the speech signal frame to the speech signal characteristic information extractor when harmonic peaks are used or outputting the morphological analysis-based characteristic frequency regions of the speech signal frame when harmonic peaks are not used.

2. The speech signal pre-processing system of claim 1, wherein the pre-set determination condition is a characteristic of the input signal or the speech signal processing system.

3. The speech signal pre-processing system of claim 1, further comprising a harmonic peak extractor for searching for and extracting harmonic peaks from the speech signal frame.

4. The speech signal pre-processing system of claim 1, further comprising a noise canceller for canceling noise from the speech signal frame.

5. The speech signal pre-processing system of claim 1, wherein the morphological analyzer comprises:

a morphological filter for performing a morphological operation of the speech signal frame based on a pre-set window size and extracting a characteristic frequency from a result of the morphological operation by performing morphological closing and pre-processing with respect to the converted speech signal waveform; and

a structuring set size (SSS) determiner for determining an optimal SSS of the morphological filter, which performs the morphological closing with respect to the speech signal frame.

6. The speech signal pre-processing system of claim 1, wherein the speech signal characteristic information extractor comprises:

an envelope extractor for extracting at least one of envelope information of harmonic peaks and envelope information of non-harmonic peaks from the speech signal frame or characteristic frequency regions according to a morphological analysis result;

a pitch extractor for extracting pitch information using the speech signal frame or the characteristic frequency regions according to the morphological analysis result; and

a neural network system for detecting characteristic information from the speech signal frame or the characteristic frequency regions according to the morphological analysis result, granting a pre-set weight to each piece of the detected characteristic information, and determining according to a neural network recognition result whether the speech signal frame corresponds to a voiced sound, an unvoiced sound, or background noise.

7. The speech signal preprocessing system of claim 6, wherein the neural network system has two neural networks.

8. The speech signal pre-processing system of claim 7, wherein if a determination result of the speech signal frame or a speech signal corresponding to the characteristic frequency regions according to first neural network recognition, does not indicate a voiced sound, the neural network system reserves the determination of the speech signal frame or the characteristic frequency regions, performs second neural network recognition using a voiced sound/unvoiced sound/background noise determination result of a first neural network with respect to at least one different speech signal frame or characteristic frequency regions, and secondary statistical values of various kinds of characteristic information extracted from the different speech signal frames or characteristic frequency regions, and determines according to a result of the second neural network recognition whether the input speech signal is a voiced sound, an unvoiced sound, or background noise.

9. The speech signal pre-processing system of claim 6, wherein the pitch extractor extracts the pitch information by detecting an energy ratio of a harmonic area to a noise area from the characteristic frequency regions and determining peaks having a maximum energy ratio as the pitch information.

10. The speech signal pre-processing system of claim 5, wherein the envelope extractor extracts the harmonic peak envelope information by detecting a maximum peak as a first harmonic peak from the speech signal frame or the characteristic frequency regions for a first pitch period, selecting harmonic peaks through a process of detecting maximum harmonic peaks of subsequent search zones, and applying interpolation to the selected harmonic peaks.

11. The speech signal pre-processing system of claim 10, wherein the envelope extractor extracts the non-harmonic peak envelope information by selecting peaks, which have not been selected as the harmonic peaks, and applying interpolation to the selected peaks.

12. The speech signal pre-processing system of claim 11, wherein the controller determines, using the harmonic peak envelope information and the non-harmonic peak envelope information, whether the speech signal frame corresponds to a voiced sound or an unvoiced sound.

13. The speech signal pre-processing system of claim 12, further comprising a voiced grade calculator for calculating a voiced grade by calculating an energy ratio of the harmonic peak envelope information to the non-harmonic peak envelope information.

14. The speech signal pre-processing system of claim 13, wherein the controller determines whether the speech signal frame or a speech signal corresponding to the characteristic frequency regions is a voiced sound, an unvoiced sound, or background noise, by comparing the calculated voiced grade to a pre-set voiced threshold or both the pre-set voiced threshold and a pre-set unvoiced threshold.

15. The speech signal pre-processing system of claim 13, wherein the envelope extractor extracts secondary harmonic peak envelope information by selecting secondary harmonic peaks from the selected harmonic peaks using the harmonic peak envelope information and applying interpolation to the selected secondary harmonic peaks.

16. The speech signal pre-processing system of claim 15, wherein the voiced grade calculator calculates a voiced grade by calculating an energy ratio of the secondary harmonic peak envelope information to the non-harmonic peak envelope information.

17. The speech signal pre-processing system of claim 13, wherein the controller determines whether the speech signal frame or a speech signal corresponding to the characteristic frequency regions is a voiced sound, an unvoiced sound, or background noise, by comparing the calculated voiced grade to a pre-set voiced threshold or both the pre-set voiced threshold and a pre-set unvoiced threshold.

18. A method of extracting characteristic information of a speech signal, the method comprising the steps of:

generating a speech signal frame by recognizing speech from an input signal, extracting the speech, converting the received input signal of a time domain to a speech signal of a frequency domain, and outputting the speech signal;

determining, according to a pre-set determination condition, whether characteristic information of the speech signal is extracted using harmonic peaks of the speech signal frame;

performing a morphological analysis of the speech signal frame according to a harmonic peaks usage determination result and extracting characteristic frequency regions according to a morphological analysis result;

extracting speech signal characteristic information requested by a speech signal processing system in a next stage using the characteristic frequency regions or the speech signal frame according to a harmonic peaks usage determination result; and

outputting the extracted speech signal characteristic information to the speech signal processing system.

19. The method of claim 18, wherein the step of generating a speech signal frame comprises:

recognizing a speech signal from the input signal;

generating a speech signal frame by converting the received speech signal of a time domain to a speech signal of a frequency domain; and

canceling noise from the speech signal frame.

20. The method of claim 19, wherein the step of canceling noise comprises setting a larger amplitude ratio of a signal having an amplitude less than a pre-set threshold to a signal having an amplitude greater than or equal to the pre-set threshold by setting weights according to an amplitude of the speech signal frame performing a square operation of each amplitude based on the set weights, and granting a (+) or (−

) sign to a result of the square operation based on the pre-set threshold.

21. The method of claim 18, wherein the step of determining comprises determining according to a characteristic of the speech signal frame or the speech signal processing system in a next stage whether characteristic information of the speech signal is extracted using harmonic peaks of the speech signal frame.

22. The method of claim 18, wherein the step of performing comprises:

determining an optimal structuring set size (SSS) of the morphological filter, which performs morphological closing with respect to the speech signal frame;

performing a morphological operation with respect to the speech signal frame based on a window size according to the determined SSS; and

extracting a characteristic frequency by performing the morphological closing of the speech signal frame using the morphological operation result and performing pre-processing in which only harmonic signals are obtained by removing staircase signals from the converted speech signal.

23. The method of claim 22, wherein the step of determining an optimal SSS is represented by the equation below

window size=(structuring set size (SSS)×

2+1).

24. The method of claim 18, wherein the step of extracting the speech signal characteristic information comprises extracting envelope information from the speech signal frame or the characteristic frequency regions.

25. The method of claim 24, wherein the step of extracting envelope information comprises:

receiving the speech signal frame or the characteristic frequency regions;

detecting a maximum peak as a first harmonic peak from the speech signal frame or the characteristic frequency regions for a first pitch period;

selecting harmonic peaks of subsequent search zones; and

extracting harmonic peak envelope information by applying interpolation to the selected harmonic peaks.

26. The method of claim 25, further comprising extracting non-harmonic peak envelope information by selecting peaks, which have not been selected as the harmonic peaks, and applying interpolation to the selected peaks which have not been selected as the harmonic peaks.

27. The method of claim 18, wherein the step of extracting the speech signal characteristic information comprises extracting pitch information from the speech signal frame or the characteristic frequency regions.

28. The method of claim 27, wherein the step of extracting pitch information comprises:

detecting an energy ratio of a harmonic area to a noise area from the speech signal frame or the characteristic frequency regions; and

extracting the pitch information by determining peaks having a maximum energy ratio as the pitch information.

29. The method of claim 18, wherein the step of extracting the speech signal characteristic information comprises determining whether the speech signal frame or a speech signal corresponding to the characteristic frequency regions corresponds to a voiced sound, an unvoiced sound, or background noise.

30. The method of claim 29, wherein the step of determining comprises:

determining according to a pre-set condition whether envelope information extracted from the speech signal frame or the characteristic frequency regions is used or a neural network recognition method using characteristic information extracted from the speech signal frame or the characteristic frequency regions is used; and

determining whether the speech signal frame or a speech signal corresponding to the characteristic frequency regions corresponds to a voiced sound, an unvoiced sound, or background noise, by selecting the method using the envelope information or the neural network recognition method according to the determination result according to the pre-set condition.

31. The method of claim 30, wherein the method using the envelope information comprises:

receiving the speech signal frame or the characteristic frequency regions;

selecting harmonic peaks from the speech signal frame or the characteristic frequency regions;

extracting harmonic peak envelope information by applying interpolation to the selected harmonic peaks;

extracting non-harmonic peak envelope information by selecting peaks, which have not been selected as the harmonic peaks, and applying interpolation to the selected peaks which have not been selected as the harmonic peaks;

calculating an energy ratio of the harmonic peak envelope information to the non-harmonic peak envelope information as a voiced grade; and

determining according to the voiced grade whether the speech signal frame or a speech signal corresponding to the characteristic frequency regions corresponds to a voiced sound or an unvoiced sound.

32. The method of claim 31, wherein the step of extracting harmonic peak envelope information comprises:

selecting secondary harmonic peaks from the selected harmonic peaks using the extracted harmonic peak envelope information; and

extracting envelope information of the secondary harmonic peaks by applying interpolation to the selected secondary harmonic peaks and extracting the information of the secondary harmonic peaks as secondary harmonic peak envelope information.

33. The method of claim 32, wherein the step of calculating a voiced grade comprises calculating an energy ratio of the secondary harmonic peak envelope information to the non-harmonic peak envelope information as the voiced grade.

34. The method of claim 31, wherein the step of determining comprises comparing the calculated voiced grade to a pre-set voiced threshold and determining according to the comparison result whether the speech signal frame or a speech signal corresponding to the characteristic frequency regions is a voiced sound or an unvoiced sound.

35. The method of claim 31, wherein the step of determining comprises comparing the calculated voiced grade to both a pre-set voiced threshold and a pre-set unvoiced threshold and determining according to the comparison result whether the speech signal frame or a speech signal corresponding to the characteristic frequency regions is a voiced sound, an unvoiced sound, or background noise.

36. The method of claim 30, wherein the neural network recognition method comprises:

extracting characteristic information from the speech signal frame or the characteristic frequency regions; and

determining whether the speech signal frame or a speech signal corresponding to the characteristic frequency regions is a voiced sound, an unvoiced sound, or background noise, by granting pre-set weights to the extracted characteristic information and performing a neural network operation based on the granted weights.

37. The method of claim 30, wherein the neural network recognition method comprises:

extracting characteristic information from the speech signal frame or the characteristic frequency regions;

determining whether the speech signal frame or a speech signal corresponding to the characteristic frequency regions is a voiced sound, by inputting the extracted characteristic information and weights granted to the extracted characteristic information to a first neural network;

outputting the first neural network recognition result as a determination result of the speech signal frame or the speech signal corresponding to the characteristic frequency regions if it is determined as a first neural network recognition result that the speech signal frame or the speech signal, and reserving determination of the speech signal frame or the speech signal corresponding to the characteristic frequency regions if it is determined as the first neural network recognition result that the speech signal frame or the speech signal corresponding to the characteristic frequency regions is not a voiced sound;

checking whether a determination-reserved speech signal exists if it is determined as the first neural network recognition result that the speech signal frame or the speech signal corresponding to the characteristic frequency regions is a voiced sound;

storing characteristic information extracted from more than a pre-set number of speech signal frames or characteristic frequency regions if it is determined as the checking result that a determination-reserved speech signal exists;

determining whether the speech signal frame or the speech signal corresponding to the characteristic frequency regions is a unvoiced sound or background noise, by inputting the first neural network recognition result of the determination-reserved speech signal, secondary statistical values of the information extracted from more than a pre-set number of speech signal frames or characteristic frequency regions, and weights set to the first neural network recognition result and the secondary statistical values to a second neural network; and

determining according to a second neural network recognition result whether the determination-reserved speech signal is a voiced sound, an unvoiced sound, or background noise.