Speech analysis apparatus, speech analysis method and computer program

• US 8,165,873 B2
• Filed: 07/21/2008
• Issued: 04/24/2012
• Est. Priority Date: 07/25/2007
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Yamada, Keiichi
• Primary Examiner(s): Smits, Talivaldis Ivars
• Assistant Examiner(s): Sirjani, Fariba
• Application Number:
  US12/176,739
• Publication Number:
  US 20090030690A1
• Time in Patent Office: 1,373 Days
• Field of Search: 704/207, 704216-218, 704/260, 704/201, 704/2, 704/4
• US Class Current: 704/207
• CPC Class Codes: G10L 15/1807 using prosody or stress
• CPC Class Codes: G10L 25/90 Pitch determination of speech signals

Abstract

A speech analysis apparatus analyzing prosodic characteristics of speech information and outputting a prosodic discrimination result includes an input unit inputting speech information, an acoustic analysis unit calculating relative pitch variation and a discrimination unit performing speech discrimination processing, in which the acoustic analysis unit calculates a current template relative pitch difference, determining whether a difference absolute value between the current template relative pitch difference and a previous template relative pitch difference is equal to or less than a predetermined threshold or not, when the value is not less than the threshold, calculating an adjacent relative pitch difference, and when the adjacent relative pitch difference is equal to or less than a previously set margin value, executing correction processing of adding or subtracting an octave of the current template relative pitch difference to calculate the relative pitch variation by applying the relative pitch difference as the relative pitch difference of the current analysis frame.

Claims

1. A speech analysis apparatus analyzing prosodic characteristics of speech information and outputting a prosodic discrimination result, comprising:

an input unit performing input of speech information;

an acoustic analysis unit analyzing frequency characteristics of respective analysis frames set in time series with respect to speech information inputted from the input unit and calculating relative pitch variation as variation information of frequency characteristics of respective analysis frames; and

a discrimination unit performing speech discrimination processing based on the relative pitch variation generated by the acoustic analysis unit, and

wherein the acoustic analysis unit calculates a current template relative pitch difference which is a relative pitch difference between a frequency characteristic of a current analysis frame and a previously set template frequency characteristic, determining whether a difference absolute value between the current template relative pitch difference and a previous template relative pitch difference which is a relative pitch difference between a frequency characteristic of a previous frame which is temporally previous to the current analysis frame and the template frequency characteristic is equal to or less than a predetermined threshold or not, when the difference absolute value is not equal to or less than the predetermined threshold, calculating an adjacent relative pitch difference which is a relative pitch difference between the frequency characteristic of the current analysis frame and the frequency characteristic of the previous frame, and when the adjacent relative pitch difference is equal to or less than a previously set margin value, executing correction processing of adding or subtracting an octave of the current template relative pitch difference to calculate the relative pitch variation by applying the template relative pitch difference as the relative pitch difference of the current analysis frame;

wherein the acoustic analysis unit calculates the relative pitch variation by applying the current template relative pitch difference as the relative pitch difference of the current analysis frame when the difference absolute value between the previous template relative pitch difference and the current template relative pitch difference is equal to or less than the predetermined threshold;

wherein the acoustic analysis unit calculates the relative pitch variation by applying the current template relative pitch difference as the relative pitch difference of the current analysis frame when the difference absolute value between the previous template relative pitch difference and the current template relative pitch difference is not equal or less than the predetermined threshold as well as the adjacent relative pitch difference is not equal or less than the previously set margin value;

wherein the previously set template frequency characteristic is a data creating a frequency characteristic in simulation, in which amplitude of harmonic components are linearly attenuated with respect to a fundamental pitch derived from stored speech signal;

wherein the acoustic analysis unit calculates a cross-correlation matrix defining the relation between two frequency characteristics for calculating the template relative pitch difference, calculating a value corresponding to a shift amount of an edge line connecting peak positions of values of configuration data of the cross-correlation matrix from the principal diagonal of the cross-correlation matrix as the template relative pitch difference;

wherein the acoustic analysis unit calculates a cross-correlation matrix defining the relation between two frequency characteristics for calculating the adjacent relative pitch difference, calculating a value corresponding to a shift amount of an edge line connecting peak positions of values of configuration data of the cross-correlation matrix from the principal diagonal of the cross-correlation matrix as the adjacent relative pitch difference; and

wherein the acoustic analysis unit generates frequency characteristic information in which the frequency characteristic information is expressed on a logarithmic frequency axis, and when the predetermined threshold is T and the previously set margin value is δ

, the predetermined threshold T and the previously set margin value are related according to the following formula

T=log(2)−

δ

.

2. The speech analysis apparatus according to claim 1,

wherein the discrimination unit performs speech discrimination processing by comparing parameters corresponding to a prosodic discrimination unit dictionary previously stored in a storage unit to relative pitch variation data generated by the acoustic analysis unit.

3. The speech analysis apparatus according to claim 2,

wherein the discrimination unit performs speech discrimination processing according to processing applying any of a DP (Dynamic Programming) matching, a neural network, a HMM (Hidden Markov Model).

4. The speech analysis apparatus according to claim 1, further comprising:

a speech recognition unit outputting text information corresponding to input speech data from the input unit as a speech recognition result; and

a result selection unit outputting the speech recognition result by applying a discrimination result of the discrimination unit and a discrimination result of the speech recognition unit.

5. The speech analysis apparatus according to claim 4,

wherein the result selection unit outputs the speech recognition result to which a prosodic discrimination result discriminated in the discrimination unit is added when the speech recognition result corresponds to a specific word as a result of comparison between the speech recognition result in the speech recognition unit and specific words stored in the specific word storage unit, and outputs the speech recognition result as it is when the speech recognition result does not correspond to a specific word.

6. A speech analysis method analyzing prosodic characteristics of speech information and outputting a prosodic discrimination result, comprising the steps of:

performing input of speech information by an input unit;

analyzing frequency characteristics of respective analysis frames set in time series with respect to speech information inputted from the input unit and calculating relative pitch variation as variation information of frequency characteristics of respective analysis frames by an acoustic analysis unit; and

performing speech discrimination processing by a discrimination unit based on the relative pitch variation generated by the acoustic analysis unit, and

wherein the acoustic analysis unit has the steps of calculating a current template relative pitch difference which is a relative pitch difference between a frequency characteristic of a current analysis frame and a previously set template frequency characteristic, determining whether a difference absolute value between the current template relative pitch difference and a previous template relative pitch difference which is a relative pitch difference between a frequency characteristic of a previous frame which is temporally previous to the current analysis frame and the template frequency characteristic is equal to or less than a predetermined threshold or not, when the difference absolute value is not equal to or less than the predetermined threshold, calculating an adjacent relative pitch difference which is a relative pitch difference between the frequency characteristic of the current analysis frame and the frequency characteristic of the previous frame, and when the adjacent relative pitch difference is equal to or less than a previously set margin value, executing correction processing of adding or subtracting an octave of the current template relative pitch difference to calculate the relative pitch variation by applying the template relative pitch difference as the relative pitch difference of the current analysis frame;

wherein the step performed by the acoustic analysis unit is the step of calculating the relative pitch variation by applying the current template relative pitch difference as the relative pitch difference of the current analysis frame when the difference absolute value between the previous template relative pitch difference and the current template relative pitch difference is equal to or less than the predetermined threshold;

wherein the step performed by the acoustic analysis unit is the step of calculating the relative pitch variation by applying the current template relative pitch difference as the relative pitch difference of the current analysis frame when the difference absolute value between the previous template relative pitch difference and the current template relative pitch difference is not equal or less than the predetermined threshold as well as the adjacent relative pitch difference is not equal or less than the previously set margin value;

wherein the previously set template frequency characteristic is a data creating a frequency characteristic in simulation, in which amplitude of harmonic components are linearly attenuated with respect to a fundamental pitch derived from stored speech signal;

wherein the step performed by the acoustic analysis unit has the steps of calculating a cross-correlation matrix defining the relation between two frequency characteristics for calculating the template relative pitch difference and calculating a value corresponding to a shift amount of an edge line connecting peak positions of values of configuration data of the cross-correlation matrix from the principal diagonal of the cross-correlation matrix as the template relative pitch difference;

wherein the step performed by the acoustic analysis unit has the steps of calculating a cross-correlation matrix defining the relation between two frequency characteristics for calculating the adjacent relative pitch difference and calculating a value corresponding to a shift amount of an edge line connecting peak positions of values of configuration data of the cross-correlation matrix from the principal diagonal of the cross-correlation matrix as the adjacent relative pitch difference; and

wherein the steps performed by the acoustic analysis unit has the steps of generating frequency characteristic information in which the frequency characteristic information is expressed on a logarithmic frequency axis, and when the predetermined threshold is T and the previously set margin value is δ

, the predetermined threshold T and the previously set margin value δ

are related according to the following formula

T=log(2)−

δ

.

7. The speech analysis method according to claim 6,

wherein the step performed by the discrimination unit is the step of performing speech discrimination processing by comparing parameters corresponding to a prosodic discrimination unit dictionary previously stored in a storage unit to relative pitch variation data generated by the acoustic analysis unit.

8. The speech analysis method according to claim 7,

wherein the step performed by the discrimination unit is the step of performing speech discrimination processing according to processing applying any of a DP (Dynamic Programming) matching, a neural network, a HMM (Hidden Markov Model).

9. The speech analysis method according to claim 6, further comprising the steps of:

outputting text information corresponding to input speech data from the input unit as a speech recognition result by a speech recognition unit; and

outputting the speech recognition result by applying a discrimination result of the discrimination unit and a discrimination result of the speech recognition unit by the result selection unit.

10. The speech analysis method according to claim 9,

wherein the step performed by the result selection unit is the step of outputting the speech recognition result to which a prosodic discrimination result discriminated in the discrimination unit is added when the speech recognition result corresponds to a specific word as a result of comparison between the speech recognition result in the speech recognition unit and specific words stored in the specific word storage unit, or outputting the speech recognition result as it is when the speech recognition result does not correspond to a specific word.

11. A non-transitory computer readable storage device storing a computer program, which when executed by a speech analysis apparatus performs a method for analyzing prosodic characteristics of speech information and outputting a prosodic discrimination result, comprising the steps of:

receiving input of speech information by an input unit;

analyzing, with an acoustic analysis unit, frequency characteristics of respective analysis frames set in time series with respect to speech information inputted from the input unit and calculating relative pitch variation as variation information of frequency characteristics of respective analysis frames; and

performing, by a discrimination unit, speech discrimination processing based on the relative pitch variation generated by the acoustic analysis unit, and

wherein the step performed by the acoustic analysis unit has the steps of calculating a current template relative pitch difference which is a relative pitch difference between a frequency characteristic of a current analysis frame and a previously set template frequency characteristic, determining whether a difference absolute value between the current template relative pitch difference and a previous template relative pitch difference which is a relative pitch difference between a frequency characteristic of a previous frame which is temporally previous to the current analysis frame and the template frequency characteristic is equal to or less than a predetermined threshold or not, when the difference absolute value is not equal to or less than the predetermined threshold, calculating an adjacent relative pitch difference which is a relative pitch difference between the frequency characteristic of the current analysis frame and the frequency characteristic of the previous frame, and when the adjacent relative pitch difference is equal to or less than a previously set margin value, executing correction processing of adding or subtracting an octave of the current template relative pitch difference to calculate the relative pitch variation by applying the template relative pitch difference as the relative pitch difference of the current analysis frame;

wherein the acoustic analysis unit calculates the relative pitch variation by applying the current template relative pitch difference as the relative pitch difference of the current analysis frame when the difference absolute value between the previous template relative pitch difference and the current template relative pitch difference is equal to or less than the predetermined threshold;

wherein the acoustic analysis unit calculates the relative pitch variation by applying the current template relative pitch difference as the relative pitch difference of the current analysis frame when the difference absolute value between the previous template relative pitch difference and the current template relative pitch difference is not equal or less than the predetermined threshold as well as the adjacent relative pitch difference is not equal or less than the previously set margin value;

wherein the previously set template frequency characteristic is a data creating a frequency characteristic in simulation, in which amplitude of harmonic components are linearly attenuated with respect to a fundamental pitch derived from stored speech signal;

wherein the steps performed by the acoustic analysis unit has the steps of calculating a cross-correlation matrix defining the relation between two frequency characteristics for calculating the template relative pitch difference and calculating a value corresponding to a shift amount of an edge line connecting peak positions of values of configuration data of the cross-correlation matrix from the principal diagonal of the cross-correlation matrix as the template relative pitch difference;

wherein the steps performed by the acoustic analysis unit has the steps of calculating a cross-correlation matrix defining the relation between two frequency characteristics for calculating the adjacent relative pitch difference, calculating a value corresponding to a shift amount of an edge line connecting peak positions of values of configuration data of the cross-correlation matrix from the principal diagonal of the cross-correlation matrix as the adjacent relative pitch difference; and

wherein the steps performed by the acoustic analysis unit has the steps of generating frequency characteristic information in which the frequency characteristic information is expressed on a logarithmic frequency axis, and when the predetermined threshold is T and the previously set margin value is δ

the predetermined threshold T and the previously set margin value δ

are related according to the following formula

T=log(2)−

δ

.