Apparatus and method for creating pitch wave signals, apparatus and method for compressing, expanding, and synthesizing speech signals using these pitch wave signals and text-to-speech conversion using unit pitch wave signals

• US 7,647,226 B2
• Filed: 03/09/2007
• Issued: 01/12/2010
• Est. Priority Date: 08/31/2001
• Status: Active Grant

• Current Assignee: Rakuten Group, Inc.
• Original Assignee: Kabushiki Kaisha Kenwood
• Inventors: Sato, Yasushi
• Primary Examiner(s): Smits; Talivaldis Ivars
• Application Number:
  US11/715,937
• Publication Number:
  US 20070174056A1
• Time in Patent Office: 1,040 Days
• Field of Search: 704/260, 704/207
• US Class Current: 704/260
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 19/09 Long term prediction, i.e. removing periodical redundancies, e.g. by using adaptive codebook or pitch predictor
• CPC Class Codes: G10L 21/003 Changing voice quality, e.g. pitch or formants
• CPC Class Codes: G10L 21/013 Adapting to target pitch
• CPC Class Codes: G10L 21/04 Time compression or expansion

Abstract

A pitch wave signal creation method as a preliminary process for efficiently coding a speech wave signal having a fluctuated pitch period is provided. A speech signal compressing/expanding apparatus and a speech signal synthesizing apparatus using the method, and a signal processing associated therewith are further provided. The pitch wave creation method of the invention is essentially comprised of a method of detecting the instantaneous pitch period of each pitch wave element of the speech wave signal, and a process of converting a corresponding pitch wave element into a normalized pitch wave element having a predetermined fixed time length by expanding and compressing the pitch wave element on a time axis while retaining its wave pattern based on the each detected instantaneous pitch period. The speech signal having a pitch fluctuation can be compressed in high quality and high efficiency by coding or synthesizing the speech wave signal using the pitch wave signal creation method of the invention. Text-to-speech conversion using pitch wave signals.

Claims

1. A speech synthesizing apparatus, the apparatus comprising:

division means for dividing an input speech signal into a plurality of unit speech samples;

signal creating means for creating a pitch wave signal from each of the unit speech samples, the pitch wave signal comprising a plurality of normalized pitch wave elements which have a substantially identical time length and uniform phase, wherein the pitch wave signal is created in such a way that a pitch signal representing pitch periods in the unit speech sample is generated and the phase of a speech wave in each pitch period is shifted so as to maximize the correlation between the speech wave in the pitch period and the pitch signal and that the phase shifted speech wave in each pitch period is resampled with the same number of samples to make uniform the time length of the speech wave in each pitch period to the same time length;

storage means for storing rhythm information representing the rhythm of each unit speech sample, pitch information representing the pitch of the sample, the spectrum information showing variation with time in the fundamental frequency component and harmonic wave component of the pitch wave signal in such a manner that each of the rhythm information, the pitch information and the spectrum information corresponds to the sample;

prediction means for inputting text information representing a text, and creating prediction information representing the result of predicting the pitch and spectrum of a unit speech constituting the text based on the text information;

retrieval means for identifying a sample having a pitch and spectrum having the highest correlation with the pitch and spectrum of the unit speech constituting the text based on the pitch information, spectrum information and prediction information; and

signal synthesizing means for creating a synthesized speech signal representing a speech in which the speech has a rhythm represented by the rhythm information brought into correspondence with the sample identified by the retrieval means, the variation with time in the fundamental frequency component and harmonic wave component is represented by the spectrum information brought into correspondence with the sample identified by the retrieval means, and the time length of one pitch period is a time length represented by the pitch information brought into correspondence with the sample identified by the retrieval means.

2. The speech synthesizing apparatus according to claim 1, wherein the spectrum information is constituted by data representing the result of nonlinearly quantizing the value representing variation with time in the fundamental frequency component and harmonic wave component of the pitch wave signal, and wherein the phase to be shifted of the speech wave in one pitch period has a value of φ

giving the maximum cor, in accordance with the following expression;

$\mathrm{cor}=\underset{i=1}{\overset{n}{\sum <br><br>}}<br><br>\left\{f<br><br>\left(i-\mathrm{\phi <br><br>}\right)·<br><br>g<br><br>\left(i\right)\right\}$

(where, n is a total number of samples in one pitch period, f(β

) is a value of β

-th sample in a speech wave signal within one pitch period, and g(γ

) is a value of γ

-th sample in the pitch signal within the one pitch period).

3. A speech synthesizing method, the method comprising the steps of:

dividing an input speech signal into a plurality of unit speech samples;

creating a pitch wave signal from each of the unit speech samples, the pitch wave signal comprising a plurality of normalized pitch wave elements which have a substantially identical time length and uniform phase, wherein the pitch wave signal is created in such a way that a pitch signal representing pitch periods in the unit speech sample is generated and the phase of a speech wave in each pitch period is shifted so as to maximize the correlation between the speech wave in the pitch period and the pitch signal and that the phase shifted speech wave in each pitch period is resampled with the same number of samples to make uniform the time length of the speech wave in each pitch period to the same time length;

storing rhythm information representing the rhythm of each unit speech sample, pitch information representing the pitch of the sample, and spectrum information showing variation with time in the fundamental frequency component and harmonic wave component of the pitch wave signal in such a manner that each of the rhythm information, the pitch information and the spectrum information corresponds to the sample;

inputting text information representing a text is inputted to create prediction information representing the result of predicting the pitch and spectrum of a unit speech constituting the text on the basis of the text information;

identifying a sample having a pitch and spectrum having the highest correlation with the pitch and spectrum of the unit speech constituting the text on the basis of the pitch information, spectrum information and prediction information; and

creating a synthesized speech signal representing a speech in which the speech has a rhythm represented by the rhythm information brought into correspondence with the identified sample, the variation with time in the fundamental frequency component and harmonic wave component is represented by the spectrum information brought into correspondence with the sample identified by the retrieval means, and the time length of one pitch period is a time length represented by the pitch information brought into correspondence with the sample identified by the retrieval means.

4. The speech synthesizing method according to claim 3, wherein the phase to be shifted of the speech wave in one pitch period has a value of φ

giving the maximum cor, in accordance with the following expression;

$\mathrm{cor}=\underset{i=1}{\overset{n}{\sum <br><br>}}<br><br>\left\{f<br><br>\left(i-\mathrm{\phi <br><br>}\right)·<br><br>g<br><br>\left(i\right)\right\}$

(where, n is a total number of samples in one pitch period, f(β

) is a value of β

-th sample in a speech wave signal within one pitch period, and g(γ

) is a value of γ

-th sample in the pitch signal within the one pitch period).