Voice converter with extraction and modification of attribute data

• US 7,606,709 B2
• Filed: 10/29/2002
• Issued: 10/20/2009
• Est. Priority Date: 06/15/1998
• Status: Expired due to Fees

• Current Assignee: Pompeu Fabra University, Yamaha Corporation
• Original Assignee: Pompeu Fabra University, Yamaha Corporation
• Inventors: Serra, Xavier, Bonada, Jordi, Yoshioka, Yasuo, Kayama, Hiraku
• Primary Examiner(s): Armstrong; Angela A
• Application Number:
  US10/282,536
• Publication Number:
  US 20030055646A1
• Time in Patent Office: 2,548 Days
• Field of Search: 704200-201, 704/205, 704207-209, 704267-269, 704/258, 704/278, 84622-625, 846/27
• US Class Current: 704/258
• CPC Class Codes: G10L 13/033 Voice editing, e.g. manipulating the voice of the synthesiser
• CPC Class Codes: G10L 19/093 using sinusoidal excitation models
• CPC Class Codes: G10L 2021/0135 Voice conversion or morphing
• CPC Class Codes: G10L 21/02 Speech enhancement, e.g. noise reduction or echo cancellation reducing echo effects in line transmission systems H04B3/20; echo suppression in hands-free telephones H04M9/08
• CPC Class Codes: G10L 25/93 Discriminating between voiced and unvoiced parts of speech signals G10L25/90 takes precedence

Abstract

An apparatus is constructed for converting an input voice signal into an output voice signal according to a target voice signal. In the apparatus, an input device provides the input voice signal composed of original sinusoidal components and original residual components other than the original sinusoidal components. An extracting device extracts original attribute data from at least the sinusoidal components of the input voice signal. The original attribute data is characteristic of the input voice signal. A synthesizing device synthesizes new attribute data based on both of the original attribute data derived from the input voice signal and target attribute data being characteristic of the target voice signal composed of target sinusoidal components and target residual components other than the sinusoidal components. The target attribute data is derived from at least the target sinusoidal components. An output device operates based on the new attribute data and either of the original residual component and the target residual component for producing the output voice signal.

Claims

1. An apparatus for converting an input voice signal into an output voice signal by modifying a spectral shape, the apparatus comprising:

an input device that provides the input voice signal containing wave components;

a separating device that separates sinusoidal ones of the wave components from the input voice signal such that each sinusoidal wave component is identified by a pair of a frequency and an amplitude;

a computing device that computes a spectral shape of the input voice signal based on a set of the separated sinusoidal wave components such that the spectral shape represents an envelope having a series of break points corresponding to the pairs of the frequencies and the amplitudes of the sinusoidal wave components, the computing device further computing a spectral tilt of the spectral shape of the input voice signal and computing a modification amount of the spectral tilt of the spectral shape of the input voice signal according to a predetermined pitch of the output voice signal;

a modifying device that modifies the spectral shape to form a new spectral shape representing a modified envelope having a series of new break points by shifting the envelope along an axis of the frequency on a coordinates system of the frequency and the amplitude, the modifying device using a function defining a relation between a modification degree and a pitch, and determining the modification degree of a frequency or an amplitude of each break point of the new spectral shape according to the predetermined pitch of the output voice signal by using the function, the modifying device further modifying the spectral tilt of the new spectral shape by the computed modification amount;

a generating device that determines a series of frequencies according to the predetermined pitch of the output voice signal, and that selects a series of points which are positioned along the modified envelope of the new spectral shape in correspondence to the series of the determined frequencies, but which are different from the series of the new break points of the modified envelope, and that generates a set of new sinusoidal wave components each identified by each pair of a frequency and an amplitude, which corresponds to each of the series of the selected points; and

an output device that produces the output voice signal based on the set of the new sinusoidal wave components.

2. The apparatus according to claim 1, wherein the computing device computes the modification amount of the spectral tilt according to a rate function which defines a relationship between the modification amount and the predetermined pitch of the output voice signal.

3. A method of converting an input voice signal into an output voice signal by modifying a spectral shape, the method comprising:

providing the input voice signal containing wave components;

separating sinusoidal ones of the wave components from the input voice signal such that each sinusoidal wave component is identified by a pair of a frequency and an amplitude;

computing a spectral shape of the input voice signal based on a set of the separated sinusoidal wave components such that the spectral shape represents an envelope having a series of break points corresponding to the pairs of the frequencies and the amplitudes of the sinusoidal wave components, and further computing a spectral tilt of the spectral shape of the input voice signal and computing a modification amount of the spectral tilt of the spectral shape of the input voice signal according to a predetermined pitch of the output voice signal;

modifying the spectral shape to form a new spectral shape representing a modified envelope having a series of new break points by shifting the envelope along an axis of the frequency on a coordinates system of the frequency and the amplitude, using a function defining a relation between a modification degree and a pitch, and determining the modification degree of a frequency or an amplitude of each break point of the new spectral shape according to the predetermined pitch of the output voice signal by using the function, and further modifying the spectral tilt of the new spectral shape by the computed modification amount;

determining a series of frequencies according to the predetermined pitch of the output voice signal, and selecting a series of points which are positioned along the modified envelope of the new spectral shape in correspondence to the series of the determined frequencies, but which are different from the series of the new break points of the modified envelope, and generating a set of new sinusoidal wave components each identified by each pair of a frequency and an amplitude, which corresponds to each of the series of the selected points; and

producing the output voice signal based on the set of the new sinusoidal wave components.

4. The method according to claim 3, wherein the modification amount of the spectral tilt is computed according to a rate function which defines a relationship between the modification amount and the predetermined pitch of the output voice signal.

5. A machine readable medium used in a computer machine having a CPU, the medium containing program instructions executable by the CPU to cause the computer machine to perform a process of converting an input voice signal into an output voice signal by modifying a spectral shape, the process comprising:

providing the input voice signal containing wave components;

separating sinusoidal ones of the wave components from the input voice signal such that each sinusoidal wave component is identified by a pair of a frequency and an amplitude;

computing a spectral shape of the input voice signal based on a set of the separated sinusoidal wave components such that the spectral shape represents an envelope having a series of break points corresponding to the pairs of the frequencies and the amplitudes of the sinusoidal wave components, and further computing a spectral tilt of the spectral shape of the input voice signal and computing a modification amount of the spectral tilt of the spectral shape of the input voice signal according to a predetermined pitch of the output voice signal;

modifying the spectral shape to form a new spectral shape representing a modified envelope having a series of new break points by shifting the envelope along an axis of the frequency on a coordinates system of the frequency and the amplitude, using a function defining a relation between a modification degree and a pitch, and determining the modification degree of a frequency or an amplitude of each break point of the new spectral shape according to the predetermined pitch of the output voice signal by using the function, and further modifying the spectral tilt of the new spectral shape by the computed modification amount;

determining a series of frequencies according to the predetermined pitch of the output voice signal, and selecting a series of points which are positioned along the modified envelope of the new spectral shape in correspondence to the series of the determined frequencies, but which are different from the series of the new break points of the modified envelope, and generating a set of new sinusoidal wave components each identified by each pair of a frequency and an amplitude, which corresponds to each of the series of the selected points; and

producing the output voice signal based on the set of the new sinusoidal wave components.

6. The machine readable medium according to claim 5, wherein, in the computing step of said process, the modification amount of the spectral tilt is computed according to a rate function which defines a relationship between the modification amount and the predetermined pitch of the output voice signal.

7. A method of generating a female voice signal from a male voice signal by modifying a spectral shape, the method comprising:

providing the male voice signal containing wave components;

separating sinusoidal ones of the wave components from the male voice signal such that each sinusoidal wave component is identified by a pair of a frequency and an amplitude;

computing a spectral shape of the male voice signal based on a set of the separated sinusoidal wave components such that the spectral shape represents an envelope having a series of break points corresponding to the pairs of the frequencies and the amplitudes of the sinusoidal wave components, and further computing a spectral tilt of the spectral shape of the male voice signal as a straight line that is approximated to the respective amplitudes of the sinusoidal wave components and computing a modification amount of the spectral tilt according to a rate function which defines a relationship between the modification amount and a predetermined pitch of the female voice signal;

modifying the spectral shape to form a new spectral shape representing a modified envelope having a series of new break points by shifting the envelope along an axis of the frequency on a coordinates system of the frequency and the amplitude, using a function defining a relation between a modification degree and a pitch, and determining the modification degree of a frequency or an amplitude of each break point of the new spectral shape according to the predetermined pitch of the female voice signal by using the function, and further modifying the spectral tilt of the new spectral shape of the male voice signal by the computed modification amount to a spectral tilt for the female voice signal said spectral tilt of the female voice signal having a slope larger than a slope of the spectral tilt of the male voice signal;

determining a series of frequencies according to the predetermined pitch of the female voice signal, and selecting a series of points which are positioned along the modified envelope of the new spectral shape in correspondence to the series of the determined frequencies, but which are different from the series of the new break points of the modified envelope, and generating a set of new sinusoidal wave components each identified by each pair of a frequency and an amplitude, which corresponds to each of the series of the selected points; and

producing the female voice signal based on the set of the new sinusoidal wave components.

8. A machine readable medium used in a computer machine having a CPU, the medium containing program instructions executable by the CPU to cause the computer machine to perform a process of generating a female voice signal from a male voice signal by modifying a spectral shape, the process comprising:

providing the male voice signal containing wave components;

separating sinusoidal ones of the wave components from the male voice signal such that each sinusoidal wave component is identified by a pair of a frequency and an amplitude;

computing a spectral shape of the male voice signal based on a set of the separated sinusoidal wave components such that the spectral shape represents an envelope having a series of break points corresponding to the pairs of the frequencies and the amplitudes of the sinusoidal wave components, and further computing a spectral tilt of the spectral shape of the male voice signal as a straight line that is approximated to the respective amplitudes of the sinusoidal wave components and computing a modification amount of the spectral tilt according to a rate function which defines a relationship between the modification amount and a predetermined pitch of the female voice signal;

modifying the spectral shape to form a new spectral shape representing a modified envelope having a series of new break points by shifting the envelope along an axis of the frequency on a coordinates system of the frequency and the amplitude, using a function defining a relation between a modification degree and a pitch, and determining the modification degree of a frequency or an amplitude of each break point of the new spectral shape according to the predetermined pitch of the female voice signal by using the function, and further modifying the spectral tilt of the new spectral shape of the male voice signal by the computed modification amount to a spectral tilt for the female voice signal said spectral tilt of the female voice signal having a slope larger than a slope of the spectral tilt of the male voice signal;

determining a series of frequencies according to the predetermined pitch of the female voice signal, and selecting a series of points which are positioned along the modified envelope of the new spectral shape in correspondence to the series of the determined frequencies, but which are different from the series of the new break points of the modified envelope, and generating a set of new sinusoidal wave components each identified by each pair of a frequency and an amplitude, which corresponds to each of the series of the selected points; and

producing the female voice signal based on the set of the new sinusoidal wave components.

9. An apparatus for converting a male voice signal into a female voice signal by modifying a spectral shape, the apparatus comprising:

an input device that provides the male voice signal containing wave components;

a separating device that separates sinusoidal ones of the wave components from the male voice signal such that each sinusoidal wave component is identified by a pair of a frequency and an amplitude;

a computing device that computes a spectral shape of the male voice signal based on a set of the separated sinusoidal wave components such that the spectral shape represents an envelope having a series of break points corresponding to the pairs of the frequencies and the amplitudes of the sinusoidal wave components, the computing device further computing a spectral tilt of the spectral shape of the male voice signal as a straight line that is approximated to the respective amplitudes of the sinusoidal wave components and computing a modification amount of the spectral tilt according to a rate function which defines a relationship between the modification amount and a predetermined pitch of the female voice signal;

a modifying device that modifies the spectral shape to form a new spectral shape representing a modified envelope having a series of new break points by shifting the envelope along an axis of the frequency on a coordinates system of the frequency and the amplitude, the modifying device using a function defining a relation between a modification degree and a pitch, and determining the modification degree of a frequency or an amplitude of each break point of the new spectral shape according to the predetermined pitch of the female voice signal by using the function, the modifying device further modifying the spectral tilt of the new spectral shape of the male voice signal by the computed modification amount to a spectral tilt for the female voice signal said spectral tilt of the female voice signal having a slope larger than a slope of the spectral tilt of the male voice signal;

a generating device that determines a series of frequencies according to the predetermined pitch of the female voice signal, and that selects a series of points which are positioned along the modified envelope of the new spectral shape in correspondence to the series of the determined frequencies, but which are different from the series of the new break points of the modified envelope, and that generates a set of new sinusoidal wave components each identified by each pair of a frequency and an amplitude, which corresponds to each of the series of the selected points; and

an output device that produces the female voice signal based on the set of the new sinusoidal wave components.