Sound synthesis processing system

• US 20060195315A1
• Filed: 02/17/2004
• Published: 08/31/2006
• Est. Priority Date: 02/17/2003
• Status: Abandoned Application

• Current Assignee: National Institute of Advanced Industrial Science and Technology (Government of Japan), Kabushiki Kaisha Kenwood
• Original Assignee: National Institute of Advanced Industrial Science and Technology (Government of Japan), Kabushiki Kaisha Kenwood
• Inventors: Kojima, Hiroaki, Sato, Yasushi, Tanaka, Kazuyo
• Application Number:
  US10/546,072
• Publication Number:
  US 20060195315A1
• Time in Patent Office: Days
• US Class Current: 704/207
• CPC Class Codes: G10L 13/06 Elementary speech units used in speech synthesisers; Concatenation rules
• CPC Class Codes: G10L 19/097 using prototype waveform decomposition or prototype waveform interpolative [PWI] coders

Abstract

To provide a pitch waveform signal division device and the like for making it possible to compress a data capacity of data representing a sound efficiently. A computer C1 arranges time lengths of sections for a unit pitch of sound data, which the computer C1 compresses, to be identical to thereby generate a pitch waveform signal, detects a boundary of adjacent phonemes included in a sound represented by the pitch waveform signal and an end of this sound on the basis of intensity of a difference between two sections for adjacent unit pitches of this pitch waveform signal, divides the pitch waveform signal in the detected boundary and end, and outputs obtained data as phoneme data.

Claims

1. A pitch waveform signal division device comprising:

a filter for acquiring a sound signal representing a waveform of sound and filtering the sound signal to extract a pitch signal;

phase adjusting means for delimiting the sound signal into sections based on the pitch signal extracted by the filter and adjusting the phase for each section based on the correlation between the section and the pitch signal;

sampling means for determining a sampling length for each section with the phase adjusted by the phase adjusting means, based on the phase, and performing sampling with the sampling length to generate a sampling signal;

sound signal processing means for processing the sampling signal into a pitch waveform signal based on the result of the adjustment by the phase adjusting means and the value of the sampling length; and

pitch waveform signal dividing means for detecting a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or an end of the sound, and dividing the pitch waveform signal at the detected boundary and/or end.

2. The pitch waveform signal division device according to claim 1, wherein the pitch waveform signal dividing means determines whether the intensity of the difference between two adjacent sections for a unit pitch of the pitch waveform signal is a predetermined amount or more, and if it is determined to be the predetermined amount or more, then it detects the boundary between the two sections as a boundary of adjacent phonemes or an end of sound.

3. The pitch waveform signal division device according to claim 2, wherein the pitch waveform signal dividing means determines whether the two sections represent fricative based on the intensity of a portion of the pitch signal belonging to the two sections, and if it is determined that they represent fricative, then it determines that the boundary of the two sections is not a boundary of adjacent phonemes or an end of sound regardless of whether the intensity of the difference between the two sections is the predetermined amount or more.

4. The pitch waveform signal division device according to claim 2, wherein the pitch waveform signal dividing means determines whether a portion of the pitch signal belonging to the two sections is a predetermined amount or less, and if it is determined to be the amount or less, then it determines that the boundary of the two sections is not a boundary of adjacent phonemes or an end of sound regardless of whether the intensity of the difference between the two sections is the predetermined amount or more.

5. A pitch waveform signal division device comprising:

sound signal processing means for acquiring a sound signal representing a waveform of sound, and processing the sound signal into a pitch waveform signal by substantially equalizing the phases of sections where the sound signal is divided into the sections for a unit pitch of the sound; and

pitch waveform signal dividing means for detecting a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or an end of the sound, and dividing the pitch waveform signal at the detected boundary and/or end.

6. A pitch waveform signal division device comprising:

means for detecting, for pitch waveform signal representing a waveform of sound, a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or an end of the sound; and

means for dividing the pitch waveform signal at the detected boundary and/or end.

7. A sound signal compression device comprising:

a filter for acquiring a sound signal representing a waveform of sound and filtering the sound signal to extract a pitch signal;

phase adjusting means for delimiting the sound signal into sections based on the pitch signal extracted by the filter and adjusting the phase for each section based on the correlation between the section and the pitch signal;

sampling means for determining a sampling length for each section with the phase adjusted by the phase adjusting means, based on the phase, and performing sampling with the sampling length to generate a sampling signal;

sound signal processing means for processing the sampling signal into a pitch waveform signal based on the result of the adjustment by the phase adjusting means and the value of the sampling length;

phoneme data generating means for detecting a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or an end of the sound, and dividing the pitch waveform signal at the detected boundary and/or end to generate phoneme data; and

data compressing means for subjecting the generated phoneme data to entropy coding to perform data compression.

8. The sound signal compression device according to claim 7, wherein the pitch waveform signal dividing means determines whether the intensity of the difference between two adjacent sections for a unit pitch of the pitch waveform signal is a predetermined amount or more, and if it is determined to be the predetermined amount or more, then it detects the boundary between the two sections as a boundary of an adjacent phonemes or an end of sound.

9. The sound signal compression device according to claim 8, wherein the pitch waveform signal dividing means determines whether the two sections represent fricative based on the intensity of a portion of the pitch signal belonging to the two sections, and if it is determined that they represent fricative, then it determines that the boundary of the two sections is not a boundary of adjacent phonemes or an end of sound regardless of whether the intensity of the difference between the two sections is the predetermined amount or more.

10. The sound signal compression device according to claim 8, wherein the pitch waveform signal dividing means determines whether a portion of the pitch signal belonging to the two sections is a predetermined amount or less, and if it is determined to be the amount or less, then it determines that the boundary of the two sections is not a boundary of adjacent phonemes or an end of sound regardless of whether the intensity of the difference between the two sections is the predetermined amount or more.

11. A sound signal compression device comprising:

sound signal processing means for acquiring a sound signal representing a waveform of sound, and processing the sound signal into a pitch waveform signal by substantially equalizing the phases of sections where the sound signal is divided into the sections for a unit pitch of the sound;

phoneme data generating means for detecting a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or an end of the sound, and dividing the pitch waveform signal at the detected boundary and/or end to generate phoneme data; and

data compressing means for subjecting the generated phoneme data to entropy coding to perform data compression.

12. A sound signal compression device comprising:

means for detecting, for pitch waveform signal representing a waveform of sound, a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or an end of the sound;

phoneme data generating means for dividing the pitch waveform signal at the detected boundary and/or end to generate phoneme data; and

data compressing means for subjecting the generated phoneme data to entropy coding to perform data compression.

13-16. -16. (canceled)

17. A database for storing phoneme data, wherein the phoneme data is acquired by dividing a pitch waveform signal at a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or end of the sound, the pitch waveform signal being acquired by substantially equalizing the phases of sections where the sound signal representing a waveform of sound is divided into the sections for a unit pitch of the sound.

18. A database for storing phoneme data, wherein the phoneme data is acquired by dividing a pitch waveform signal representing a waveform of sound at a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or end of the sound.

19-20. -20. (canceled)

21. A computer readable recording medium for storing phoneme data, wherein the phoneme data is acquired by dividing a pitch waveform signal at a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or end of the sound, the pitch waveform signal being acquired by substantially equalizing the phases of sections where the sound signal representing a waveform of sound is divided into the sections for a unit pitch of the sound.

22. A computer readable recording medium for storing phoneme data, wherein the phoneme data is acquired by dividing a pitch waveform signal representing a waveform of sound at a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or end of the sound.

23-24. -24. (canceled)

25. A sound signal restoration device comprising:

data acquiring means for acquiring phoneme data which is acquired by dividing a pitch waveform signal at a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or end of the sound, the pitch waveform signal being acquired by substantially equalizing the phases of sections where the sound signal representing a waveform of sound is divided into the sections for a unit pitch of the sound; and

restoring means for decoding the acquired phoneme data.

26-29. -29. (canceled)

30. A sound synthesis device comprising:

data acquiring means for acquiring phoneme data which is acquired by dividing a pitch waveform signal at a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and end of the sound, the pitch waveform signal being acquired by substantially equalizing the phases of sections where the sound signal representing a waveform of sound is divided into the sections for a unit pitch of the sound;

restoring means for decoding the acquired phoneme data;

phoneme data storing means for storing the acquired phoneme data or the decoded phoneme data;

sentence input means for inputting sentence information representing a sentence; and

synthesizing means for retrieving from the phoneme data storing means, phoneme data representing waveforms of phonemes composing the sentence, and combining the retrieved phoneme data pieces to generate data representing synthesized sound.

31. The sound synthesis device according to claim 30, further comprising:

sound piece storing means for phoneme data pieces representing sound pieces;

rhythm predicting means for predicting a rhythm of a sound piece composing an inputted sentence; and

selecting means for selecting from the sound data pieces, sound data that represents a waveform of a sound piece having the same reading as a sound piece composing the sentence and has a rhythm closest to the prediction result, wherein the synthesizing means comprises

lacked part synthesizing means for retrieving from the phoneme data storing means, for a sound piece of which sound data has not been selectable by the selecting means among the sound pieces composing the sentence, phoneme data representing a waveform of phonemes composing the sound piece having not been selectable, and combining the retrieved phoneme data pieces to synthesize data representing the sound piece having not been selectable, and

means for generating data representing synthesized sound by combining the sound data selected by the selecting means and the sound data synthesized by the lacked part synthesizing means.

32. The sound synthesis device according to claim 31, wherein the sound piece storing means stores actual measured rhythm data representing temporal change in pitch of the sound piece represented by sound data, in correspondence with the sound data, and

the selecting means selects from the sound data pieces, sound data which represents a waveform having the same reading as a sound piece composing the sentence, the temporal change in pitch represented by the actual measured rhythm data in correspondence with the sound data being closest to the prediction result of rhythm.

33-35. -35. (canceled)

36. A pitch waveform signal division method comprising:

acquiring a sound signal representing a waveform of sound and filtering the sound signal to extract a pitch signal;

delimiting the sound signal into sections based on the extracted pitch signal and adjusting the phase for each section based on the correlation between the section and the pitch signal;

determining a sampling length for each section with the adjusted phase based on the phase, and performing sampling with the sampling length to generate a sampling signal;

processing the sampling signal into a pitch waveform signal based on the result of the adjustment by the phase adjusting means and the value of the sampling length; and

detecting a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or an end of the sound, and dividing the pitch waveform signal at the detected boundary and/or end.

37. A pitch waveform signal division method comprising:

acquiring a sound signal representing a waveform of sound, and processing the sound signal into a pitch waveform signal by substantially equalizing the phases of sections where the sound signal is divided into the sections for a unit pitch of the sound; and

detecting a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or an end of the sound, and dividing the pitch waveform signal at the detected boundary and/or end.

38. A pitch waveform signal division method comprising:

detecting, for pitch waveform signal representing a waveform of sound, a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or an end of the sound; and

dividing the pitch waveform signal at the detected boundary and/or end.

39. A sound signal compression method comprising:

acquiring a sound signal representing a waveform of sound and filtering the sound signal to extract a pitch signal;

delimiting the sound signal into sections based on the pitch signal extracted by the filter and adjusting the phase for each section based on the correlation between the section and the pitch signal;

determining a sampling length for each section with the adjusted phase based on the phase, and performing sampling with the sampling length to generate a sampling signal;

processing the sampling signal into a pitch waveform signal based on the result of the adjustment of the phase and the value of the sampling length;

detecting a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or an end of the sound, and dividing the pitch waveform signal at the detected boundary and/or end to generate phoneme data; and

subjecting the generated phoneme data to entropy coding to perform data compression.

40. A sound signal compression method comprising:

acquiring a sound signal representing a waveform of sound, and processing the sound signal into a pitch waveform signal by substantially equalizing the phases of sections where the sound signal is divided into the sections for a unit pitch of the sound;

detecting a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or an end of the sound, and dividing the pitch waveform signal at the detected boundary and/or end to generate phoneme data; and

subjecting the generated phoneme data to entropy coding to perform data compression.

41. A sound signal compression method comprising:

detecting, for pitch waveform signal representing a waveform of sound, a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or an end of the sound;

dividing the pitch waveform signal at the detected boundary and/or end to generate phoneme data; and

subjecting the generated phoneme data to entropy coding to perform data compression.

42. A sound signal restoration method comprising:

acquiring phoneme data which is acquired by dividing a pitch waveform signal at a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and end of the sound, the pitch waveform signal being acquired by substantially equalizing the phases of sections where the sound signal representing a waveform of sound is divided into the sections for a unit pitch of the sound; and

decoding the acquired phoneme data.

43. A sound synthesis method comprising:

acquiring phoneme data which is acquired by dividing a pitch waveform signal at a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or end of the sound, the pitch waveform signal being acquired by substantially equalizing the phases of sections where the sound signal representing a waveform of sound is divided into the sections for a unit pitch of the sound;

restoring the phase of the acquired phoneme data to the phase before the process;

storing the acquired phoneme data or the phoneme data with the restored phase;

inputting sentence information representing a sentence; and

retrieving phoneme data representing waveforms of phonemes composing the sentence from the stored phoneme data, and combining the retrieved phoneme data pieces to generate data representing synthesized sound.

44. A program for making a computer act as:

a filter for acquiring a sound signal representing a waveform of sound and filtering the sound signal to extract a pitch signal;

phase adjusting means for delimiting the sound signal into sections based on the pitch signal extracted by the filter and adjusting the phase for each section based on the correlation between the section and the pitch signal;

sampling means for determining a sampling length for each section with the phase adjusted by the phase adjusting means, based on the phase, and performing sampling with the sampling length to generate a sampling signal;

sound signal processing means for processing the sampling signal into a pitch waveform signal based on the result of the adjustment by the phase adjusting means and the value of the sampling length; and

pitch waveform signal dividing means for detecting a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or an end of the sound, and dividing the pitch waveform signal at the detected boundary and/or end.

45. A program for making a computer act as:

sound signal processing means for acquiring a sound signal representing a waveform of sound, and processing the sound signal into a pitch waveform signal by substantially equalizing the phases of sections where the sound signal is divided into the sections for a unit pitch of the sound; and

pitch waveform signal dividing means for detecting a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and an end of the sound, and dividing the pitch waveform signal at the detected boundary and end.

46. A program for making a computer act as:

means for detecting, for pitch waveform signal representing a waveform of sound, a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or an end of the sound; and

means for dividing the pitch waveform signal at the detected boundary and/or end.

47. A program for making a computer act as:

a filter for acquiring a sound signal representing a waveform of sound and filtering the sound signal to extract a pitch signal;

phase adjusting means for delimiting the sound signal into sections based on the pitch signal extracted by the filter and adjusting the phase for each section based on the correlation between the section and the pitch signal;

sampling means for determining a sampling length for each section with the phase adjusted by the phase adjusting means, based on the phase, and performing sampling with the sampling length to generate a sampling signal;

sound signal processing means for processing the sampling signal into a pitch waveform signal based on the result of the adjustment by the phase adjusting means and the value of the sampling length;

phoneme data generating means for detecting a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or an end of the sound, and dividing the pitch waveform signal at the detected boundary and/or end to generate phoneme data; and

data compressing means for subjecting the generated phoneme data to entropy coding to perform data compression.

48. A program for making a computer act as:

sound signal processing means for acquiring a sound signal representing a waveform of sound, and processing the sound signal into a pitch waveform signal by substantially equalizing the phases of sections where the sound signal is divided into the sections for a unit pitch of the sound;

phoneme data generating means for detecting a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or an end of the sound, and dividing the pitch waveform signal at the detected boundary and/or end to generate phoneme data; and

data compressing means for subjecting the generated phoneme data to entropy coding to perform data compression.

49. A program for making a computer act as:

means for detecting, for pitch waveform signal representing a waveform of sound, a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or an end of the sound;

phoneme data generating means for dividing the pitch waveform signal at the detected boundary and/or end to generate phoneme data; and

data compressing means for subjecting the generated phoneme data to entropy coding to perform data compression.

50. A program for making a computer act as:

data acquiring means for acquiring phoneme data which is acquired by dividing a pitch waveform signal at a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or end of the sound, the pitch waveform signal being acquired by substantially equalizing the phases of sections where the sound signal representing a waveform of sound is divided into the sections for a unit pitch of the sound; and

restoring means for decoding the acquired phoneme data.

51. A program for making a computer act as:

data acquiring means for acquiring phoneme data which is acquired by dividing a pitch waveform signal at a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or end of the sound, the pitch waveform signal being acquired by substantially equalizing the phases of sections where the sound signal representing a waveform of sound is divided into the sections for a unit pitch of the sound;

restoring means for decoding the acquired phoneme data;

phoneme data storing means for storing the acquired phoneme data or the decoded phoneme data;

sentence input means for inputting sentence information representing a sentence; and

synthesizing means for retrieving from the phoneme data storing means, phoneme data representing waveforms of phonemes composing the sentence, and combining the retrieved phoneme data pieces to generate data representing synthesized sound.

52. A computer readable recording medium having a program recorded thereon for making a computer act as:

a filter for acquiring a sound signal representing a waveform of sound and filtering the sound signal to extract a pitch signal;

phase adjusting means for delimiting the sound signal into sections based on the pitch signal extracted by the filter and adjusting the phase for each section based on the correlation between the section and the pitch signal;

sampling means for determining a sampling length for each section with the phase adjusted by the phase adjusting means, based on the phase, and performing sampling with the sampling length to generate a sampling signal;

sound signal processing means for processing the sampling signal into a pitch waveform signal based on the result of the adjustment by the phase adjusting means and the value of the sampling length; and

pitch waveform signal dividing means for detecting a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or an end of the sound, and dividing the pitch waveform signal at the detected boundary and/or end.

53. A computer readable recording medium having a program recorded thereon for making a computer act as:

sound signal processing means for acquiring a sound signal representing a waveform of sound, and processing the sound signal into a pitch waveform signal by substantially equalizing the phases of sections where the sound signal is divided into the sections for a unit pitch of the sound; and

pitch waveform signal dividing means for detecting a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or an end of the sound, and dividing the pitch waveform signal at the detected boundary and/or end.

54. A computer readable recording medium having a program recorded thereon for making a computer act as:

means for detecting, for pitch waveform signal representing a waveform of sound, a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or an end of the sound; and

means for dividing the pitch waveform signal at the detected boundary and/or end.

55. A computer readable recording medium having a program recorded thereon for making a computer act as:

a filter for acquiring a sound signal representing a waveform of sound and filtering the sound signal to extract a pitch signal;

phase adjusting means for delimiting the sound signal into sections based on the pitch signal extracted by the filter and adjusting the phase for each section based on the correlation between the section and the pitch signal;

sampling means for determining a sampling length for each section with the phase adjusted by the phase adjusting means, based on the phase, and performing sampling with the sampling length to generate a sampling signal;

sound signal processing means for processing the sampling signal into a pitch waveform signal based on the result of the adjustment by the phase adjusting means and the value of the sampling length;

phoneme data generating means for detecting a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or an end of the sound, and dividing the pitch waveform signal at the detected boundary and/or end to generate phoneme data; and

data compressing means for subjecting the generated phoneme data to entropy coding to perform data compression.

56. A computer readable recording medium having a program recorded thereon for making a computer act as:

sound signal processing means for acquiring a sound signal representing a waveform of sound, and processing the sound signal into a pitch waveform signal by substantially equalizing the phases of sections where the sound signal is divided into the sections for a unit pitch of the sound;

phoneme data generating means for detecting a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or an end of the sound, and dividing the pitch waveform signal at the detected boundary and/or end to generate phoneme data; and

data compressing means for subjecting the generated phoneme data to entropy coding to perform data compression.

57. A computer readable recording medium having a program recorded thereon for making a computer act as:

means for detecting, for pitch waveform signal representing a waveform of sound, a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or an end of the sound;

phoneme data generating means for dividing the pitch waveform signal at the detected boundary and/or end to generate phoneme data; and

data compressing means for subjecting the generated phoneme data to entropy coding to perform data compression.

58. A computer readable recording medium having a program recorded thereon for making a computer act as:

data acquiring means for acquiring phoneme data which is acquired by dividing a pitch waveform signal at a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or end of the sound, the pitch waveform signal being acquired by substantially equalizing the phases of sections where the sound signal representing a waveform of sound is divided into the sections for a unit pitch of the sound; and

restoring means for decoding the acquired phoneme data.

59. A computer readable recording medium having a program recorded thereon for making a computer act as:

data acquiring means for acquiring phoneme data which is acquired by dividing a pitch waveform signal at a boundary of adjacent phonemes included in the sound represented by the pitch waveform signal and/or end of the sound, the pitch waveform signal being acquired by substantially equalizing the phases of sections where the sound signal representing a waveform of sound is divided into the sections for a unit pitch of the sound;

restoring means for restoring the phase of the acquired phoneme data to the phase before the process;

phoneme data storing means for storing the acquired phoneme data or the phoneme data with the restored phase;

sentence input means for inputting sentence information representing a sentence; and

synthesizing means for retrieving from the phoneme data storing means, phoneme data representing waveforms of phonemes composing the sentence, and combining the retrieved phoneme data pieces to generate data representing synthesized sound.