Speech Synthesizing Device, Speech Synthesizing Method, and Program

• US 20100145706A1
• Filed: 02/01/2007
• Published: 06/10/2010
• Est. Priority Date: 02/08/2006
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Kato, Masanori
• Granted Patent: US 8,209,180 B2
• Time in Patent Office: Days
• US Class Current: 704/266
• CPC Class Codes: G10H 2240/081 Genre classification, i.e. descriptive metadata for classification or selection of musical pieces according to style analysis genre G10H2210/036
• CPC Class Codes: G10H 2250/455 Gensound singing voices, i.e. generation of human voices for musical applications, vocal singing sounds or intelligible words at a desired pitch or with desired vocal effects, e.g. by phoneme synthesis formant synthesis G10H2250/481; parcor synthesis G10H2250/505; modulation effects G10H2210/195; ensemble effects G10H2210/245; speech synthesis in general G10L13/00
• CPC Class Codes: G10L 13/10 Prosody rules derived from text; Stress or intonation

Abstract

An object of the present invention is to provide a device and a method for generating a synthesized speech that has an utterance form that matches music. A musical genre estimation unit of the speech synthesizing device estimates the musical genre to which a received music signal belongs, an utterance form selection unit references an utterance form information storage unit to determine an utterance form from the musical genre. A prosody generation unit references a prosody generation rule storage unit, selected from prosody generation rule storage units 15₁ to 15_N according to the utterance form, and generates prosody information from a phonetic symbol sequence. A unit waveform selection unit references a unit waveform data storage unit, selected from unit waveform data storage units 16₁ to 16_N according to the utterance form, and selects a unit waveform from the phonetic symbol sequence and the prosody information. A waveform generation unit generates a synthesized speech waveform from the prosody information and the unit waveform data.

Claims

1. A speech synthesizing device comprising:

an utterance form selection unit that analyzes a music signal reproduced in a user environment and determines an utterance form that matches an analysis result of the music signal; and

a speech synthesizing unit that synthesizes a speech according to the utterance form wherein

said speech synthesizing device automatically selects an utterance form according to music reproduced in a user environment.

2. The speech synthesizing device as defined by claim 1 wherein said speech synthesizing unit comprises:

a prosody generation unit that generates prosody information according to the utterance form; and

a unit waveform selection unit that selects a unit waveform according to the utterance form.

3. The speech synthesizing device as defined by claim 1 wherein said speech synthesizing unit comprises:

prosody generation rule storage units that store prosody generation rules, one for each utterance form;

unit waveform storage units that store unit waveforms, one for each utterance form;

a prosody generation unit that references a prosody generation rule selected according to the utterance form and generates prosody information from a phonetic symbol sequence;

a unit waveform selection unit that selects a unit waveform from the unit waveforms stored in said unit waveform storage units according to the phonetic symbol sequence and the prosody information; and

a waveform generation unit that synthesizes the unit waveform according to the prosody information and generates a synthesized speech waveform.

4. The speech synthesizing device as defined by claim 1, further comprising:

a music attribute information search unit that searches a music attribute information storage unit, in which a correspondence between music and attribute thereof is stored, for data corresponding to an analysis result of a received music signal and estimates an attribute of the received music wherein

said utterance form selection unit selects an utterance form according to the attribute of the received music to determine the utterance form.

5. The speech synthesizing device as defined by claim 1, further comprising:

a musical genre estimation unit that analyzes the music signal and estimates a musical genre to which the music belongs wherein

said utterance form selection unit selects an utterance form according to the musical genre to determine the utterance form.

8. The speech synthesizing device as defined by claim 1, further comprising:

a synthesized speech power adjustment unit that adjusts a power of the synthesized speech waveform, generated according to the utterance form, according to a power of the music signal.

9. The speech synthesizing device as defined by claim 1, further comprising:

a music signal power calculation unit that analyzes the music signal and calculates a power of the music signal;

a synthesized speech power calculation unit that analyzes the synthesized speech waveform and calculates a power of the synthesized speech; and

a synthesized speech power adjustment unit that references a ratio predetermined for each utterance form between a power of the music signal and a power of the synthesized speech and adjusts a power of the synthesized speech waveform, generated according to the utterance form, according to the power of the music signal.

6. (canceled)

7. (canceled)

10. A speech synthesizing method that generates a synthesized speech using a speech synthesizing device, said method comprising:

analyzing, by said speech synthesizing device, a music signal reproduced in a user environment and determining an utterance form that matches an analysis result of the music signal; and

synthesizing, by said speech synthesizing device, a speech according to the utterance form.

11. The speech synthesizing method as defined by claim 10, further comprising:

generating, by said speech synthesizing device, prosody information according to the utterance form; and

selecting, by said speech synthesizing device, a unit waveform according to the utterance form wherein

said speech synthesizing device uses the prosody information and the unit waveform to synthesize a speech.

12. The speech synthesizing method as defined by claim 10 wherein

said synthesizing, by said speech synthesizing device, a speech according to the utterance form comprises:

referencing, by said speech synthesizing device, a prosody generation rule selected from prosody generation rules, which are stored in prosody generation rule storage units, according to the utterance form and generating prosody information from a phonetic symbol sequence;

selecting, by said speech synthesizing device, a unit waveform from unit waveforms, which are prepared for each said utterance form, according to the phonetic symbol sequence and the prosody information; and

synthesizing, by said speech synthesizing device, the unit waveform according to the prosody information and generating a synthesized speech waveform.

13. The speech synthesizing method as defined by claim 10, further comprising:

searching, by said speech synthesizing device, a music attribute information storage unit, in which a correspondence between music and attribute thereof is stored, for data corresponding to an analysis result of the received music signal and estimating an attribute of the received music, wherein

an utterance form is selected according to the attribute of the received music signal to determine the utterance form that matches the analysis result of the music signal.

14. The speech synthesizing method as defined by claim 10, further comprising:

analyzing, by said speech synthesizing device, the music signal and estimating a musical genre to which the music belongs; and

selecting, by said speech synthesizing device, an utterance form according to the musical genre to determine the utterance form that matches the analysis result of the music signal.

17. The speech synthesizing method as defined by claim 10, further comprising:

adjusting, by said speech synthesizing device, a power of the synthesized speech waveform, generated according to the utterance form, according to a power of the music signal.

18. The speech synthesizing method as defined by claim 10, further comprising:

analyzing, by said speech synthesizing device, the music signal and calculating a power of the music signal;

analyzing, by said speech synthesizing device, the synthesized speech waveform and calculating a power of the synthesized speech; and

referencing, by said speech synthesizing device, a ratio predetermined for each utterance form between a power of the music signal and a power of the synthesized speech and adjusting a power of the synthesized speech waveform, generated according to the utterance form, according to the power of the music signal.

15. (canceled)

16. (canceled)

19. A program causing a computer, which constitutes a speech synthesizing device, to execute:

processing for analyzing a received music signal reproduced in a user environment and determining an utterance form, which matches an analysis result of the music signal, from utterance forms prepared in advance; and

processing for synthesizing a speech according to the utterance form.

20. The program as defined by claim 19, further comprising:

processing for generating prosody information according to the utterance form;

processing for selecting a unit waveform according to the utterance form; and

, after that,

processing for synthesizing a speech using the prosody information and the unit waveform.

21. The program as defined by claim 19, further comprising:

processing for referencing a prosody generation rule selected from prosody generation rules, which are stored in prosody generation rule storage units connected to said computer, according to the utterance form and generating prosody information from a phonetic symbol sequence;

processing for selecting a unit waveform from unit waveforms prepared in unit waveform storage units, connected to said computer, for each said utterance form according to the phonetic symbol sequence and the prosody information; and

, after that,

processing for synthesizing the unit waveform according to the prosody information and synthesizing a speech.

22. The program as defined by claim 19, further comprising:

processing for searching a music attribute information storage unit, in which a correspondence between music and attribute thereof is stored, for data corresponding to an analysis result of the received music signal and estimating an attribute of the received music wherein

an utterance form is selected according to the attribute of the received music to determine the utterance form that matches the analysis result of the music signal.

23. The program as defined by claim 19, further comprising:

processing for analyzing the music signal and estimating a musical genre to which the music belongs; and

processing for selecting an utterance form according to the musical genre to determine the utterance form that matches the analysis result of the music signal.

26. The program as defined by claim 19, further comprising:

processing for adjusting a power of the synthesized speech waveform, generated according to the utterance form, according to a power of the music signal.

27. The program as defined by claim 19 further comprising:

processing for analyzing the music signal and calculating a power of the music signal;

processing for analyzing the synthesized speech waveform and calculating a power of the synthesized speech; and

processing for referencing a ratio predetermined for each utterance form between a power of the music signal and a power of the synthesized speech and adjusting a power of the synthesized speech waveform, generated according to the utterance form, according to the power of the music signal.

24. (canceled)

25. (canceled)