SOUND SYNTHESIS DEVICE, SOUND SYNTHESIS METHOD AND STORAGE MEDIUM

• US 20160180833A1
• Filed: 12/15/2015
• Published: 06/23/2016
• Est. Priority Date: 12/22/2014
• Status: Active Grant

• Current Assignee: Casio Computer Company Limited
• Original Assignee: Casio Computer Company Limited
• Inventors: TANAKA, Hyuta
• Granted Patent: US 9,805,711 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G10L 13/0335 Pitch control
• CPC Class Codes: G10L 13/10 Prosody rules derived from text; Stress or intonation

Abstract

A sound synthesis device that includes a processor configured to perform the following: extracting intonation information from prosodic information contained in sound data and digitally smoothing the extracted intonation information to obtain smoothed intonation information; obtaining a plurality of digital sound units based on text data and concatenating the plurality of digital sound units so as to construct a concatenated series of digital sound units that corresponds to the text data; and modifying the concatenated series of digital sound units in accordance with the smoothed intonation information with respect to at least one of parameters of the concatenated series of digital sound units to generate synthesized sound data corresponding to the text data.

Claims

1. A sound synthesis device, comprising a processor configured to perform the following:

extracting intonation information from prosodic information contained in sound data and digitally smoothing the extracted intonation information to obtain smoothed intonation information;

obtaining a plurality of digital sound units based on text data and concatenating the plurality of digital sound units so as to construct a concatenated series of digital sound units that corresponds to the text data; and

modifying the concatenated series of digital sound units in accordance with the smoothed intonation information with respect to at least one of parameters of the concatenated series of digital sound units to generate synthesized sound data corresponding to the text data.

2. The sound synthesis device according to claim 1, wherein said processor concatenates the plurality of digital sound units to construct the concatenated series of digital sound units that meets a prescribed matching condition with respect to the text data.

3. The sound synthesis device according to claim 2,

wherein the sound data represents speech by a user.

4. The sound synthesis device according to claim 1, wherein said one of the parameters of the concatenated series of digital sound units is a pitch sequence, and said processor modifies the pitch sequence in the concatenated series of digital sound units so as to substantially match the smoothed intonation information.

5. The sound synthesis device according to claim 1, wherein, in obtaining said smoothed intonation information, said processor smoothes a pitch sequence in the prosodic information contained in the sound data, and extracts as said smoothed intonation information the pitch sequence that has been smoothed.

6. The sound synthesis device according to claim 5, wherein, in smoothing said pitch sequence in the prosodic information, said processor quantizes pitches of the pitch sequence in the prosodic information contained in the sound data, and smoothes the pitch sequence by acquiring a weighted moving average of the quantized pitches.

7. The sound synthesis device according to claim 4, wherein, in modifying the pitch sequence, said processor adjusts respective time scales of a pitch sequence in the prosodic information included in the sound data and of said pitch sequence in the concatenated series of digital sound units, and adjusts at least one of the pitch sequence in the prosodic information and the pitch sequence in the concatenated series of digital sound units so that periods during which pitches exist substantially match with each other.

8. The sound synthesis device according to claim 1, wherein said one of the parameters of the concatenated series of digital sound units is a power sequence, and said processor modifies the power sequence in the concatenated series of digital sound units so as to substantially match the smoothed intonation information.

9. The sound synthesis device according to claim 8, wherein, in obtaining said smoothed intonation information, said processor smoothes a power sequence in the prosodic information included in the sound data and extracts the smoothed power sequence as the smoothed intonation information, and

wherein, in modifying the power sequence in the concatenated series of digital sound units, said processor smoothes the power sequence in the concatenated series of digital sound units, acquires a sequence of ratios between the smoothed power sequence in the concatenated series of digital sound units and the smoothed power sequence in the prosodic information, and corrects the smoothed power sequence in the concatenated series of digital sound units in accordance with said sequence of ratios.

10. The sound synthesis device according to claim 9, wherein said processor smoothes the power sequence in the prosodic information by acquiring a weighted average of respective powers in the power sequence in the prosodic information.

11. The sound synthesis device according to claim 9, wherein, in modifying the power sequence in the concatenated series of digital sound units, said processor adjusts respective time scales of the power sequence in the prosodic information included in the sound data and of the power sequence in the concatenated series of digital sound units.

14. The sound synthesis device according to claim 1, wherein said processor is further configured to perform the following:

receiving the sound data;

receiving the text data; and

causing a database to be accessed to obtain said plurality of digital sound units.

12. A method of synthesizing sound performed by a processor in a sound synthesis device, the method comprising:

extracting intonation information from prosodic information contained in sound data and digitally smoothing the extracted intonation information to obtain smoothed intonation information;

obtaining a plurality of digital sound units based on text data and concatenating the plurality of digital sound units so as to construct a concatenated series of digital sound units that corresponds to the text data; and

modifying the concatenated series of digital sound units in accordance with the smoothed intonation information with respect to at least one of parameters of the concatenated series of digital sound units to generate synthesized sound data corresponding to the text data.

15. The method of synthesizing sound according to claim 12, further comprising:

receiving the sound data;

receiving the text data; and

causing a database to be accessed to obtain said plurality of digital sound units.

13. A non-transitory storage medium that stores instructions executable by a processor included in a sound synthesis device, said instructions causing the processor to perform the following:

extracting intonation information from prosodic information contained in sound data and digitally smoothing the extracted intonation information to obtain smoothed intonation information;

obtaining a plurality of digital sound units based on text data and concatenating the plurality of digital sound units so as to construct a concatenated series of digital sound units that corresponds to the text data; and

modifying the concatenated series of digital sound units in accordance with the smoothed intonation information with respect to at least one of parameters of the concatenated series of digital sound units to generate synthesized sound data corresponding to the text data.

16. The non-transitory storage medium according to claim 13, wherein said instructions cause the processor to further perform the following:

receiving the sound data;

receiving the text data; and

causing a database to be accessed to obtain said plurality of digital sound units.