Synchronization control apparatus and method, and recording medium

• US 20010007096A1
• Filed: 12/27/2000
• Published: 07/05/2001
• Est. Priority Date: 12/28/1999
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Yamada, Keiichi, Kobayashi, Erika, Shimakawa, Masato, Akabane, Makoto, Kobayashi, Kenichiro, Nitta, Tomoaki, Yamazaki, Nobuhide
• Granted Patent: US 6,865,535 B2
• Time in Patent Office: Days
• US Class Current: 704/258
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 2021/105 Synthesis of the lips movements from speech, e.g. for talking heads

Abstract

In a synchronization control apparatus, a voice-language-information generating section generates the voice language information of a word which a robot utters. A voice synthesizing section calculates phoneme information and a phoneme continuation duration according to the voice language information, and also generates synthesized-voice data according to an adjusted phoneme continuation duration. An articulation-operation generating section calculates an articulation-operation period according to the phoneme information. A voice-operation adjusting section adjusts the phoneme continuation duration and the articulation-operation period. An articulation-operation executing section operates an organ of articulation according to the adjusted articulation-operation period.

Claims

1. A synchronization control apparatus for synchronizing the output of a voice signal and the operation of a movable portion, comprising:

phoneme-information generating means for generating phoneme information formed of a plurality of phonemes by using language information;

calculation means for calculating a phoneme continuation duration according to the phoneme information generated by the phoneme-information generating means;

computing means for computing the operation period of the movable portion according to the phoneme information generated by the phoneme-information generating means;

adjusting means for adjusting the phoneme continuation duration calculated by the calculation means and the operation period computed by the computing means;

synthesized-voice-information generating means for generating synthesized-voice information according to the phoneme continuation duration adjusted by the adjusting means;

synthesizing means for synthesizing the voice signal according to the synthesized-voice information generated by the synthesized-voice-information generating means; and

operation control means for controlling the operation of the movable portion according to the operation period adjusted by the adjusting means.

2. A synchronization control apparatus according to

claim 1

, wherein the adjusting means compares the phoneme continuation duration and the operation period corresponding to each of the phonemes and performs adjustment by substituting whichever is the longer for the shorter.

3. A synchronization control apparatus according to

claim 1

, wherein the adjusting means performs adjustment by synchronizing at least one of the start timing and the end timing, of the phoneme continuation duration and the operation period corresponding to any of the phonemes.

4. A synchronization control apparatus according to

claim 1

, wherein the adjusting means performs adjustment by substituting one of the phoneme continuation duration and the operation period corresponding to all of the phonemes, for the other.

5. A synchronization control apparatus according to

claim 1

, wherein the adjusting means performs adjustment by synchronizing at least one of the start timing and the end timing, of the phoneme continuation duration and the operation period corresponding to each of the phonemes, and by placing no-process periods at lacking intervals.

6. A synchronization control apparatus according to

claim 1

, wherein the adjusting means compares the phoneme continuation duration and the operation period corresponding to all of the phonemes and performs adjustment by extending whichever is the shorter in proportion.

7. A synchronization control apparatus according to

claim 1

, wherein the operation control means controls the operation of the movable portion which imitates the operation of an organ of articulation of an animal.

8. A synchronization control apparatus according to

claim 1

, further comprising detection means for detecting an external force operation applied to the movable portion.

9. A synchronization control apparatus according to

claim 8

, wherein at least one of the synthesizing means and the operation control means changes a process currently being executed, in response to a detection result obtained by the detection means.

10. A synchronization control apparatus according to

claim 1

, wherein the synchronization control apparatus is a robot.

11. A synchronization control method of synchronizing the output of a voice signal and the operation of a movable portion, comprising:

a phoneme-information generating step of generating phoneme information formed of a plurality of phonemes by using language information;

a calculation step of calculating a phoneme continuation duration according to the phoneme information generated in the phoneme-information generating step;

a computing step of computing the operation period of the movable portion according to the phoneme information generated in the phoneme-information generating step;

an adjusting step for adjusting the phoneme continuation duration calculated in the calculation step and the operation period computed in the computing step;

a synthesized-voice-information generating step of generating synthesized-voice information according to the phoneme continuation duration adjusted in the adjusting step;

a synthesizing step of synthesizing the voice signal according to the synthesized-voice information generated in the synthesized-voice-information generating step; and

an operation control step of controlling the operation of the movable portion according to the operation period adjusted in the adjusting step.

12. A recording medium storing a computer-readable program for synchronizing the output of a voice signal and the operation of a movable portion, the program comprising:

a phoneme-information generating step of generating phoneme information formed of a plurality of phonemes by using language information;

a calculation step of calculating a phoneme continuation duration according to the phoneme information generated in the phoneme-information generating step;

a computing step of computing the operation period of the movable portion according to the phoneme information generated in the phoneme-information generating step;

an adjusting step for adjusting the phoneme continuation duration calculated in the calculation step and the operation period computed in the computing step;

a synthesized-voice-information generating step of generating synthesized-voice information according to the phoneme continuation duration adjusted in the adjusting step;

a synthesizing step of synthesizing the voice signal according to the synthesized-voice information generated in the synthesized-voice-information generating step; and

an operation control step of controlling the operation of the movable portion according to the operation period adjusted in the adjusting step.