Robot behavior control system and method, and robot apparatus

• US 8,145,492 B2
• Filed: 04/06/2005
• Issued: 03/27/2012
• Est. Priority Date: 04/07/2004
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Fujita, Masahiro
• Primary Examiner(s): COLUCCI, MICHAEL C
• Application Number:
  US11/101,257
• Publication Number:
  US 20050240412A1
• Time in Patent Office: 2,547 Days
• Field of Search: 704/232, 704/257, 704/258, 704/275, 704/500, 704/203, 704/219, 704/221, 704/223, 704/230, 704/235, 704/254, 704/260, 704/266, 704/267, 704/270.1, 323/2, 325/7, 325/8, 706/21, 706/10
• US Class Current: 704/270
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/16 using artificial neural networks
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units

Abstract

A behavior control system of a robot for learning a phoneme sequence includes a sound inputting device inputting a phoneme sequence, a sound signal learning unit operable to convert the phoneme sequence into a sound synthesis parameter and to learn or evaluate a relationship between a sound synthesis parameter of a phoneme sequence that is generated by the robot and a sound synthesis parameter used for sound imitation, and a sound synthesizer operable to generate a phoneme sequence based on the sound synthesis parameter obtained by the sound signal learning unit.

Claims

1. A voice control system, of a robot, for learning a phoneme sequence, comprising:

sound inputting means for inputting a phoneme sequence, the sound inputting means including a cochlear filter bank;

sound signal learning means for converting the input phoneme sequence into a first sound synthesis parameter in which speaker emotional individuality is reflected and for evaluating and learning a difference between (a) a second sound synthesis parameter of a robot phoneme sequence that is generated by the robot by imitating the input phoneme sequence, the second sound synthesis parameter generated based upon an internal state of the robot determined from self-analysis of the robot'"'"'s voice via the cochlear filter bank of the sound inputting means and (b) the first sound synthesis parameter, of the speaker, which is used for sound imitation; and

sound synthesizing means for generating the robot phoneme sequence which imitates the input phoneme sequence of the speaker, the robot phoneme sequence to be output based on the first and second sound synthesis parameters obtained by the sound signal learning means, wherein the robot phoneme sequence to be output is pronounced with different characteristics based on the internal state of the robot, the internal state including emotions such as instincts and feelings, and wherein the robot phoneme sequence output by the sound synthesizing means is input to the cochlear filter bank of the sound inputting means to generate the second sound synthesis parameter for self-analysis of the robot'"'"'s voice.

2. The system according to claim 1, wherein the sound signal learning means is formed of a feed-forward neural network.

3. The system according to claim 1, further comprising:

internal state generating means for generating the internal state of the robot due to a body constraint of the robot; and

mapping means for mapping the internal state to a sound synthesis parameter.

4. The system according to claim 3, wherein the mapping means inversely estimates the internal state corresponding to an input sound synthesis parameter using an inverse function during mapping.

5. A voice control method, of a robot, for learning a phoneme sequence, comprising the steps of:

inputting a phoneme sequence at an input unit, including a cochlear filter bank;

converting the input phoneme sequence into a first sound synthesis parameter in which speaker emotional individuality is reflected, and evaluating and learning a difference between (a) a second sound synthesis parameter of a robot phoneme sequence that is generated by the robot by imitating the input phoneme sequence, the second sound synthesis parameter generated based upon an internal state of the robot determined from self-analysis of the robot'"'"'s voice via the cochlear filter bank of the input unit and (b) the first sound synthesis parameter, of the speaker, which is used for sound imitation; and

generating the robot phoneme sequence which imitates the speaker, the robot phoneme sequence to be output based on the first and second sound synthesis parameters, wherein the robot phoneme sequence to be output is pronounced with different characteristics based on the internal state of the robot, the internal state including emotions such as instincts and feelings, and wherein the output robot phoneme sequence is fed back to the cochlear filter bank of the input unit to generate the second sound synthesis parameter used for self analysis of the robot'"'"'s voice.

6. The method according to claim 5, wherein in the step of learning or evaluating, learning based on a feed-forward neural network is performed.

7. The method according to claim 5, further comprising the steps of:

generating the internal state of the robot due to a body constraint of the robot; and

mapping the internal state to a sound synthesis parameter.

8. The method according to claim 5, further comprising the step of inversely estimating the internal state corresponding to an input sound synthesis parameter using an inverse function during mapping performed in the step of mapping.

9. A robot apparatus for learning a phoneme sequence, comprising:

a plurality of movable sections;

driving means for driving the plurality of movable sections;

driving control means for controlling the driving means;

sound inputting means for inputting a phoneme sequence, the sound inputting means including a cochlear filter bank;

sound signal learning means for converting the input phoneme sequence into a first sound synthesis parameter in which speaker emotional individuality is reflected and for evaluating and learning a difference between (a) a second sound synthesis parameter of a robot phoneme sequence that is generated by the robot apparatus imitating the input phoneme sequence, the second sound synthesis parameter generated based upon an internal state of the robot determined from self-analysis of the robot'"'"'s voice via the cochlear filter bank of the sound inputting means and (b) the first sound synthesis parameter, of the speaker, which is used for sound imitation; and

sound synthesizing means for generating the robot phoneme sequence which imitates the input phoneme sequence of the speaker, the robot phoneme sequence to be output based on the first and second sound synthesis parameters obtained by the sound signal learning means, wherein the robot phoneme sequence to be output is pronounced with different characteristics based on the internal state of the robot, the internal state including emotions such as instincts and feelings, and wherein the robot phoneme sequence output by the sound synthesizing means is input to the cochlear filter bank of the sound inputting means to generate the second sound synthesis parameter for self-analysis of the robot'"'"'s voice.

10. A voice control system, of a robot, for learning a phoneme sequence, comprising:

a sound inputting device inputting a phoneme sequence, the sound inputting device including a cochlear filter bank;

a sound signal learning unit operable to convert the input phoneme sequence into a first sound synthesis parameter in which speaker emotional individuality is reflected and to evaluate and to learn a difference between (a) a second sound synthesis parameter of a robot phoneme sequence that is generated by the robot by imitating the input phoneme sequence, the second sound synthesis parameter generated based upon an internal state of the robot determined from self-analysis of the robot'"'"'s voice via the cochlear filter bank of the sound inputting device and (b) the first sound synthesis parameter, of the speaker, which is used for sound imitation; and

a sound synthesizer operable to generate the robot phoneme sequence which imitates the input phoneme sequence of the speaker, the robot phoneme sequence to be output based on the first and second sound synthesis parameters obtained by the sound signal learning unit, wherein the robot phoneme sequence to be output is pronounced with different characteristics based on the internal state of the robot, the internal state including emotions such as instincts and feelings, and wherein the robot phoneme sequence output by the sound synthesizer is input to the cochlear filter bank of the sound inputting device to generate the second sound synthesis parameter for self-analysis of the robot'"'"'s voice.

11. A robot apparatus for learning a phoneme sequence, comprising:

a plurality of movable sections;

a driving unit driving the plurality of movable sections;

a driving control unit controlling the driving unit;

a sound inputting device inputting a phoneme sequence, the sound inputting device including a cochlear filter bank;

a sound signal learning unit operable to convert the input phoneme sequence into a first sound synthesis parameter in which speaker emotional individuality is reflected and to evaluate and to learn a difference between (a) a second sound synthesis parameter of a robot phoneme sequence that is generated by the robot apparatus by imitating the input phoneme sequence, the second sound synthesis parameter generated based upon an internal state of the robot determined from self-analysis of the robot'"'"'s voice via the cochlear filter bank of the sound inputting device and (b) the first sound synthesis parameter, of the speaker, which is used for sound imitation; and

a sound synthesizer operable to generate the robot phoneme sequence which imitates the input phoneme sequence of the speaker, the robot phoneme sequence to be output based on the first and second sound synthesis parameters obtained by the sound signal learning unit, wherein the robot phoneme sequence to be output is pronounced with different characteristics based on the internal state of the robot, the internal state including emotions such as instincts and feelings, and wherein the robot phoneme sequence output by the sound synthesizer is input to the cochlear filter bank of the sound inputting device to generate the second sound synthesis parameter for self-analysis of the robot'"'"'s voice.