Robot control apparatus and method with echo back prosody

• US 7,203,642 B2
• Filed: 10/10/2001
• Issued: 04/10/2007
• Est. Priority Date: 10/11/2000
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Toyoda, Takashi, Hiroi, Jun, Ishii, Kazuo, Onogi, Wataru
• Primary Examiner(s): Lerner; Martin
• Application Number:
  US10/148,914
• Publication Number:
  US 20030055653A1
• Time in Patent Office: 2,008 Days
• Field of Search: 704/258, 704/260, 704/266, 704/268, 704/269, 704/272, 704/275, 704/231, 446/297, 446/301
• US Class Current: 704/231
• CPC Class Codes: A63H 11/00 Self-movable toy figures
• CPC Class Codes: A63H 2200/00 Computerized interactive toys, e.g. dolls
• CPC Class Codes: B25J 13/003 by means of an audio-responsive input audible safety signals B25J19/061
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 13/04 Details of speech synthesis systems, e.g. synthesiser structure or memory management
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/225 Feedback of the input speech

Abstract

The present invention relates to robot control apparatuses which make a user recognize that speech is accepted. A user'"'"'s speech input to a microphone 15 is subjected to speech recognition by a speech recognizer 50A. Based on the speech recognition result, an action determining unit 52 determines subsequent actions to be taken by a robot. At the same time, an echo back unit 56 analyzes prosodic information of the user'"'"'s speech input to the microphone 15 and, based on the prosodic information, generates echo back speech which is prosody-controlled speech. The echo back speech is supplied to a loudspeaker 18 through an output controller 57 and is output from the loudspeaker 18.

Claims

1. A robot control apparatus for controlling a robot which acts at least based on a speech recognition result, comprising:

analyzing means for analyzing prosodic information of a user'"'"'s speech which is subjected to speech recognition, the prosodic information indicating emotions of the user'"'"'s speech;

generating means for generating, based on the prosodic information, prosody-controlled speech data which is speech data in which prosody is controlled; and

output control means to control echo back of the prosodic-controlled speech data to the user,

wherein the echo back of the prosodic-controlled speech data is the user'"'"'s speech from which phonemes have been removed and replaced with a voice of the robot containing substantially the same emotions as indicated by the prosodic information.

2. A robot control apparatus according to claim 1, wherein the generating means generates the prosodic-controlled speech data without any phonemes.

3. A robot control apparatus according to claim 1, further comprising:

model storage means for storing models representing emotional, instinctual, or growth states of the robot,

wherein the generating means generates the prosody-controlled speech data further based on the emotional, instinctual, or growth states of the robot, the states being represented by the models stored in the model storage means.

4. A robot control apparatus according to claim 1, further comprising:

data storage means for storing the prosody-controlled speech data,

wherein the output control means outputs the prosody-controlled speech data stored in the data storage means when a predetermined event occurs.

5. A robot control apparatus according to claim 1, further comprising:

action control means for controlling the actions of the robot,

wherein the output control means limits the outputting of the prosody-controlled speech data when speech is output under the control of the action control means.

6. A robot control apparatus according to claim 1, further comprising:

action control means for causing the robot to move its mouth in synchronization with the outputting of the prosody-controlled speech data.

7. A robot control apparatus according to claim 6, wherein the action control means controls the movement of the robot'"'"'s mouth based on the prosody-controlled speech data.

8. A robot control method for controlling a robot which acts at least based on a speech recognition result, comprising:

an analyzing step of analyzing prosodic information of a user'"'"'s speech which is subjected to speech recognition, the prosodic information indicating emotions of the user'"'"'s speech;

a generating step of generating, based on the prosodic information, prosody-controlled speech data which is speech data in which prosody is controlled; and

an output control step of controlling echo back of the prosody-controlled speech data to the user,

wherein the echo back of the prosodic-controlled speech data is the user'"'"'s speech from which phonemes have been removed and replaced with a voice of the robot containing substantially the same emotions as indicated by the prosodic information.

9. A recording medium having recorded therein a program for causing a computer to perform a robot control process for controlling a robot which acts at least based on a speech recognition result, the program comprising:

an analyzing step of analyzing prosodic information of a user'"'"'s speech which is subjected to speech recognition, the prosodic information indicating emotions of the user'"'"'s speech;

a generating step of generating, based on the prosodic information, prosody-controlled speech data which is speech data in which prosody is controlled; and

an output control step of controlling echo back of the prosody-controlled speech data to the user,

wherein the echo back of the prosodic-controlled speech data is the user'"'"'s speech from which phonemes have been removed and replaced with a voice of the robot containing substantially the same emotions as indicated by the prosodic information.

10. A program for causing a computer to perform a robot control process for controlling a robot which acts at least based on a speech recognition result, comprising:

analyzing step of analyzing prosodic information of a user'"'"'s speech which is subjected to speech recognition, the prosodic information indicating emotions of the user'"'"'s speech;

a generating step of generating, based on the prosodic information, prosody-controlled speech data which is speech data in which prosody is controlled; and

an output control step of controlling echo back of the prosody-controlled speech data to the user

wherein the echo back of the prosodic-controlled speech data is the user'"'"'s speech from which phonemes have been removed and replaced with a voice of the robot containing substantially the same emotions as indicated by the prosodic information.