Information processing device, information processing method and storage medium

• US 6,509,707 B2
• Filed: 12/27/2000
• Issued: 01/21/2003
• Est. Priority Date: 12/28/1999
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Tamaru, Hideshi, Honda, Hitoshi, Lucke, Helmut, Ogawa, Hiroaki, Yamashita, Junichi, Fujita, Yaeko
• Primary Examiner(s): Masih, Karen
• Application Number:
  US09/749,360
• Publication Number:
  US 20010020837A1
• Time in Patent Office: 755 Days
• Field of Search: 318/567, 318/568.12, 318/568.13, 318/568.16, 318/569, 901/1, 706/14, 706/18, 700/31
• US Class Current: 318/567
• CPC Class Codes: B25J 13/003 by means of an audio-responsive input audible safety signals B25J19/061
• CPC Class Codes: B25J 19/023 including video camera means
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A robot performing a variety of actions is disclosed.

The voice of a user picked up by a microphone is recognized by a voice recognizer. A gesture of the user picked up by a CCD is recognized by an image recognizer. A behavior decision unit decides the behavior to be taken by the robot based on the voice information provided by the voice recognizer and the image information provided by the image recognizer.

Claims

1. An information processing device for use in a robot, comprising:

voice recognition means for recognizing a voice;

image recognizer means for recognizing an image;

decision means for deciding the behavior of the robot based on at least one of the voice recognition result provided by the voice recognizer means and the image recognition result provided by the image recognizer means;

determining means for determining, based upon said decided behavior, whether further measurement results are required; and

measuring means for measuring said further measurement results when required.

2. The information processing device of claim 1, wherein said determining means determines that said further measurement results are required when said decided behavior requires a distance to be known.

3. An information processing device according to claim 1, further comprising storage means for storing a table that describes the relationship of the voice recognition result provided by the voice recognizer means, the image recognition result provided by the image recognizer means, and the behavior of the robot that is uniquely decided in accordance with the voice recognition result and the image recognition result.

4. An information processing device according to claim 1, wherein the decision means decides the behavior of the robot in accordance with the image recognition result uniquely determined by the image recognizer means when the voice recognition is not uniquely determined by the voice recognizer means.

5. An information processing device according to claim 1, wherein the decision means decides the behavior of the robot in accordance with the voice recognition result uniquely determined by the voice recognizer means when a plurality of objects is present within the area of an image recognized by the image recognizer means.

6. An information processing device according to claim 1, wherein the image recognizer means recognizes an image of a scene which is present in a direction to which one of predetermined portions of a user, from among a finger, a face, an eye, and a jaw of the user, points.

7. An information processing device according to claim 1, further comprising storage means for storing data of gestures performed by a user, wherein the image recognizer means recognizes the image of the user to detect a gesture matching the gesture data stored in the storage means, and regards the detected gesture as an image recognition result.

8. An information processing device according to claim 1, further comprising:

detector means for detecting the face of a user; and

range finder means for measuring a distance between the user and the robot based on the size of the face of the user detected by the detector means,

wherein the decision means decides the behavior of the robot using the measured distance.

9. An information processing device according to claim 1, wherein the voice recognizer means detects a rhythm contained in background sounds, and treats the detected rhythm as a voice recognition result.

10. An information processing device according to claim 1, wherein the voice recognizer means detects an acoustic phenomenon from background sounds, and treats the detected acoustic phenomenon as a voice recognition result.

11. An information processing method of an information processing device for use in a robot, comprising the steps of:

recognizing a voice;

recognizing an image;

deciding the behavior of the robot based on at least one of the voice recognition result provided in the voice recognizing step and the image recognition result provided in the image recognizing step;

determining, based upon said decided behavior, whether further measurement results are required; and

measuring said further measuring results when required.

12. A software program for an information processing device for use in a robot, comprising:

an instruction for recognizing a voice;

an instruction for recognizing an image;

an instruction for deciding the behavior of the robot based on at least one of the voice recognition result provided in the voice recognizing step and the image recognition result provided in the image recognizing step;

an instruction for determining, based upon said decided behavior, whether further measurement results are required; and

an instruction for causing said device to measure said further measuring results when required.

13. A storage medium storing a software program for an information processing device for use in a robot, said robot comprising:

a reader for reading said software program from said storage medium;

said software program comprising;

an instruction for recognizing a voice;

an instruction for recognizing an image;

an instruction for deciding the behavior of the robot based on at least one of the voice recognition result provided in the voice recognizing step and the image recognition result provided in the image recognizing step;

an instruction for determining, based upon said decided behavior, whether further measurement results are required; and

an instruction for causing said device to measure said further measuring results when required.

14. An information processing device for use in a robot, comprising:

voice recognizing means for recognizing a voice;

image recognizing means for recognizing an image;

decision means for deciding the behavior of said robot based on said image recognition result provided by the image recognizer means;

determining means for determining whether a particular single behavior has been decided; and

if a single behavior has not been determined, then said decision means employs said voice recognition result provided by the voice recognizer means to decide on a particular behavior.

15. An information processing method for use in a robot, comprising:

voice recognizer means for recognizing a voice;

image recognizing means for recognizing an image;

decision means for deciding the behavior of said robot based on said image recognition result provided by the image recognizer means;

determining means for determining whether a particular single behavior has been decided; and

if a single behavior has not been determined, then said decision means employs said voice recognition result provided by the voice recognizer means to decide on a particular behavior.