Speech communication system and method, and robot apparatus

• US 8,209,179 B2
• Filed: 07/02/2004
• Issued: 06/26/2012
• Est. Priority Date: 07/03/2003
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Aoyama, Kazumi, Shimomura, Hideki
• Primary Examiner(s): COLUCCI, MICHAEL C
• Application Number:
  US10/882,360
• Publication Number:
  US 20050043956A1
• Time in Patent Office: 2,916 Days
• Field of Search: 702/188, 702/153, 318/567, 704/270, 704/275, 704/276, 704/206, 701/301, 700/245, 382/118, 382/107, 382/103
• US Class Current: 704/257
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

This invention realizes a speech communication system and method, and a robot apparatus capable of significantly improving entertainment property. A speech communication system with a function to make conversation with a conversation partner is provided with a speech recognition means for recognizing speech of the conversation partner, a conversation control means for controlling conversation with the conversation partner based on the recognition result of the speech recognition means, an image recognition means for recognizing the face of the conversation partner, and a tracking control means for tracing the existence of the conversation partner based on one or both of the recognition result of the image recognition means and the recognition result of the speech recognition means. The conversation control means controls conversation so as to continue depending on tracking of the tracking control means.

Claims

1. A speech communication system enabling a conversation with a conversation partner, said system comprising:

a speech recognition unit configured to recognize a speech content of the conversation partner;

a conversation control unit configured to continue the conversation using available resources;

an image recognition unit configured to recognize a face of the conversation partner; and

a tracking control unit configured to, in a normal mode, track a position of the conversation partner in response to both of a recognition result of the image recognition unit and a recognition result of the speech recognition unit, wherein

while the speech communication system is in a failure mode as a result of an inability of the tracking control unit to track the position of the conversation partner, the conversation control unit is configured to continue the conversation without tracking including behaviors relying, in the normal mode, on tracking performed by the tracking control unit, the conversation is continued, without tracking, through the speech content of the conversation partner being recognized with the speech recognition unit and matching a predicted response.

2. The speech communication system according to claim 1, wherein upon the failure to track the position of the conversation partner by the tracking control unit and the speech contents of the conversation partner not matching the predicted response,

a notification is issued to request the conversation partner to make a response to said notification,

the notification is repeated a prescribed number of times when the response to said notification is not received from the conversation partner, and

the conversation control unit is configured to terminate the conversation when the response to said notification is not received from the conversation partner after the notification is repeated the prescribed number of times.

3. The speech communication system according to claim 2, wherein upon a failure to receive the response to said notification from the conversation partner after a prescribed period of time,

the conversation control unit is configured to issue the notification to request the conversation partner to make the response to said notification.

4. The speech communication system according to claim 1, further comprising:

a contact recognition unit configured to recognize a physical contact of the conversation partner with the contact recognition unit, wherein

the tracking control unit is configured to track the position of the conversation partner based on a combination of the recognition result of the speech recognition unit, the recognition result of the image recognition unit, and a recognition result of the contact recognition unit.

5. The speech communication system according to claim 4, wherein upon the failure to track the position of the conversation partner by the tracking control unit,

the conversation control unit is configured to continue the conversation when the recognition result of the contact recognition unit indicates the physical contact of the conversation partner with the contact recognition unit.

6. The speech communication system according to claim 4, wherein upon the failure to track the position of the conversation partner by the tracking control unit, the speech contents of the conversation partner not matching the predicted response, and the response to said notification is not received after the notification is repeated the prescribed number of times,

another notification is issued to request the conversation partner to physically contact the contact recognition unit, and

the conversation control unit is configured to continue the conversation based on the recognition result of the contact recognition unit indicating that the conversation partner physically contacted the contact recognition unit.

7. The speech communication system according to claim 1, wherein the tracking control unit utilizes at least one audio sensing device which is not utilized by the speech recognition unit.

8. A speech communication method for enabling a conversation with a conversation partner, said method comprising:

tracking a position of the conversation partner, in a normal mode, in response to both of a speech recognition result obtained by recognizing speech of the conversation partner and a image recognition result obtained by recognizing a face of the conversation partner; and

continuing the conversation with the conversation partner without tracking including behaviors relying, in the normal mode, on tracking performed by the tracking, the conversation is continued, without tracking, through the speech content of the conversation partner being recognized and matching a predicted response while the speech communication system is in a failure mode as a result of an inability to track the position of the conversation partner.

9. The speech communication method according to claim 8, further comprising:

issuing a notification to request the conversation partner to make a response to said notification upon the failure to track the position of the conversation partner by the tracking control unit and the speech contents of the conversation partner not matching the predicted response;

repeating the notification a prescribed number of times when the response to said notification is not received from the conversation partner; and

terminating the conversation when the response to said notification is not received from the conversation partner after the notification is repeated the prescribed number of times.

10. The speech communication method according to claim 9, wherein,

the notification to request the conversation partner to make the response to said notification is issued upon a failure to receive the response to said notification from the conversation partner after a prescribed period of time.

11. The speech communication method according to claim 8, wherein,

the tracking the position of the conversation partner is further based on a combination of the speech recognition result, the image recognition result, and a contact recognition result obtained by recognizing a physical contact of the conversation partner with a contact recognition unit.

12. The speech communication method according to claim 11,

further comprising:

continuing the conversation upon the failure to track the position of the conversation partner and the contact recognition result indicating the physical contact of the conversation partner with the contact recognition unit.

13. The speech communication method according to claim 11, further comprising:

issuing another notification to request the conversation partner to physically contact the contact recognition unit upon the failure to track the position of the conversation partner by the tracking control unit, the speech contents of the conversation partner not matching the predicted response, and the response to said notification not received after the notification is repeated the prescribed number of times; and

continuing the conversation based on the recognition result indicating the physical contact of the conversation partner.

14. A robot apparatus comprising:

a speech communication system enabling a conversation with a conversation partner, said system including

a speech recognition unit configured to recognize a speech content of the conversation partner;

a conversation control unit configured to the conversation using available resources;

an image recognition unit configured to recognize a face of the conversation partner; and

a tracking control unit configured to, in a normal mode, track a position of the conversation partner in response to both of a recognition result of the image recognition unit and a recognition result of the speech recognition unit, wherein while the speech communication system is in a failure mode as a result of an inability of the tracking control unit to track the position of the conversation partner, the conversation control unit is configured to continue the conversation without tracking including behaviors relying, in the normal mode, on tracking performed by the tracking control unit, the conversation is continued, without tracking, through the speech content of the conversation partner being recognized with the speech recognition unit and matching a predicted response.