Speech communication system and method, and robot apparatus

• US 8,538,750 B2
• Filed: 11/02/2012
• Issued: 09/17/2013
• Est. Priority Date: 07/03/2003
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Aoyama, Kazumi, Shimomura, Hideki
• Primary Examiner(s): COLUCCI, MICHAEL C
• Application Number:
  US13/667,978
• Publication Number:
  US 20130060566A1
• Time in Patent Office: 319 Days
• Field of Search: 704/276, 704/275, 704/270, 704/206, 702/188, 702/153, 701/301, 700/257, 700/245, 382/118, 382/107, 382/103, 318/567
• US Class Current: 704/231
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

This invention realizes a speech communication system and method, and a robot apparatus capable of significantly improving entertainment property. A speech communication system with a function to make conversation with a conversation partner is provided with a speech recognition means for recognizing speech of the conversation partner, a conversation control means for controlling conversation with the conversation partner based on the recognition result of the speech recognition means, an image recognition means for recognizing the face of the conversation partner, and a tracking control means for tracing the existence of the conversation partner based on one or both of the recognition result of the image recognition means and the recognition result of the speech recognition means. The conversation control means controls conversation so as to continue depending on tracking of the tracking control means.

Claims

1. A speech communication system enabling a conversation with a conversation partner, comprising:

a microprocessor;

a speech recognition unit configured to recognize a speech content of the conversation partner;

a speech processing unit configured to generate a response based on the speech content recognized by the speech recognition unit;

an audio output unit configured to output auditory communications corresponding to the response generated by the speech processing unit;

a touch sensing unit configured to recognize a touch input by the conversation partner;

a control unit configured to determine whether or not to continue the conversation based on a recognition result from the touch sensing unit; and

a network interface configured to communicate with an external network.

2. The speech communication system according to claim 1, wherein the speech processing unit is further configured to generate the response based on a conversation history with the conversation partner.

3. The speech communication system according to claim 2, wherein the control unit is further configured to determine whether or not to continue the conversation based on a recognition result from the touch sensing unit and the image capturing unit.

4. The speech communication system according to claim 2, further comprising:

a camera; and

a microphone.

5. The speech communication system according to claim 2, wherein the speech communication apparatus implements a game function.

6. The speech communication system according to claim 2, wherein the network interface is configured to receive, via the external network, information generated as a result of calculations performed at a device which is remote to the network interface.

7. The speech communication system according to claim 6, wherein the calculations performed at the device are performed based on input from the speech recognition unit.

8. The speech communication system according to claim 2, wherein the network interface is a wireless network interface.

9. The speech communication system according to claim 2, wherein the external network is the Internet.

10. The speech communication system according to claim 2, wherein a conversation partner speech utterance is accepted in response to the touch sensing unit recognizing the touch input by the conversation partner.

11. The speech communication system according to claim 10, wherein auditory communications output by the audio output unit are generated based on the input conversation partner speech utterance.

12. The speech communication apparatus according to claim 2, wherein conversation history with the conversation partner includes a previous conversation partner speech utterance which was recognized by the speech recognition unit.

13. A speech communication apparatus enabling a conversation with a conversation partner, comprising:

a microprocessor;

a speech input unit configured to input a speech content of the conversation partner;

an audio output unit configured to output auditory communications based on a conversation history with the conversation partner;

an image capturing unit configured to capture a face of the conversation partner;

a touch sensing unit configured to recognize a touch input by the conversation partner;

a control unit configured to determine whether or not to continue the conversation based on a recognition result from the touch sensing unit; and

a network interface configured to communicate with an external network.

14. The speech communication apparatus according to claim 13, wherein the control unit is further configured to determine whether or not to continue the conversation based on a recognition result from the touch sensing unit and the image capturing unit.

15. The speech communication apparatus according to claim 13, further comprising:

a camera; and

a microphone.

16. The speech communication apparatus according to claim 13, wherein the speech communication apparatus implements a game function.

17. The speech communication apparatus according to claim 13, wherein the network interface is configured to receive, via the external network, information generated as a result of calculations performed at a device which is remote to the speech communication apparatus.

18. The speech communication apparatus according to claim 17, wherein the calculations performed at the device which is remote to the speech communication apparatus are performed based on input from the speech communication apparatus.

19. The speech communication apparatus according to claim 13, wherein the network interface is a wireless network interface.

20. The speech communication apparatus according to claim 13, wherein the external network is the Internet.

21. The speech communication apparatus according to claim 13, wherein a conversation partner speech utterance is accepted in response to the touch sensing unit recognizing the touch input by the conversation partner.

22. The speech communication apparatus according to claim 21, wherein auditory communications output by the audio output unit are generated based on the input conversation partner speech utterance.

23. The speech communication apparatus according to claim 13, wherein conversation history with the conversation partner includes a previous conversation partner speech utterance which was recognized by the speech recognition unit.

24. A speech communication apparatus enabling a conversation with a conversation partner, comprising:

a microprocessor;

a speech recognition unit configured to recognize a speech content of the conversation partner;

an audio output unit configured to output auditory communications;

a touch sensing unit configured to recognize a touch input by the conversation partner;

a control unit configured to determine whether or not to continue the conversation based on a recognition result from the touch sensing unit; and

a network interface configured to communicate with an external network.

25. The speech communication apparatus according to claim 24, wherein the audio output unit is further configured to output auditory communications based on a conversation history with the conversation partner.

26. The speech communication apparatus according to claim 25, wherein the control unit is further configured to determine whether or not to continue the conversation based on a recognition result from the touch sensing unit and the image capturing unit.

27. The speech communication apparatus according to claim 25, further comprising:

a camera; and

a microphone.

28. The speech communication apparatus according to claim 25, wherein the speech communication apparatus implements a game function.

29. The speech communication apparatus according to claim 25, wherein the network interface is configured to receive, via the external network, information generated as a result of calculations performed at a device which is remote to the speech communication apparatus.

30. The speech communication apparatus according to claim 29, wherein the calculations performed at the device which is remote to the speech communication apparatus are performed based on input from the speech communication apparatus.

31. The speech communication apparatus according to claim 25, wherein the network interface is a wireless network interface.

32. The speech communication apparatus according to claim 25, wherein the external network is the Internet.

33. The speech communication apparatus according to claim 25, wherein a conversation partner speech utterance is accepted in response to the touch sensing unit recognizing the touch input by the conversation partner.

34. The speech communication apparatus according to claim 33, wherein auditory communications output by the audio output unit are generated based on the input conversation partner speech utterance.

35. The speech communication apparatus according to claim 25, wherein conversation history with the conversation partner includes a previous conversation partner speech utterance which was recognized by the speech recognition unit.

36. A non-transitory computer readable medium having stored thereon a program that when executed by a computer causes the computer to implement a speech communication system enabling a conversation with a conversation partner, comprising:

a speech recognition unit configured to recognize a speech content of the conversation partner;

a speech processing unit configured to generate a response based on the speech content recognized by the speech recognition unit;

an audio output unit configured to output auditory communications corresponding to the response generated by the speech processing unit;

a touch sensing unit configured to recognize a touch input by the conversation partner;

a control unit configured to determine whether or not to continue the conversation based on a recognition result from the touch sensing unit; and

a network interface unit configured to communicate with an external network.

37. The non-transitory computer readable medium according to claim 36, wherein the speech processing unit is further configured to generate the response based on a conversation history with the conversation partner.

38. The non-transitory computer readable medium according to claim 37, wherein the control unit is further configured to determine whether or not to continue the conversation based on a recognition result from the touch sensing unit and the image capturing unit.

39. The non-transitory computer readable medium according to claim 37, wherein the network interface unit is configured to receive, via the external network, information generated as a result of calculations performed at a remote device.

40. The non-transitory computer readable medium according to claim 39, wherein the calculations performed at the remote device are performed based on input from the speech recognition unit.

41. The non-transitory computer readable medium according to claim 37, wherein the network interface unit implements a wireless networking interface.

42. The non-transitory computer readable medium according to claim 37, wherein the external network is the Internet.

43. The non-transitory computer readable medium according to claim 37, wherein a conversation partner speech utterance is accepted in response to the touch sensing unit recognizing the touch input by the conversation partner.

44. The non-transitory computer readable medium according to claim 43, wherein auditory communications output by the audio output unit are generated based on the input conversation partner speech utterance.

45. The non-transitory computer readable medium according to claim 37, wherein conversation history with the conversation partner includes a previous conversation partner speech utterance which was recognized by the speech recognition unit.