Communication robot and control program of communication robot

• US 11,474,499 B2
• Filed: 10/22/2019
• Issued: 10/18/2022
• Est. Priority Date: 10/24/2018
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Inventors: Yamamoto, Takashi
• Primary Examiner(s): Belousov, Andrey
• Application Number:
  US16/659,858
• Publication Number:
  US 20200133239A1
• Time in Patent Office: 1,092 Days
• CPC Class Codes: G05B 19/4155 characterised by programme execution, i.e. part programme or machine function execution, e.g. selection of a programme
• CPC Class Codes: G05B 2219/34456 Authorize control of machine, robot if control panel has been connected
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: H04L 67/125 involving control of end-device applications over a network
• CPC Class Codes: H04L 67/75 Indicating network or usage conditions on the user display

Abstract

A communication robot includes: an operation part; and a communication arbitration unit configured to exhibit one of a robot mode for autonomously operating the operation part by applying a first operational criterion and an avatar mode for operating the operation part based on an operation instruction sent from a remote operator by applying a second operational criterion to arbitrate communication with a service user, in which the communication arbitration unit operates, when it is intended to exhibit the avatar mode in accordance with an operation request from the remote operator, the operation part in the robot mode and asks the service user whether to allow the mode to be switched to the avatar mode.

Claims

1. A communication robot comprising:

an operation part;

a communication arbitration unit configured to exhibit a robot mode for autonomously determining and outputting a voice in response to a voice of a service user by autonomously operating the operation part and an avatar mode for operating the operation part based on an operation instruction sent from a remote operator to arbitrate communication with a service user; and

a memory configured to store a first operational criterion and a second operational criterion, the first operational criterion being a criterion for operating the operation part in the robot mode, and the second operational criterion being a criterion for operating the operation part in the avatar mode and different from the first operational criterion,

wherein the communication arbitration unit operates, when it is intended to exhibit the avatar mode in accordance with an operation request from the remote operator, the operation part in the robot mode and asks the service user whether to allow the mode to be switched to the avatar mode, and

wherein the first operational criterion and the second operational criterion define at least a temperament change frequency, the temperament change frequency being a frequency of emotion expressed by the operation part.

2. The communication robot according to claim 1, wherein the communication arbitration unit presents information regarding the remote operator to the service user via an operation by the robot mode when it asks the service user whether to allow the mode to be switched.

3. The communication robot according to claim 2, wherein the communication arbitration unit changes an aspect in which the information is presented to the service user in accordance with an attribute of the remote operator.

4. The communication robot according to claim 1, wherein the communication arbitration unit causes, when the service user has allowed the mode to be switched to the avatar mode, the avatar mode to be exhibited after the communication arbitration unit executes an exit operation in the robot mode.

5. The communication robot according to claim 1, wherein the first operational criterion is updated based on a history of the communication.

6. The communication robot according to claim 1, comprising a notification part for notifying which one of the robot mode and the avatar mode is exhibited.

7. The communication robot according to claim 1, wherein, when the communication arbitration unit has received the operation request from a remote operator that matches a preset condition, the communication arbitration unit rejects the operation request without asking the service user whether to allow the mode to be switched.

9. The communication robot of claim 1, further comprising:

a camera configured to capture an image of the service user,

wherein the communication arbitration unit is configured to;

estimate an emotion of the service user based on the image of the service user; and

determine a facial expression displayed during the robot mode based on the estimated emotion of the service user.

10. The communication robot of claim 1, wherein the communication arbitration unit is configured to:

obtain a feedback from the service user regarding a facial expression displayed during the robot mode; and

adjust a frequency of the facial expression being displayed based on the feedback.

13. The communication robot according to claim 1, wherein the first operational criterion and the second operational criterion define at least a moving speed of the operation part.

8. A non-transitory computer readable medium storing a control program of a communication robot for causing a computer to execute a reference step for referring to a first operational criterion and a second operational criterion stored a memory and a communication arbitration step for exhibiting a robot mode for autonomously determining and outputting a voice in response to a voice of a service user by autonomously operating an operation part and an avatar mode for operating the operation part based on an operation instruction sent from a remote operator to arbitrate communication with a service user, the first operational criterion being a criterion for operating the operation part in the robot mode, and the second operational criterion being a criterion for operating the operation part in the avatar mode and different from the first operational criterion, wherein

the communication arbitration step comprises a check step for operating, when it is intended to exhibit the avatar mode in accordance with an operation request from the remote operator, the operation part in the robot mode and asking the service user whether to allow the mode to be switched to the avatar mode, and

wherein the first operational criterion and the second operational criterion define at least a temperament change frequency, the temperament change frequency being a frequency of emotion expressed by the operation part.

11. The non-transitory computer readable medium of claim 8, wherein the computer is configured to:

obtain an image of the service user;

estimate an emotion of the service user based on the image of the service user; and

determine a facial expression displayed during the robot mode based on the estimated emotion of the service user.

12. The non-transitory computer readable medium of claim 8, wherein the computer is configured to:

obtain a feedback from the service user regarding a facial expression displayed during the robot mode; and

adjust a frequency of the facial expression being displayed based on the feedback.

14. A communication robot comprising:

an operation part;

a communication arbitration unit configured to exhibit a robot mode for autonomously determining and outputting a voice in response to a voice of a service user by autonomously operating the operation part and an avatar mode for operating the operation part based on an operation instruction sent from a remote operator to arbitrate communication with a service user; and

a memory configured to store the first operational criterion and the second operational criterion, the first operational criterion being a criterion for operating the operation part in the robot mode, and the second operational criterion being a criterion for operating the operation part in the avatar mode and different from the first operational criterion,

wherein the communication arbitration unit operates, when it is intended to exhibit the avatar mode in accordance with an operation request from the remote operator, the operation part in the robot mode and asks the service user whether to allow the mode to be switched to the avatar mode, and

wherein the first operational criterion and the second operational criterion define at least a gesture during conversation, the gesture during conversation being a degree of a gesture executed in accordance with a content of a conversation by the operation part.