System, method and computer program product for controlling a mission-oriented robot based on a user's emotional state

• US 10,220,517 B2
• Filed: 10/31/2016
• Issued: 03/05/2019
• Est. Priority Date: 10/31/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Delazari Binotto, Alecio Pedro, Guimaraes, Rodrigo Laiola, Caetano dos Santos, Davi Francisco
• Primary Examiner(s): Moyer, Dale
• Application Number:
  US15/339,201
• Publication Number:
  US 20180117769A1
• Time in Patent Office: 855 Days
• CPC Class Codes: B25J 11/001 with emotions simulating means
• CPC Class Codes: B25J 9/1694 characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
• CPC Class Codes: G05B 19/042 using digital processors G05B19/05 takes precedence
• CPC Class Codes: G05B 2219/45098 Vacuum cleaning robot
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means

Abstract

A robot control method, system, and computer program product, include extracting robot data from the robot and user emotional state data of a user interacting with the robot, configuring a mission of a robot comprising a sequence of a plurality of tasks based on the robot data and the user emotional state data, and reconfiguring an order of the sequence of the plurality of tasks if a change in the user emotional state data is detected.

Claims

1. A computer-implemented robot control method, the method comprising:

extracting robot data from a robot and user emotional state data of a user interacting with the robot;

configuring a mission of a robot comprising a sequence of a plurality of tasks based on the robot data and the user emotional state data; and

reconfiguring an order of the sequence of the plurality of tasks to create a new sequence of the plurality of tasks if a change in the user emotional state data is detected,

wherein the new sequence of the plurality of tasks is configured to increase an emotional satisfaction level of the user towards the robot.

2. The computer-implemented method of claim 1, wherein the reconfiguring reconfigures the order of the sequence of the plurality of tasks if a change in the robot data is detected.

3. The computer-implemented method of claim 1, wherein the new sequence of the plurality of tasks is configured to maximize an emotional satisfaction level of the user towards the robot.

4. The computer-implemented method of claim 1, further comprising generating an explanation of a reason for performing a task in the sequence of the plurality of tasks if the user confronts the robot about a rationale for performing the task.

5. The computer-implemented method of claim 4, wherein the explanation is generated to maximize an objective function factoring the user emotional state data, the robot data, the mission, and the sequence of the plurality of tasks.

6. The computer-implemented method of claim 4, wherein the generating the explanation computes and scores a plurality of explanations based on the robot data, the user emotional state data, and past explanations from a database, and

wherein the generating outputs to the user the explanation having a highest score.

7. The computer-implemented method of claim 6, wherein the generating adjusts a score of each of the plurality of explanations based on a user feedback of the output explanation.

8. The computer-implemented method of claim 4, wherein the reconfiguring the order of the sequence of the plurality of tasks is overridden if a cost associated with the new sequence is greater than a predetermined threshold, and

wherein the generating generates the explanation for the user to explain a reason for overriding the new sequence when the change in the user emotional state data is detected.

9. The computer-implemented method of claim 1, wherein the robot data comprises:

a current power level;

a mapping of an environment of the robot;

a current location,

a position of the user relative to the robot;

historical data about previous tasks performed by the robot; and

environment conditions.

10. The computer-implemented method of claim 1, wherein the user emotional state data is extracted from one of:

an audio-visual stream;

user data;

a historical profile of the user input to the robot; and

social media data.

11. The computer-implemented method of claim 1, embodied in a cloud-computing environment.

12. A computer program product for robot control, the computer program product comprising a computer-readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform:

extracting robot data from a robot and user emotional state data of a user interacting with the robot;

configuring a mission of a robot comprising a sequence of a plurality of tasks based on the robot data and the user emotional state data; and

reconfiguring an order of the sequence of the plurality of tasks if a change in the user emotional state data is detected,

wherein the new sequence of the plurality of tasks is configured to increase an emotional satisfaction level of the user towards the robot.

13. The computer program product of claim 12, wherein the reconfiguring reconfigures the order of the sequence of the plurality of tasks if a change in the robot data is detected.

14. The computer program product of claim 12, wherein the new sequence of the plurality of tasks is configured to maximize an emotional satisfaction level of the user towards the robot.

15. The computer program product of claim 12, further comprising generating an explanation of a reason for performing a task in the sequence of the plurality of tasks if the user confronts the robot about a rationale for performing the task.

16. The computer program product of claim 15, wherein the explanation is generated to maximize an objective function factoring the user emotional state data, the robot data, the mission, and the sequence of the plurality of tasks.

17. The computer program product of claim 15, wherein the generating the explanation computes and scores a plurality of explanations based on the robot data, the user emotional state data, and past explanations from a database, and

wherein the generating outputs to the user the explanation having a highest score.

18. The computer program product of claim 17, wherein the generating adjusts a score of each of the plurality of explanations based on a user feedback of the output explanation.

19. A robot control system, said system comprising:

a processor; and

a memory, the memory storing instructions to cause the processor to perform;

extracting robot data from a robot and user emotional state data of a user interacting with the robot;

configuring a mission of a robot comprising a sequence of a plurality of tasks based on the robot data and the user emotional state data; and

reconfiguring;

an order of the sequence of the plurality of tasks if a change in the user emotional state data is detected,

wherein the new sequence of the plurality of task is configured to increase an emotional satisfaction level of the user towards the robot.

20. The system of claim 19, embodied in a cloud-computing environment.