ADAPTING ROBOT BEHAVIOR BASED UPON HUMAN-ROBOT INTERACTION

• US 20140249676A1
• Filed: 03/04/2013
• Published: 09/04/2014
• Est. Priority Date: 03/04/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Florencio, Dinei A., Bohus, Dan, Macharet, Douglas Guimares
• Granted Patent: US 9,956,687 B2
• Time in Patent Office: Days
• US Class Current: 700/259
• CPC Class Codes: B25J 9/1694 characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
• CPC Class Codes: Y10S 901/01 Mobile robot
• CPC Class Codes: Y10S 901/47 Optical

Abstract

Technologies pertaining to human-robot interaction are described herein. The robot includes a computer-readable memory that comprises a model that, with respect to successful completions of a task, is fit to observed data, where at least some of such observed data pertains to a condition that is controllable by the robot, such as position of the robot or distance between the robot and a human. A task that is desirably performed by the robot is to cause the human to engage with the robot. The model is updated while the robot is online, such that behavior of the robot adapts over time to increase the likelihood that the robot will successfully complete the task.

Claims

1. A method executed by a processor in a mobile robotic device, the method comprising:

receiving a first signal output by a first sensor, the first signal indicating that a human is in an environment of the mobile robotic device;

receiving a second signal output by a second sensor, the second signal being indicative of a first condition pertaining to the mobile robotic device, the first condition being subject to control by the mobile robotic, and the first condition identified as being relevant to a task assigned to the mobile robotic device, the task involving interaction with the human;

identifying an action to be undertaken by the mobile robotic device, the action identified to complete the task, the action identified based at least in part upon the first signal output by the first sensor, the second signal output by the second sensor, and past successes and failures of the mobile robotic device when the robotic mobile device attempted to complete the task with other humans in the environment; and

transmitting a signal to an actuator of the mobile robotic device to cause the mobile robotic device to perform the action.

2. The method of claim 1, further comprising:

ascertaining whether the mobile robotic device successfully completed the task or failed to complete the task; and

responsive to the ascertaining of whether the mobile robotic device successfully completed the task or failed to complete the task, updating a model that is employed in connection with identifying the action to be undertaken by the mobile robotic device, the model updated to increase a likelihood that the mobile robotic device will successfully complete the task with respect to another human that subsequently enters the environment.

3. The method of claim 2, wherein the ascertaining of whether the mobile robotic device successfully completed the task or failed to complete the task comprises:

identifying a spoken utterance set forth by the human to the mobile robotic device; and

ascertaining whether the mobile robotic device successfully completed the task or failed to complete the task based at least in part upon the identifying of the spoken utterance set forth by the human to the mobile robotic device.

4. The method of claim 2, wherein the ascertaining of whether the mobile robotic device successfully completed the task or failed to complete the task comprises:

identifying a gesture set forth by the human to the mobile robotic device; and

ascertaining whether the mobile robotic device successfully completed the task or failed to complete the task based at least in part upon the identifying of the gesture set forth by the human to the mobile robotic device.

5. The method of claim 2, wherein the updating of the model comprises utilizing at least one computer-executable reinforcement learning algorithm to update the model.

6. The method of claim 5, wherein identifying of the action comprises executing a Gaussian Process Regression algorithm over the model.

7. The method of claim 1, the first sensor being a video camera, the first signal being a video signal output by the video camera, the second sensor being a depth sensor, and the second signal being indicative of a distance between the mobile robotic device and the human.

8. The method of claim 7, wherein the action to be undertaken by the mobile robotic device is a transition from a current position in the environment to a different position relative to the human to alter the distance between the mobile robotic device and the human, wherein the action is identified to increase a likelihood that the task will be completed successfully relative to a probability that the task will completed successfully if the mobile robotic device remains at the current position.

9. The method of claim 8, wherein the task is to cause the human to engage with the mobile robotic device.

10. The method of claim 1, further comprising receiving a third signal output by a third sensor, the third signal being indicative of a second condition pertaining to the environment, the second condition being uncontrollable by the mobile robotic device, wherein the identifying of the action to be undertaken by the mobile robotic device is based at least in part upon the third signal output by the third sensor.

11. The method of claim 10, wherein the second condition is time of day and the task is to cause the human to engage with the mobile robotic device.

12. A mobile robot, comprising:

a motor that, when actuated, causes the robot to transition in an environment;

a first sensor that outputs a first signal that is indicative of a first condition corresponding to the robot, the first signal being controllable by the robot;

a second sensor that outputs a second signal that is indicative of a second condition corresponding to the environment, the second condition being uncontrollable by the robot;

a third sensor that outputs a third signal that is indicative of existence of a human in the environment;

a processor that receives the first signal, the second signal, and the third signal, the processor being in communication with the motor; and

a memory that comprises a plurality of components that are executed by the processor, the plurality of components comprising;

an action determiner component that, responsive to the processor receiving the third signal, accesses a learned model for completing a predefined task with respect to the human and determines an action to be undertaken by the robot to complete the predefined task based at least in part upon the learned model, the first signal, and the second signal, the action determiner component determining the action to, in accordance with the learned model, optimize a first probability that the predefined task with respect to the human will be successfully completed, and wherein the processor transmits a command to the motor in connection with causing the robot to perform the action responsive to the action determiner component determining the action; and

a learner component that updates the learned model based upon the first signal, the second signal, and an indication as to whether the robot successfully completed the task or failed to complete the task, wherein the learner component updates the learned model to maximize a second probability that the predefined task will be successfully completed when the processor detects that another human has entered the environment.

13. The mobile robot of claim 12, wherein the predefined task is causing the human to engage with the robot in the environment.

14. The mobile robot of claim 13, further comprising a microphone that receives audible feedback from the human, and wherein the plurality of components further comprise a task identifier component that identifies whether the robot has successfully completed the task or failed to complete the task based upon the audible feedback received by the microphone from the human.

15. The mobile robot of claim 13, further comprising a video camera that captures a video signal that includes the human, wherein the plurality of components further comprise a task identifier component that identifies whether the robot has successfully completed the task or failed to complete the task based upon the video signal captured by the video camera.

16. The mobile robot of claim 13, wherein the action determiner component determines a desired distance between the robot and the human in connection with determining the action to be undertaken by the computing device to complete the predefined task.

17. The mobile robot of claim 16, wherein the action is transitioning the robot from a first position to a second position in the environment, wherein when the robot is at the second position a distance between the robot and the human is the desired distance.

18. The mobile robot of claim 12, the first sensor being a depth sensor, the first condition being a distance between the robot and the human.

19. The mobile robot of claim 12, wherein the action determiner component determines the action through utilization of a Gaussian Regression Process algorithm over the learned model.

20. A mobile robot comprising a computer-readable storage medium that comprises instructions that, when executed by a processor, causes the processor to perform acts comprising:

identifying that a human is in an environment with the mobile robot;

determining a current time of day;

accessing a learned model responsive to the identifying that the human is in the environment with the mobile robot;

utilizing the learned model, determining a distance between the mobile robot and the human that, in accordance with the learned model, is amongst a threshold number of distances that have a highest probability from amongst considered distances of the human engaging with the mobile robot;

transmitting a signal to a motor of the mobile robot, the signal causing the motor to drive the mobile robot from a first location in the environment to a second location in the environment, wherein when the mobile robot is at the second location a distance between the mobile robot and the human is the distance that, in accordance with the learned model, maximizes the probability that the human will engage with the mobile robot;

detecting that the mobile robot is at the second location;

responsive to the detecting that the mobile robot is at the second location, identifying whether the human engaged with the mobile robot or failed to engage with the mobile robot;

immediately responsive to the identifying whether the human engaged with the mobile robot or failed to engage with the mobile robot, updating the model based at least in part upon the current time of day, the distance, and the identifying whether the human engaged with the mobile robot or failed to engage with the mobile robot, wherein the model is updated to maximize a probability that a human subsequently identified as being in the environment will engage with the mobile robot.