Adapting robot behavior based upon human-robot interaction

• US 9,956,687 B2
• Filed: 03/04/2013
• Issued: 05/01/2018
• Est. Priority Date: 03/04/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Florencio, Dinei A., Macharet, Douglas Guimares, Bohus, Dan
• Primary Examiner(s): MANCHO, RONNIE M
• Application Number:
  US13/783,405
• Publication Number:
  US 20140249676A1
• Time in Patent Office: 1,884 Days
• Field of Search: 700259, 700245, 700264, 901 47, 901 30, 901 31, 901 32, 901 33, 901 34, 901 35, 382155, 3185681, 31836811, 706 12
• CPC Class Codes: B25J 9/1694 characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
• CPC Class Codes: Y10S 901/01 Mobile robot
• CPC Class Codes: Y10S 901/47 Optical

Abstract

Technologies pertaining to human-robot interaction are described herein. The robot includes a computer-readable memory that comprises a model that, with respect to successful completions of a task, is fit to observed data, where at least some of such observed data pertains to a condition that is controllable by the robot, such as position of the robot or distance between the robot and a human. A task that is desirably performed by the robot is to cause the human to engage with the robot. The model is updated while the robot is online, such that behavior of the robot adapts over time to increase the likelihood that the robot will successfully complete the task.

Claims

1. A method executed by a processor in a mobile robotic device, the method comprising:

receiving a first signal output by a first sensor, the first signal indicating that a human is in an environment of the mobile robotic device, wherein the mobile robotic device is tasked with causing the human to engage with the mobile robotic device;

receiving a second signal output by a second sensor, the second signal being indicative of a first condition pertaining to the mobile robotic device, the first condition being subject to control by the mobile robotic device, and the first condition identified as being relevant to causing the human to engage with the mobile robotic device;

identifying an action to be undertaken by the mobile robotic device, the action identified to facilitate causing the human to engage with the mobile robotic device, the action identified based upon the first signal output by the first sensor, the second signal output by the second sensor, and past successes and failures of the mobile robotic device when attempting to cause other humans to engage with the mobile robotic device in the environment, wherein the action is further identified based upon the mobile robotic device tuning its operation over time with respect to the first condition to optimize a probability that the mobile robotic device will cause the human to engage with the mobile robotic device; and

transmitting a signal to an actuator of the mobile robotic device to cause the mobile robotic device to perform the action.

2. The method of claim 1, further comprising:

ascertaining whether the mobile robotic device successfully caused the human to engage with the mobile robotic device or failed to cause the human to engage with the mobile robotic device; and

responsive to the ascertaining of whether the mobile robotic device successfully caused the human to engage with the mobile robotic device or failed to cause the human to engage with the mobile robotic device, updating a model that is employed in connection with identifying the action to be undertaken by the mobile robotic device, the model updated to increase a likelihood that the mobile robotic device will successfully cause another human to engage with the mobile robotic device when attempting to interact with the another human that subsequently enters the environment.

3. The method of claim 2, wherein the ascertaining of whether the mobile robotic device successfully caused the human to engage with the mobile robotic device or failed to cause the human to engage with the mobile robotic device comprises:

identifying a spoken utterance set forth by the human to the mobile robotic device; and

ascertaining whether the mobile robotic device successfully caused the human to engage with the mobile robotic device or failed to cause the human to engage with the mobile robotic device based upon the identifying of the spoken utterance set forth by the human to the mobile robotic device.

4. The method of claim 2, wherein the ascertaining of whether the mobile robotic device successfully caused the human to engage with the mobile robotic device or failed to cause the human to engage with the mobile robotic device comprises:

identifying a gesture set forth by the human to the mobile robotic device; and

ascertaining whether the mobile robotic device successfully caused the human to engage with the mobile robotic device or failed to cause the human to engage with the mobile robotic device based upon the identifying of the gesture set forth by the human to the mobile robotic device.

5. The method of claim 2, wherein the updating of the model comprises utilizing at least one computer-executable reinforcement learning algorithm to update the model.

6. The method of claim 5, wherein identifying of the action comprises executing a Gaussian Process Regression algorithm over the model.

7. The method of claim 1, the first sensor being a video camera, the first signal being a video signal output by the video camera, the second sensor being a depth sensor, and the second signal being indicative of a distance between the mobile robotic device and the human.

8. The method of claim 7, wherein the action to be undertaken by the mobile robotic device is a transition from a current position in the environment to a different position relative to the human to alter the distance between the mobile robotic device and the human, wherein the action is identified to increase a likelihood that the human will engage with mobile robotic device relative to a probability that the human will engage with the mobile robotic device if the mobile robotic device remains at the current position.

9. The method of claim 1, further comprising receiving a third signal output by a third sensor, the third signal being indicative of a second condition pertaining to the environment, the second condition being uncontrollable by the mobile robotic device, wherein identifying the action to be undertaken by the mobile robotic device is based upon the third signal output by the third sensor.

10. The method of claim 9, wherein the second condition is time of day.

11. A mobile robot, comprising:

a motor that, when actuated, causes the robot to transition in an environment;

a first sensor that outputs a first signal that is indicative of a first condition corresponding to the robot, the first condition being controllable by the robot;

a second sensor that outputs a second signal that is indicative of a second condition corresponding to the environment, the second condition being uncontrollable by the robot;

a third sensor that outputs a third signal that is indicative of existence of a human in the environment;

at least one processor that receives the first signal, the second signal, and the third signal, the processor being in communication with the motor; and

memory that stores instructions that, when executed by the at least one processor, cause the at least one processor to perform acts comprising;

responsive to the processor receiving the third signal, accessing a learned model for causing the human to engage with the mobile robot;

based upon the learned model, the first signal, and the second signal, determining an action to be undertaken by the robot to cause the human to engage with the mobile robot, the action determined to, in accordance with the learned model, optimize a first probability that the human will engage with the mobile robot; and

transmitting a command to the motor in connection with causing the mobile robot to perform the action; and

updating the learned model based upon the first signal, the second signal, and an indication as to whether the robot successfully caused the human to engage with the mobile robot or failed to cause the human to engage with the mobile robot, wherein the learned model is updated to tune operation of the mobile robot in order to maximize a second probability that mobile robot will cause another human to engage with the mobile robot when the processor detects that the another human has subsequently entered the environment.

12. The mobile robot of claim 11, further comprising a microphone that receives audible feedback from the human, and the acts further comprising identifying whether the mobile robot has successfully caused the human to engage with the mobile robot or failed to cause the human to engage with the mobile robot based upon the audible feedback received by the microphone from the human.

13. The mobile robot of claim 11, further comprising a video camera that captures a video signal that includes the human, the acts further comprising identifying whether the mobile robot has successfully caused the human to engage with the mobile robot or failed to cause the human to engage with the mobile robot based upon the video signal captured by the video camera.

14. The mobile robot of claim 11, determining the action to be undertaken by the robot to cause the human to engage with the mobile robot comprises determining a desired distance between the robot and the human.

15. The mobile robot of claim 14, wherein the action is transitioning the mobile robot from a first position to a second position in the environment, wherein when the mobile robot is at the second position a distance between the mobile robot and the human is the desired distance.

16. The mobile robot of claim 11, the first sensor being a depth sensor, the first condition being a distance between the robot and the human.

17. The mobile robot of claim 11, wherein determining the action comprises determining the action through utilization of a Gaussian Regression Process algorithm over the learned model.

18. A mobile robot comprising a computer-readable storage medium that comprises instructions that, when executed by a processor, cause the processor to perform acts comprising:

identifying that a human is in an environment with the mobile robot;

accessing a learned model responsive to identifying that the human is in the environment with the mobile robot;

utilizing the learned model, determining a distance between the mobile robot and the human that, in accordance with the learned model, is amongst a threshold number of distances that have a highest probability of the human engaging with the mobile robot, the threshold number of distances being from amongst a plurality of considered distances;

transmitting a signal to a motor of the mobile robot, the signal causing the motor to drive the mobile robot from a first location in the environment to a second location in the environment, wherein when the mobile robot is at the second location a distance between the mobile robot and the human is the distance that, in accordance with the learned model, maximizes the probability that the human will engage with the mobile robot;

detecting that the mobile robot is at the second location;

responsive to detecting that the mobile robot is at the second location, identifying whether the human engaged with the mobile robot or failed to engage with the mobile robot; and

immediately responsive to identifying whether the human engaged with the mobile robot or failed to engage with the mobile robot, updating the model based at least in part upon the distance and whether the human engaged with the mobile robot or failed to engage with the mobile robot, wherein the model is updated to tune operation of the mobile robot such to maximize a probability that another human subsequently identified as being in the environment will engage with the mobile robot.

19. The mobile robot of claim 18, wherein the distance between the mobile robot and the human is determined based upon distances between the mobile robot and other humans when the mobile robot previously attempted to engage with the other humans in the environment.

20. The mobile robot of claim 18, wherein identifying whether the human engaged with the mobile robot or failed to engage with the mobile robot comprises identifying whether the human accepted material from the mobile robot.