Adapting robot behavior based upon human-robot interaction

• US 10,300,606 B2
• Filed: 04/09/2018
• Issued: 05/28/2019
• Est. Priority Date: 03/04/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Florencio, Dinei A., Macharet, Douglas Guimaraes, Bohus, Dan
• Primary Examiner(s): Mancho, Ronnie M
• Application Number:
  US15/949,055
• Publication Number:
  US 20180229370A1
• Time in Patent Office: 414 Days
• Field of Search: 700245, 700259
• CPC Class Codes: B25J 9/1694 characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
• CPC Class Codes: Y10S 901/01 Mobile robot
• CPC Class Codes: Y10S 901/47 Optical

Abstract

Technologies pertaining to human-robot interaction are described herein. The robot includes a computer-readable memory that comprises a model that, with respect to successful completions of a task, is fit to observed data, where at least some of such observed data pertains to a condition that is controllable by the robot, such as position of the robot or distance between the robot and a human. A task that is desirably performed by the robot is to cause the human to engage with the robot. The model is updated while the robot is online, such that behavior of the robot adapts over time to increase the likelihood that the robot will successfully complete the task.

Claims

1. A robotic device comprising:

a sensor;

a speaker;

memory; and

a processor that is operably coupled to the sensor, the speaker, and the memory, wherein the memory stores instructions that, when executed by the processor, cause the processor to perform acts comprising;

determining that a human is in an environment of the robotic device based upon a signal output by the sensor;

responsive to the human being determined to be in the environment of the robotic device, computing a probability that an audio signal, when emitted by the speaker of the robotic device, will cause the human to interact with the robotic device, wherein the probability is computed based upon past successes and failures of the robotic device when the robotic device previously emitted audio signals when attempting to cause other humans to interact with the robotic device in the environment; and

based upon the computed probability, causing the speaker of the robotic device to emit the audio signal.

2. The robotic device of claim 1, wherein the robotic device is a mobile robotic device.

3. The robotic device of claim 2, further comprising a motor that is operably coupled to the processor, the acts further comprising:

based upon the human being determined to be in the environment of the robotic device, causing the motor to position the mobile robotic device at a distance from the human, wherein the distance is determined by the processor to maximize the probability that the human will interact with the mobile robotic device, and further wherein the distance is based upon the past successes and failures of the mobile robotic device when attempting to cause other humans to interact with the mobile robotic device in the environment.

4. The robotic device of claim 1, wherein the signal is indicative of a condition of the environment that is outside of the control of the robotic device.

5. The robotic device of claim 1, the acts further comprising:

ascertaining whether the human interacted with the robotic device; and

responsive to ascertaining whether the human interacted with the robotic device, updating a model that is employed in connection with identifying audio signals that are to be output by the speaker to cause humans to engage with the robotic device, wherein the model is updated to increase the probability that the robotic device will cause a second human to engage with the robotic device when the second human subsequently enters the environment.

6. The robotic device of claim 5, wherein ascertaining whether the human engaged with the robotic device comprises:

identifying a spoken utterance set forth by the human to the robotic device, wherein the human is determined to have engaged with the robotic device based upon the identified spoken utterance.

7. The robotic device of claim 5, wherein ascertaining whether the human engaged with the robotic device comprises:

identifying a gesture set forth by the human to the robotic device, wherein the human is determined to have engaged with the robotic device based upon the identified gesture.

8. The robotic device of claim 5, wherein updating the model comprises utilizing a computer-executable reinforcement learning algorithm to update the model.

9. The robotic device of claim 1, the sensor being a depth sensor, and the signal being indicative of a distance between the robotic device and the human.

10. The robotic device of claim 1, wherein the probability is computed based further upon a volume of the audio signal.

11. The robotic device of claim 1, wherein the probability is computed based further upon a tone of the audio signal.

12. A method performed by a processor of a robotic device, the method comprising:

receiving, at the processor, a signal that is output by a sensor of the robotic device, the signal is indicative of a condition of an environment of the robotic device;

detecting, by the processor, that a human is in the environment with the robotic device based upon the signal output by the sensor;

responsive to detecting that the human is in the environment with the robotic device, determining, by the processor, a probability that an audio signal, when emitted by a speaker of the robotic device, will cause the human to interact with the robotic device, wherein the probability is determined based upon past successes and failures of the robotic device when the robotic device attempted to cause humans to interact with the robotic device; and

based upon the determined probability, controlling, by the processor, the speaker of the robotic device to output the audio signal.

13. The method of claim 12, wherein the robotic device is a mobile robotic device.

14. The method of claim 13, wherein the probability is determined based upon a distance between the mobile robotic device and the human, the method further comprising:

causing, by the processor, the mobile robotic device to position itself at the distance from the human based upon the determined probability.

15. The method of claim 12, wherein the probability is determined based upon a volume of the audio signal.

16. The method of claim 15, wherein the probability is determined based upon a tone of the audio.

17. The method of claim 12, further comprising:

subsequent to outputting the audio signal, determining, by the processor, that the human has interacted with the robot; and

responsive to determining that the human has interacted with the robot, updating, by the processor, a model that is employed in connection with identifying audio signals that are to be output by the speaker to cause humans to engage with the robotic device, wherein the model is updated by the processor to increase the probability that the robotic device will cause a second human to engage with the robotic device when the second human enters the environment after the human has been detected as being in the environment of the robotic device.

18. The method of claim 12, further comprising identifying, by the processor, that the human has interacted with the robotic device, wherein identifying that the human has interacted with the robotic device comprises identifying, by the processor, that the human has set forth a spoken utterance that is directed towards the robotic device.

19. The method of claim 12, further comprising identifying, by the processor, that the human has interacted with the robotic device, wherein identifying that the human has interacted with the robotic device comprises identifying, by the processor, that the human has set forth a gesture that is directed towards the robotic device.

20. A computer-readable storage medium of a robotic device comprising instructions that, when executed by a processor of the robotic device, causes the processor to perform acts comprising:

receiving a signal that is output by a sensor of the robotic device, the signal is indicative of a condition of an environment of the robotic device;

detecting that a human is in the environment with the robotic device based upon the signal output by the sensor; and

responsive to detecting that the human is in the environment with the robotic device, computing a probability that, when the robotic device emits an audio signal from a speaker of the robotic device, the human will interact with the robotic device, wherein the probability is computed based upon past successes and failures of the robotic device when attempting to cause humans to interact with the robotic device; and

outputting, from the speaker of the robotic device, the audio signal, wherein the audio signal is output based upon the computed probability.