Interactive robot initialization

• US 9,950,431 B2
• Filed: 01/25/2016
• Issued: 04/24/2018
• Est. Priority Date: 06/10/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Fouillade, Jean Sebastien, Sanchez, Russel, Papaefstathiou, Efstathios, Chalabi, Malek M.
• Primary Examiner(s): Mott, Adam R
• Application Number:
  US15/005,582
• Publication Number:
  US 20160136817A1
• Time in Patent Office: 820 Days
• CPC Class Codes: A47L 2201/00 Robotic cleaning machines, i.e. with automatic control of the travelling movement or the cleaning operation
• CPC Class Codes: B25J 11/0005 Manipulators having means for high-level communication with users, e.g. speech generator, face recognition means
• CPC Class Codes: B25J 13/003 by means of an audio-responsive input audible safety signals B25J19/061
• CPC Class Codes: B25J 19/023 including video camera means
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
• CPC Class Codes: H04L 63/083 using passwords cryptographic mechanisms or cryptographic arrangements for entity authentication using a predetermined code H04L9/3226
• CPC Class Codes: Y10S 901/01 Mobile robot
• CPC Class Codes: Y10S 901/47 Optical

Abstract

Initial interaction between a mobile robot and at least one user is described herein. The mobile robot captures several images of its surroundings, and identifies existence of a user in at least one of the several images. The robot then orients itself to face the user, and outputs an instruction to the user with regard to the orientation of the user with respect to the mobile robot. The mobile robot captures images of the face of the user responsive to detecting that the user has followed the instruction. Information captured by the robot is uploaded to a cloud-storage system, where information is included in a profile of the user and is shareable with others.

Claims

1. A robot comprising:

a camera;

a processor; and

memory that comprises instructions that, when executed by the processor, cause the robot to perform acts comprising;

recognizing that the robot is in a new environment;

responsive to the robot recognizing that the robot is in the new environment, executing an introduction protocol that is configured to introduce the robot to a plurality of new users with whom the robot has not previously interacted, the plurality of new users in the new environment with the robot, wherein executing the introduction protocol comprises, for each user in the plurality of new users;

recognizing a presence of the new user in surroundings of the robot based upon an image processed by the camera;

responsive to recognizing the presence of the new user in the surroundings of the robot, orienting a body of the robot such that the camera of the robot faces the new user;

while the body of the robot is oriented such that the camera of the robot faces the user;

using the camera, capturing an image of a face of the new user; and

outputting a request to the new user for information pertaining to the new user;

receiving a categorization of the new user from an owner of the robot; and

updating a user profile for the new user in computer-readable storage, wherein the profile of the new user is updated to comprise the categorization and the image of the face of the new user, wherein the profile for the new user is further updated based upon a response to the request; and

during subsequent interactions with the plurality of new users, presenting the new user with content based upon the user profile.

2. The robot of claim 1, the acts further comprising:

using the camera, capturing one or more images of the surroundings of the robot; and

identifying presence of the plurality of new users in the one or more images of the surroundings of the robot, the plurality of new users having had no prior interaction with the robot.

3. The robot of claim 1, the content being one or more of audio content output by a speaker of the robot or visual content displayed on a display of the robot.

4. The robot of claim 3, the request being related to a music preference of the new user, the content being music.

5. The robot of claim 3, the request being related to a television or movie preference of the new user, the content being a portion of one of a television show or a movie.

6. The robot of claim 1, the categorization of each new user comprising a priority level, the priority level usable by the robot in subsequent interactions with each new user to determine a priority of commands issued to the robot.

7. The robot of claim 1, the acts further comprising:

searching for available wireless networks; and

based upon a user input, connecting to a wireless network in the available wireless networks.

8. The robot of claim 1, further comprising:

a microphone; and

a speaker, wherein the robot outputs the request by way of the speaker and receives the response to the request by way of the microphone.

9. The robot of claim 7, wherein the robot connects to the wireless network prior to executing the introduction protocol.

10. A method performed by a mobile robot, comprising:

recognizing that the mobile robot is in a new environment; and

responsive to recognizing that the mobile robot is in the new environment, executing an introduction protocol, wherein executing the introduction protocol comprises;

capturing a plurality of images of surroundings of the robot with a camera of the mobile robot;

identifying existence of a plurality of new users in the plurality of images with whom the mobile robot has not previously interacted;

for each new user in the plurality of users;

responsive to identifying existence of the new user in the image, orienting the mobile robot to cause the new user to be in a center of a field of view of the camera;

outputting an instruction to the new user pertaining to orientation of the new user with respect to the camera of the mobile robot;

detecting that the new user has responded to the instruction;

capturing another image of the new user responsive to detecting that the new user has responded to the instruction;

retaining the another image of the new user for use in connection with identifying the new user during subsequent interactions between the mobile robot and the new user;

outputting a request to the new user by way of an output device of the mobile robot, the request being a question about the new user;

receiving a response to the question from the new user at an input device of the mobile robot;

receiving a categorization of the new user from an owner of the mobile robot; and

updating a profile of the new user in computer-readable storage based upon the categorization and the response to the question; and

providing a service to the new user in a subsequent interaction based upon the user profile.

11. The method of claim 10, wherein providing the service to the new user comprises presenting content to the new user based upon the profile of the new user.

12. The method of claim 10, wherein outputting the request to the new user comprises emitting audio from a speaker of the mobile robot.

13. The method of claim 10, wherein receiving the response to the question from the new user comprises receiving an audio signal from a microphone, the audio signal representative of audio received at the microphone.

14. A robot that comprises a computer-readable medium, the computer-readable medium comprising instructions that, when executed by a processor of the robot, causes the processor of the robot to perform acts comprising:

recognizing that the robot is in a new environment;

responsive to recognizing that the robot is in the new environment, executing an introduction protocol configured to introduce the robot to a plurality of new users who are in a same room with the robot, wherein the robot has not previously interacted with the plurality of new users, the introduction protocol comprising;

scanning surroundings of the robot with a camera to capture images of the surroundings of the robot;

recognizing that the captured images include the plurality of new users;

for each new user in the plurality of new users;

responsive to recognizing that the image includes the new user, orienting a body of the robot such that the camera of the robot faces the new user;

using the camera, capturing an image of a face of the new user subsequent to the body of the robot being oriented such that the camera of the robot faces the new user;

retaining the image of the face of the new user for use in connection with identifying the new user during a subsequent interaction of the new user with the robot; and

assigning a category for the new user to include in a profile for the new user, the category usable during the subsequent interaction of the new user with the robot to determine a priority of a command issued by the new user relative to a command issued by another user.

15. The robot of claim 14, the introduction protocol further comprising, for each new user:

outputting a request to the new user by way of an output device of the robot, the request being a question about the new user;

receiving a response to the question from the new user at an input device of the robot; and

updating the profile of the new user based upon the response to the question.

16. The robot of claim 15, the acts further comprising providing a service to the new user in a subsequent interaction based upon the user profile.

17. The robot of claim 15, receiving the response to the question from the new user comprises receiving an audio signal from a microphone, the audio signal representative of audio received at the microphone.

18. The robot of claim 14, wherein the introduction protocol is only executed after the robot connects to a wireless network in the new environment.

19. The robot of claim 14, the acts further comprising:

searching for available wireless networks; and

based upon user input, connecting to a wireless network in the available wireless networks.

20. The robot of claim 19, wherein the robot connects to the wireless network prior to executing the introduction protocol.