Interactive robot initialization

• US 9,259,842 B2
• Filed: 06/10/2011
• Issued: 02/16/2016
• Est. Priority Date: 06/10/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Fouillade, Jean Sebastien, Sanchez, Russell, Papaefstathiou, Efstathios, Chalabi, Malek M.
• Primary Examiner(s): Tran, Khoi
• Assistant Examiner(s): MOTT, ADAM R
• Application Number:
  US13/157,327
• Publication Number:
  US 20120316676A1
• Time in Patent Office: 1,712 Days
• US Class Current: 1/1
• CPC Class Codes: A47L 2201/00 Robotic cleaning machines, i.e. with automatic control of the travelling movement or the cleaning operation
• CPC Class Codes: B25J 11/0005 Manipulators having means for high-level communication with users, e.g. speech generator, face recognition means
• CPC Class Codes: B25J 13/003 by means of an audio-responsive input audible safety signals B25J19/061
• CPC Class Codes: B25J 19/023 including video camera means
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
• CPC Class Codes: H04L 63/083 using passwords cryptographic mechanisms or cryptographic arrangements for entity authentication using a predetermined code H04L9/3226
• CPC Class Codes: Y10S 901/01 Mobile robot
• CPC Class Codes: Y10S 901/47 Optical

Abstract

Initial interaction between a mobile robot and at least one user is described herein. The mobile robot captures several images of its surroundings, and identifies existence of a user in at least one of the several images. The robot then orients itself to face the user, and outputs an instruction to the user with regard to the orientation of the user with respect to the mobile robot. The mobile robot captures images of the face of the user responsive to detecting that the user has followed the instruction. Information captured by the robot is uploaded to a cloud-storage system, where information is included in a profile of the user and is shareable with others.

Claims

1. A robot, comprising:

a processor; and

memory that comprises instructions that, when executed by the processor, cause the processor to perform acts comprising;

in response to the robot initially being powered and when used for a first time, searching for available wireless networks;

based upon input from a user, connecting to a wireless network in the available wireless networks;

in response to connecting to the wireless network, executing an introduction protocol that is configured to introduce the robot to a user who is in a same room with the robot when the robot is used for the first time, wherein executing the introduction protocol comprises;

using a camera, scanning surroundings of the robot to capture images of the surroundings of the robot;

recognizing that an image in the captured images includes the user;

responsive to recognizing that the image includes the user, orienting a body of the robot such that the robot faces the user;

using the camera, capturing an image of a face of the user subsequent to the body of the robot being oriented such that the robot faces the user; and

retaining the image of the face of the user for use in connection with identifying the user during a subsequent interaction of the user with the robot.

2. The robot of claim 1, wherein the memory further comprises instructions that, when executed by the processor, cause the robot to perform acts comprising:

during a subsequent interaction with the user;

using the camera to capture another image of the user;

comparing the another image of the user with the image of the face of the user; and

identifying the user based at upon the comparing of the another image of the user with the image of the face of the user.

3. The robot of claim 1, wherein executing the introduction protocol further comprises transmitting the image of the face of the user to a cloud-based data store, the image of the face retained in the data store as a portion of a profile of the user.

4. The robot of claim 1, wherein scanning the surroundings comprises performing a 360 degree scan of the surroundings, and wherein recognizing that the image in the captured images includes the user is performed subsequent to the 360 degree scan of the surroundings being performed.

5. The robot of claim 1, wherein scanning the surroundings comprises capturing images of the surroundings until it is recognized that the image in the captured images includes the user, and wherein scanning of the surroundings continues subsequent to the image of the face of the user being captured.

6. The robot of claim 1, wherein executing the introduction protocol further comprises:

subsequent to orienting the body of the robot such that the robot faces the user, instructing the user to provide a voice sample to the robot;

capturing the voice sample from the user; and

retaining the voice sample of the user in computer-readable storage for use in connection with identifying the user during the subsequent interaction of the user with the robot.

7. The robot of claim 6, wherein the memory further comprises instructions that, when executed by the robot, cause the robot to perform acts comprising:

during a subsequent interaction with the user;

capturing an audio signal through use of a microphone;

comparing the audio signal with the captured voice sample; and

recognizing the user based upon the comparing of the audio signal with the captured voice signal.

8. The robot of claim 1, wherein executing the introduction protocol further comprises outputting a request to the user for information pertaining to the user.

9. The robot of claim 8, wherein executing the introduction protocol further comprises recognizing a voice command from the user as a response to the request.

10. The robot of claim 8, further comprising a keypad that receives a response to the request from the user.

11. The robot of claim 8, further comprising an antenna that receives a response to the request from a portable computing device.

12. The robot of claim 8, wherein executing the introduction protocol further comprises recognizing a gesture of the user as a response to the request.

13. The robot of claim 1, wherein the camera is an infrared camera.

14. A method performed by a mobile robot, comprising:

immediately subsequent to an initial powering of the robot and when the robot is being used for a first time;

connecting to a wireless local area network (LAN);

responsive to connecting to the wireless LAN, executing an introduction protocol,

the introduction protocol configured to introduce the robot to a user who will subsequently interact with the robot, wherein executing the introduction protocol comprises;

using a camera to capture a plurality of images of surroundings of the robot;

identifying existence of the user in an image in the plurality of images without determining an identity of the user;

responsive to identifying existence of the user in the image, orienting the robot to cause the user to be in a center of a field of view of the camera;

outputting an instruction to the user pertaining to orientation of the user with respect to the robot;

detecting that the user has responded to the instruction;

capturing another image of the user responsive to detecting that the user has responded to the instruction; and

retaining the another image of the user for use in connection with identifying the user during subsequent interactions between the robot and the user.

15. The method of claim 14, wherein the instruction comprises an audible command output by the robot.

16. The method of claim 14, wherein the instruction comprises a visual instruction displayed on a display of the robot.

17. The method of claim 14, wherein the instruction comprises a projected instruction that is projected from a projector on the robot.

18. The method of claim 14, further comprising:

subsequent to executing the introduction protocol;

during a subsequent interaction between the robot and the user, capturing a second image of the user; and

identifying the user based upon a comparison between the another image and the second image.

19. The method of claim 14, further comprising:

subsequent to executing the introduction protocol;

capturing data by way of a sensor on the robot;

recognizing an event based upon the another image of the user and the captured data; and

initiating an action responsive to recognizing the event.

20. A robot that comprises a computer-readable medium, the computer-readable medium comprising instructions that, when executed by a processor of the robot, causes the processor of the robot to perform acts comprising:

responsive to the robot being initially powered and during a first use of the robot, performing a scan for available wireless networks;

responsive to receiving a password for a wireless network in the wireless networks, connecting to the wireless network;

responsive to connecting to the wireless network, executing an introduction protocol, the introduction protocol configured to introduce the robot to a user who will have subsequent interactions with the robot, wherein executing the introduction protocol comprises;

performing a 360 degree video scan of surroundings of the robot;

recognizing that the user is included in an image captured during the video scan without identifying the user;

rotating a body of the robot to face the user responsive to recognizing that the user is included in the image;

outputting an instruction to the user pertaining to orientation of the user with respect to the robot;

detecting that the user has followed the instruction;

responsive to detecting that the user has followed the instruction, capturing another image of the user;

extracting features from the another image that are indicative of an identity of the user; and

retaining the features extracted from the another image to facilitate identifying the user during subsequent interaction between the robot and the individual.