Mobile Robot System

• US 20150158182A1
• Filed: 02/19/2015
• Published: 06/11/2015
• Est. Priority Date: 05/20/2010
• Status: Active Grant

• Current Assignee: iRobot Corporation
• Original Assignee: iRobot Corporation
• Inventors: Farlow, Timothy S., Rosenstein, Michael T., Halloran, Michael, Won, Chikyung, Shamlian, Steven V., Chiappetta, Mark
• Granted Patent: US 9,902,069 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B25J 11/009 Nursing, e.g. carrying sick persons, pushing wheelchairs, distributing drugs
• CPC Class Codes: B25J 19/023 including video camera means
• CPC Class Codes: B25J 5/007 mounted on wheels
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05D 1/0011 associated with a remote control arrangement
• CPC Class Codes: G05D 1/0022 characterised by the communication link data switching networks in general H04L12/00
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G05D 1/0227 using mechanical sensing means, e.g. for sensing treated area
• CPC Class Codes: G05D 1/024 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0242 using non-visible light signals, e.g. IR or UV signals
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0255 using acoustic signals, e.g. ultra-sonic singals sonar systems designed for anti-collision purposes G01S15/93
• CPC Class Codes: G05D 1/027 comprising intertial navigation means, e.g. azimuth detector inertial navigation G01C21/16; inertial navigation combined with non-inertial navigation instruments G01C21/165
• CPC Class Codes: G05D 1/0272 comprising means for registering the travel distance, e.g. revolutions of wheels measuring distance traversed on the ground by vehicles, e.g. using odometers G01C22/00
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04N 7/144 camera and display on the same optical axis, e.g. optically multiplexing the camera and display for eye to eye contact
• CPC Class Codes: Y10S 901/01 Mobile robot
• CPC Class Codes: Y10S 901/47 Optical

Abstract

A robot system includes a mobile robot having a controller executing a control system for controlling operation of the robot, a cloud computing service in communication with the controller of the robot, and a remote computing device in communication with the cloud computing service. The remote computing device communicates with the robot through the cloud computing service.

Claims

1. A method of operating a robot, the method comprising:

maneuvering the robot about a scene;

receiving sensor data indicative of the scene;

communicating the sensor data to a cloud computing service that processes the received sensor data and communicates a process resultant to the robot; and

maneuvering the robot in the scene based on the received process resultant;

wherein the cloud computing service receives image data from the robot and processes the image data into a 3-D map and/or model of the scene.

2. The method of claim 1, further comprising:

emitting light onto the scene about the robot; and

capturing images of the scene along a drive direction of the robot, the images comprising at least one of (a) a three-dimensional depth image, (b) an active illumination image, and (c) an ambient illumination image;

wherein the sensor data comprises the images and the process resultant comprises a map or a model of the scene.

3. The method of claim 2, further comprising:

emitting a speckle pattern of light onto the scene;

receiving reflections of the speckle pattern from an object in the scene;

storing reference images in cloud storage of the cloud computing service of the speckle pattern as reflected off a reference object in the scene, the reference images captured at different distances from the reference object; and

capturing at least one target image of the speckle pattern as reflected off a target object in the scene and communicating the at least one target image to the cloud computing service;

wherein the cloud computing service compares the at least one target image with the reference images for determining a distance of the reflecting surfaces of the target object.

4. The method of claim 3, further comprising determining a primary speckle pattern on the target object and computing at least one of a respective cross-correlation and a decorrelation between the primary speckle pattern and the speckle patterns of the reference images.

5. The method of claim 1, wherein the cloud computing service at least temporarily stores the received sensor data in cloud storage and optionally discards the stored sensor data after processing the data.

6. The method of claim 1, wherein the sensor data comprises image data having associated sensor system data, the sensor system data comprising at least one of accelerometer data traces, odometry data, or a timestamp.

7. The method of claim 1, wherein the cloud computing service provides a 2-D height map and/or the model to the robot, the cloud computing service computing the 2-D height map from the 3-D map.

8. The method of claim 1, wherein the map includes an indoor or outside street view or path view of the scene.

9. The method of claim 8, further comprising marking reference locations on the map and path planning through the reference locations marked on the map.

10. The method of claim 1, wherein the scene is a mall of stores and the map includes a path view with each store marked on the map as a reference location.

11. The method of claim 10, wherein each reference location marked on the map further comprises linked images, video, or promotional information.

12. The method of claim 1, wherein processing the image data further comprises building the map of the scene around the robot by linking together pictures or video captured by a camera or volumetric point cloud imaging device positioned on the robot and using reference coordinates, as provided by odometry, a global positioning system and/or way-point navigation.

13. A method of navigating a robot, the method comprising:

capturing a streaming sequence of dense images of a scene about the robot along a locus of motion of the robot at a real-time capture rate;

associating annotations with at least some of the dense images;

sending the dense images and annotations to a remote server at a send rate, the send rate being slower than the real-time capture rate;

receiving a data set from the remote server after a processing time interval, the data set derived from and representing at least a portion of the sequence of dense images and corresponding annotations, the data set excluding raw image data of the sequence of dense images; and

moving the robot with respect to the scene based on the received data set.

14. The method of claim 13, further comprising sending the dense images and annotations to a local server and a buffer and then sending the dense images and annotations to the remote server at the send rate.

15. The method of claim 13, wherein the annotations comprise:

a time stamp corresponding to at least some of the dense images; and

pose-related sensor data, the pose-related sensor data including at least one of odometry data, accelerometer data, tilt data, or angular rate data.

16. The method of claim 13, wherein associating annotations comprises associating annotations that reflect hazard events with dense images captured in a time interval relative to a hazard response of the robot.

17. The method of claim 13, wherein associating annotations comprises associating key-frame identifiers with a subset of the dense images.

18. The method of claim 13, wherein the annotations comprise a sparse set of 3-D points derived from structure and motion recovery of features tracked between dense images of the streaming sequence of dense images.

19. The method of claim 18, wherein the annotations comprise camera parameters, the camera parameters including a camera pose relative to individual 3-D points of the sparse set of 3-D points.

20. The method of claim 13, wherein the annotations comprise labels of traversable and non-traversable regions of the scene.

21. The method of claim 13, wherein the data set comprises one or more texture maps extracted from the dense images and/or a terrain map representing features within the dense images of the scene.

22. The method of claim 13, wherein the data set comprises a trained classifier for classifying features within new dense images captured of the scene.