Mobile human interface robot

• US 8,918,209 B2
• Filed: 02/22/2011
• Issued: 12/23/2014
• Est. Priority Date: 05/20/2010
• Status: Active Grant

• Current Assignee: Ava Robotics, Inc.
• Original Assignee: iRobot Corporation
• Inventors: Rosenstein, Michael T., Halloran, Michael, Shamlian, Steven V., Won, Chikyung, Chiappetta, Mark
• Primary Examiner(s): Tran, Khoi
• Assistant Examiner(s): NGUYEN, ROBERT T
• Application Number:
  US13/032,312
• Publication Number:
  US 20120185094A1
• Time in Patent Office: 1,400 Days
• Field of Search: 701/245, 701/250, 701/254, 701/255, 701/258, 701/259, 901/1, 901/46, 901/47
• US Class Current: 700/254
• CPC Class Codes: B25J 11/009 Nursing, e.g. carrying sick persons, pushing wheelchairs, distributing drugs
• CPC Class Codes: B25J 13/08 by means of sensing devices, e.g. viewing or touching devices
• CPC Class Codes: B25J 19/023 including video camera means
• CPC Class Codes: B25J 5/007 mounted on wheels
• CPC Class Codes: B25J 9/1694 characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
• CPC Class Codes: G02B 13/22 Telecentric objectives or lens systems
• CPC Class Codes: G05D 1/0227 using mechanical sensing means, e.g. for sensing treated area
• CPC Class Codes: G05D 1/024 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0242 using non-visible light signals, e.g. IR or UV signals
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0248 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0251 extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593
• CPC Class Codes: G05D 1/0255 using acoustic signals, e.g. ultra-sonic singals sonar systems designed for anti-collision purposes G01S15/93
• CPC Class Codes: G05D 1/027 comprising intertial navigation means, e.g. azimuth detector inertial navigation G01C21/16; inertial navigation combined with non-inertial navigation instruments G01C21/165
• CPC Class Codes: G05D 1/0272 comprising means for registering the travel distance, e.g. revolutions of wheels measuring distance traversed on the ground by vehicles, e.g. using odometers G01C22/00
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/271 : wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 2013/0081 : Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: Y10S 901/01 : Mobile robot
• CPC Class Codes: Y10S 901/09 : Closed loop, sensor feedback controls arm movement
• CPC Class Codes: Y10S 901/47 : Optical

Abstract

A mobile robot that includes a drive system, a controller in communication with the drive system, and a volumetric point cloud imaging device supported above the drive system at a height of greater than about one feet above the ground and directed to be capable of obtaining a point cloud from a volume of space that includes a floor plane in a direction of movement of the mobile robot. The controller receives point cloud signals from the imaging device and issues drive commands to the drive system based at least in part on the received point cloud signals.

Claims

1. A mobile robot comprising:

a robot body;

a drive system supported by the robot body, the drive system maneuvering the robot over a work surface of a scene and measuring odometry;

an imaging sensor disposed on the robot body and pointing downward toward the work surface along a forward drive direction of the drive system, the imaging sensor capturing three-dimensional images of a scene about the robot; and

a controller in communication with the drive system and the imaging sensor, wherein the controller;

receives the odometry from the drive system and attributes one or more time stamps to the drive system odometry;

receives point cloud signals from the imaging sensor and attributes one or more time stamps to the point cloud signals;

determines visual odometry based on the received point cloud signals;

compares the drive system odometry and the visual odometry based on the time stamps;

determines an error between the drive system odometry and the visual odometry; and

issues drive commands to the drive system based at least in part on the error between the drive system odometry and the visual odometry.

2. The mobile robot of claim 1, wherein the imaging sensor comprises:

a speckle emitter emitting a speckle pattern of light onto the scene; and

an imager receiving reflections of the speckle pattern from objects in the scene;

wherein the controller

stores reference images of the speckle pattern as reflected off a reference object in the scene, the reference images captured at different distances from the reference object; and

compares at least one target image of the speckle pattern as reflected off a target object in the scene with the reference images for determining a distance of reflecting surfaces of the target object.

3. The mobile robot of claim 2, wherein the imaging sensor captures images of the scene along a drive direction of the robot, the images comprising at least one of (a) a three-dimensional depth image, (b) an active illumination image, and (c) an ambient illumination image.

4. The mobile robot of claim 2, wherein the controller determines a location of an object in the scene based on the image comparison and issues drive commands to the drive system to maneuver the robot in the scene based on the object location.

5. The mobile robot of claim 2, wherein the controller determines a primary speckle pattern on the target object and computes at least one of a respective cross-correlation and a decorrelation between the primary speckle pattern and the speckle patterns of the reference images.

6. The mobile robot of claim 1, wherein the imaging sensor comprises a volumetric point cloud imaging device positioned at a height of greater than 2 feet above the ground and directed to be capable of obtaining a point cloud from a volume of space that includes a floor plane in a direction of movement of the robot.

7. The mobile robot of claim 1, wherein the drive system comprises at least one drive wheel, the imaging sensor is arranged on the robot body to view the work surface forward of the at least one drive wheels of the drive system.

8. The mobile robot of claim 1, wherein the imaging sensor has a horizontal field of view of at least 45 degrees and a vertical field of view of at least 40 degrees.

9. The mobile robot of claim 1, wherein the imaging sensor has range of between about 1 meter and about 5 meters.

10. The mobile robot of claim 1, wherein the imaging sensor scans side-to-side with respect to the forward drive direction to increase a lateral field of view of the imaging sensor.

11. The mobile robot of claim 1, wherein the imaging sensor has a latency of about 44 ms.

12. The mobile robot of claim 11, wherein imaging output of the imaging sensor receives a time stamp for compensating for latency.

13. The mobile robot of claim 1, wherein the imaging sensor comprises a serial peripheral interface bus for communicating with the controller.

14. The mobile robot of claim 1, wherein the imaging sensor is recessed within the robot body while maintaining its downward field of view.

15. The mobile robot of claim 1, further comprising an array of sonar proximity sensors disposed around a base of the robot body and aiming upward to provide a sonar detection curtain around the robot for detecting objects encroaching on the robot body.

16. The mobile robot of claim 15, wherein the array of sonar proximity sensors aim away from the robot body.

17. The mobile robot of claim 1, further comprising a laser scanner in communication with the controller and having a field of view centered on the forward drive direction and substantially parallel to the work surface.

18. The mobile robot of claim 1, wherein the drive system comprises at least one drive wheel, each drive wheel comprises first and second rows of rollers disposed about a periphery of the drive wheel, each roller having a rolling direction perpendicular to a rolling direction of the drive wheel, the rollers each defining an arcuate rolling surface, together the rollers defining an at least substantially circular rolling surface of the drive wheel.