Mobile Human Interface Robot

• US 20120185094A1
• Filed: 02/22/2011
• Published: 07/19/2012
• Est. Priority Date: 05/20/2010
• Status: Active Grant

• Current Assignee: Ava Robotics, Inc.
• Original Assignee: iRobot Corporation
• Inventors: Rosenstein, Michael, Halloran, Michael, Shamlian, Steven V., Won, Chikyung, Chiappetta, Mark
• Granted Patent: US 8,918,209 B2
• Time in Patent Office: Days
• US Class Current: 700/259
• CPC Class Codes: B25J 11/009 Nursing, e.g. carrying sick persons, pushing wheelchairs, distributing drugs
• CPC Class Codes: B25J 13/08 by means of sensing devices, e.g. viewing or touching devices
• CPC Class Codes: B25J 19/023 including video camera means
• CPC Class Codes: B25J 5/007 mounted on wheels
• CPC Class Codes: B25J 9/1694 characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
• CPC Class Codes: G02B 13/22 Telecentric objectives or lens systems
• CPC Class Codes: G05D 1/0227 using mechanical sensing means, e.g. for sensing treated area
• CPC Class Codes: G05D 1/024 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0242 using non-visible light signals, e.g. IR or UV signals
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0248 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0251 extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593
• CPC Class Codes: G05D 1/0255 using acoustic signals, e.g. ultra-sonic singals sonar systems designed for anti-collision purposes G01S15/93
• CPC Class Codes: G05D 1/027 comprising intertial navigation means, e.g. azimuth detector inertial navigation G01C21/16; inertial navigation combined with non-inertial navigation instruments G01C21/165
• CPC Class Codes: G05D 1/0272 comprising means for registering the travel distance, e.g. revolutions of wheels measuring distance traversed on the ground by vehicles, e.g. using odometers G01C22/00
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/271 : wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 2013/0081 : Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: Y10S 901/01 : Mobile robot
• CPC Class Codes: Y10S 901/09 : Closed loop, sensor feedback controls arm movement
• CPC Class Codes: Y10S 901/47 : Optical

Abstract

A mobile robot that includes a drive system, a controller in communication with the drive system, and a volumetric point cloud imaging device supported above the drive system at a height of greater than about one feet above the ground and directed to be capable of obtaining a point cloud from a volume of space that includes a floor plane in a direction of movement of the mobile robot. The controller receives point cloud signals from the imaging device and issues drive commands to the drive system based at least in part on the received point cloud signals.

Claims

1. A mobile robot comprising:

a drive system;

a controller in communication with the drive system; and

a volumetric point cloud imaging device supported above the drive system at a height of greater than about one feet above the ground and directed to be capable of obtaining a point cloud from a volume of space that includes a floor plane in a direction of movement of the mobile robot;

wherein the controller receives point cloud signals from the imaging device and issues drive commands to the drive system based at least in part on the received point cloud signals.

2. The mobile robot of claim 1, wherein the controller comprises a computer capable of processing greater than 1000 million instructions per second (MIPS).

3. The mobile robot of claim 1, wherein the imaging device emits light onto a scene about the robot and captures images of the scene along a drive direction of the robot, the images comprising at least one of (a) a three-dimensional depth image, (b) an active illumination image, and (c) an ambient illumination image;

wherein the controller determines a location of an object in the scene based on the images and issues drive commands to the drive system to maneuver the robot in the scene based on the object location.

4. The mobile robot of claim 3, wherein the imaging device determines a time-of-flight between emitting the light and receiving reflected light from the scene, the controller using the time-of-flight for determining a distance to the reflecting surfaces of the object.

5. The mobile robot of claim 3, wherein the imaging device comprises:

a light source for emitting light; and

an imager for receiving reflections of the emitted light from the scene;

wherein the imager comprises an array of light detecting pixels.

6. The mobile robot of claim 5, wherein the light sensor emits the light onto the scene in intermittent pulses.

7. The mobile robot of claim 6, wherein the light sensor emits the light pulses at a first, power saving frequency and upon receiving a sensor event emits the light pulses at a second, active frequency.

8. The mobile robot of claim 7, wherein the sensor event comprises a sensor signal indicative of the presence of an object in the scene.

9. The mobile robot of claim 1, wherein the imaging device comprises first and second portions,

the first portion arranged to emit light substantially onto the ground and receive reflections of the emitted light from the ground, and

the second portion arranged to emit light into a scene substantially above the ground and receive reflections of the emitted light from a scene about the robot.

10. The mobile robot of claim 1, wherein the imaging device comprises:

a speckle emitter emitting a speckle pattern of light onto a scene along a drive direction of the robot; and

an imager receiving reflections of the speckle pattern from the object in the scene;

wherein the controller

stores reference images of the speckle pattern as reflected off a reference object in the scene, the reference images captured at different distances from the reference object; and

compares at least one target image of the speckle pattern as reflected off a target object in the scene with the reference images for determining a distance of the reflecting surfaces of the target object.

11. The mobile robot of claim 10, wherein the controller determines a primary speckle pattern on the target object and computes at least one of a respective cross-correlation and a decorrelation between the primary speckle pattern and the speckle patterns of the reference images.

12. A mobile robot comprising:

a base;

a holonomic drive system supported by the base, the drive system having first, second, and third drive wheels, each trilaterally spaced about the vertical center axis, each drive wheel having a drive direction perpendicular to a radial axis with respect to the vertical center axis, the holonomic drive system maneuvering the robot over a work surface of a scene;

a controller in communication with the drive system;

a leg extending upward from the base and having a variable height;

a torso supported by the leg, the torso defining a shoulder having a bottom surface overhanging the base; and

an imaging sensor disposed on the bottom surface of the torso and pointing downward along a forward drive direction of the drive system, the imaging sensor capturing three-dimensional images of a scene about the robot.

13. The mobile robot of claim 12, wherein the imaging sensor comprises:

a speckle emitter emitting a speckle pattern of light onto the scene; and

an imager receiving reflections of the speckle pattern from the object in the scene;

wherein the controller

stores reference images of the speckle pattern as reflected off a reference object in the scene, the reference images captured at different distances from the reference object; and

compares at least one target image of the speckle pattern as reflected off a target object in the scene with the reference images for determining a distance of the reflecting surfaces of the target object.

14. The mobile robot of claim 13, wherein the imaging sensor captures images of the scene along a drive direction of the robot, the images comprising at least one of (a) a three-dimensional depth image, (b) an active illumination image, and (c) an ambient illumination image.

15. The mobile robot of claim 13, wherein the controller determines a location of an object in the scene based on the image comparison and issues drive commands to the drive system to maneuver the robot in the scene based on the object location.

16. The mobile robot of claim 13, wherein the controller determines a primary speckle pattern on the target object and computes at least one of a respective cross-correlation and a decorrelation between the primary speckle pattern and the speckle patterns of the reference images.

17. The mobile robot of claim 12, wherein the imaging sensor comprises a volumetric point cloud imaging device positioned at a height of greater than 2 feet above the ground and directed to be capable of obtaining a point cloud from a volume of space that includes a floor plane in a direction of movement of the robot.

18. The mobile robot of claim 12, wherein the imaging sensor is arranged on the torso to view the work surface forward of the drive wheels of the drive system.

19. The mobile robot of claim 12, wherein the imaging sensor has a horizontal field of view of at least 45 degrees and a vertical field of view of at least 40 degrees.

20. The mobile robot of claim 12, wherein the imaging sensor has range of between about 1 meter and about 5 meters.

21. The mobile robot of claim 12, wherein the imaging sensor scans side-to-side with respect to the forward drive direction to increase a lateral field of view of the imaging sensor.

22. The mobile robot of claim 12, wherein the imaging sensor has a latency of about 44 ms.

23. The mobile robot of claim 22, wherein imaging output of the imaging sensor receives a time stamp for compensating for latency.

24. The mobile robot of claim 12, wherein the imaging sensor comprises a serial peripheral interface bus for communicating with the controller.

25. The mobile robot of claim 12, wherein the imaging sensor is recessed within a body of the torso while maintaining its downward field of view.

26. The mobile robot of claim 12, further comprising an array of sonar proximity sensors disposed around the base and aiming upward to provide a sonar detection curtain around the robot for detecting objects encroaching on at least one of the leg and the torso.

27. The mobile robot of claim 26, wherein the array of sonar proximity sensors aim away from the torso.

28. The mobile robot of claim 12, further comprising a laser scanner in communication with the controller and having a field of view centered on the forward drive direction and substantially parallel to the work surface.

29. The mobile robot of claim 12, wherein each drive wheel comprises first and second rows of rollers disposed about a periphery of the drive wheel, each roller having a rolling direction perpendicular to a rolling direction of the drive wheel.

30. The mobile robot of claim 29, wherein the rollers each define an arcuate rolling surface, together the rollers defining an at least substantially circular rolling surface of the drive wheel.

31. A self-propelled teleconferencing platform for tele-presence applications, the self-propelled teleconferencing platform comprising:

a drive system chassis supporting a drive system;

a computer chassis disposed above the drive system chassis and supporting a computer capable of processing greater than 1000 million instructions per second (MIPS);

a display supported above the computer chassis; and

a camera supported above the computer chassis and movable within at least one degree of freedom separately from the display, the camera having an objective lens positioned more than 3 feet from the ground and less than 10 percent of a display height from a top edge of a display area of the display.

32. The self-propelled teleconferencing platform of claim 31, wherein the camera comprises a volumetric point cloud imaging device positioned at a height greater than about 1 feet above the ground and directed to be capable of obtaining a point cloud from a volume of space that includes a floor plane in a direction of movement of the platform.

33. The self-propelled teleconferencing platform of claim 31, wherein the camera comprises a volumetric point cloud imaging device positioned to be capable of obtaining a point cloud from a volume of space adjacent the platform.

34. The self-propelled teleconferencing platform of claim 31, wherein the display has a display area of at least 150 square inches and is movable with at least one degree of freedom.

35. The self-propelled teleconferencing platform of claim 31, wherein the objective lens of the camera comprises a zoom lens.

36. The self-propelled teleconferencing platform of claim 31, further comprising a battery configured to power the computer for at least three hours.

37. The self-propelled teleconferencing platform of claim 31, wherein the drive system comprises a motorized omni-directional drive.

38. The self-propelled teleconferencing platform of claim 31, wherein the drive system comprises first, second, and third drive wheels, each trilaterally spaced about a vertical center axis and supported by the drive system chassis, each drive wheel having a drive direction perpendicular to a radial axis with respect to the vertical center axis.

39. The self-propelled teleconferencing platform of claim 31, further comprising:

a leg extending upward from the drive system chassis and having a variable height;

a torso supported by the leg, the torso defining a shoulder having a bottom surface overhanging the base; and

a neck supported by the torso; and

a head supported by the neck, the neck panning and tilting the head with respect to the vertical center axis, the display and the camera both supported by the head.

40. The self-propelled teleconferencing platform of claim 39, further comprising a torso imaging sensor disposed on the bottom surface of the torso and pointing downward along a forward drive direction of the drive system, the torso imaging sensor capturing three-dimensional images of a scene about the robot.

41. The self-propelled teleconferencing platform of claim 39, further comprising a head imaging sensor mounted on the head and capturing three-dimensional images of a scene about the robot.