Mobile Human Interface Robot

• US 20120173018A1
• Filed: 09/23/2011
• Published: 07/05/2012
• Est. Priority Date: 12/30/2010
• Status: Active Grant

• Current Assignee: iRobot Corporation
• Original Assignee: iRobot Corporation
• Inventors: Kearns, Justin H., Taka, Orjeta, Farlow, Timothy S., Pack, Robert Todd, Won, Chikyung, Halloran, Michael, Shamlian, Steven V., Chiappetta, Mark, Allen, Thomas P., Rosenstein, Michael T.
• Granted Patent: US 8,930,019 B2
• Time in Patent Office: Days
• US Class Current: 700/245
• CPC Class Codes: B25J 11/008 Manipulators for service tasks
• CPC Class Codes: B25J 13/006 by means of a wireless system for controlling one or several manipulators
• CPC Class Codes: B25J 13/06 Control stands, e.g. consoles, switchboards
• CPC Class Codes: B25J 19/023 including video camera means
• CPC Class Codes: B25J 5/007 mounted on wheels
• CPC Class Codes: B25J 9/0003 Home robots, i.e. small robots for domestic use
• CPC Class Codes: G05D 1/024 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0248 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0253 extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow determining position or orientation from images G06T7/70
• CPC Class Codes: G05D 1/0255 using acoustic signals, e.g. ultra-sonic singals sonar systems designed for anti-collision purposes G01S15/93
• CPC Class Codes: G05D 1/027 comprising intertial navigation means, e.g. azimuth detector inertial navigation G01C21/16; inertial navigation combined with non-inertial navigation instruments G01C21/165
• CPC Class Codes: G05D 1/0272 comprising means for registering the travel distance, e.g. revolutions of wheels measuring distance traversed on the ground by vehicles, e.g. using odometers G01C22/00
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G06F 1/1632 External expansion units, e.g. docking stations

Abstract

A mobile human interface robot that includes a drive system, a controller in communication with the dive system, and an electronic display supported above the drive system and in communication with the controller. The controller includes a central processing unit, a general purpose graphics processing unit, and memory in electrical communication with the central processing unit and the general purpose graphics processing unit. Moreover, the controller has a display operating state and a driving operating state. The controller executes graphics computations on the general purpose graphics processing unit for displaying graphics on the electronic display during the display operating state; and the controller executes mobility computations on the general purpose graphics processing unit for issuing commands to the drive system during the driving operating state.

Claims

1. A mobile human interface robot comprising:

a drive system;

a controller in communication with the dive system, the controller comprising a central processing unit, a general purpose graphics processing unit, and memory in electrical communication with the central processing unit and the general purpose graphics processing unit, the controller having a display operating state and a driving operating state; and

an electronic display supported above the drive system and in communication with the controller;

wherein the controller executes graphics computations on the general purpose graphics processing unit for displaying graphics on the electronic display during the display operating state; and

wherein the controller executes mobility computations on the general purpose graphics processing unit for issuing commands to the drive system during the driving operating state.

2. The mobile human interface robot of claim 1, wherein the central processing unit can process more than about 1000 million instructions per second (MIPS).

3. The mobile human interface robot of claim 1, wherein the electronic display comprises a tablet computer detachably supported above the drive system and in wireless communication with the controller.

4. The mobile human interface robot of claim 3, wherein the tablet computer comprises a touch screen having a display area of at least 150 square inches.

5. The mobile human interface robot of claim 3, wherein the tablet computer is movable with at least one degree of freedom while attached to the robot.

6. The mobile human interface robot of claim 3, further comprising a display in electric communication with the controller, the tablet computer detachably receivable over the display, the display having an inactive state when the tablet computer is received over the display and an active state when the tablet computer is detached from the display.

7. The mobile human interface robot of claim 1, further comprising a camera supported above the drive system and movable within at least one degree of freedom separately from the display.

8. The mobile human interface robot of claim 7, wherein the camera comprises a volumetric point cloud imaging device positioned to be capable of obtaining a point cloud from a volume of space adjacent the robot.

9. The mobile human interface robot of claim 7, wherein the camera is movable to view an area proximate a front edge of the robot.

10. The mobile human interface robot of claim 1, further comprising a volumetric point cloud imaging device positioned at a height of greater than about one foot above a ground surface and directed to be capable of obtaining a point cloud from a volume of space that includes a floor plane in a direction of movement of the robot.

11. The mobile human interface robot of claim 1, wherein the drive system comprises a holonomic drive system having first, second, and third driven drive wheels, each drive wheel trilaterally spaced about a vertical center axis of the base and each having a drive direction perpendicular to a radial axis with respect to the vertical center axis.

12. The mobile human interface robot of claim 1, wherein the robot comprises:

a base defining a vertical center axis and supporting the controller;

an extendable leg extending upward from the base; and

a torso supported by the leg, actuation of the leg causing a change in elevation of the torso, the electronic display supported above the torso.

13. The mobile human interface robot of claim 12, wherein the robot further comprises:

a neck supported by the torso; and

a head supported by the neck, the neck capable of panning and tilting the head with respect to the torso, the electronic display detachably supported by the head.

14. The mobile human interface robot of claim 1, wherein the controller can operate in both the display operating state and the driving operating state simultaneously.

15. A mobile human interface robot comprising:

a drive system;

a controller in communication with the dive system; and

a portable computing device detachably supported above the drive system and in wireless communication with the controller, the portable computing device comprising a central processing unit, a general purpose graphics processing unit, and memory in electrical communication with the central processing unit and the general purpose graphics processing unit;

wherein the portable computing device executes graphics computations on the general purpose graphics processing unit for displaying graphics on an electronic display; and

wherein the portable computing device executes mobility computations on the general purpose graphics processing unit, the controller issuing drive commands to the drive system based on the mobility computations.

16. The mobile human interface robot of claim 15, wherein the central processing unit can process more than about 1000 million instructions per second (MIPS).

17. The mobile human interface robot of claim 15, wherein the portable computing device comprises a tablet computer having a touch screen defining a display area of at least 150 square inches.

18. The mobile human interface robot of claim 15, wherein the portable computing device is movable with at least one degree of freedom while attached to the robot.

19. The mobile human interface robot of claim 15, further comprising a camera supported above the drive system and movable within at least one degree of freedom separately from the portable computing device.

20. The mobile human interface robot of claim 19, wherein the camera comprises a volumetric point cloud imaging device positioned to be capable of obtaining a point cloud from a volume of space adjacent the robot.

21. The mobile human interface robot of claim 15, further comprising a volumetric point cloud imaging device positioned at a height of greater than about one foot above a ground surface and directed to be capable of obtaining a point cloud from a volume of space that includes a floor plane in a direction of movement of the robot.

22. The mobile human interface robot of claim 15, wherein the drive system comprises a holonomic drive system having first, second, and third driven drive wheels, each drive wheel trilaterally spaced about a vertical center axis of the base and each having a drive direction perpendicular to a radial axis with respect to the vertical center axis.

23. The mobile human interface robot of claim 15, wherein the robot comprises:

a base defining a vertical center axis and supporting the controller;

an extendable leg extending upward from the base; and

a torso supported by the leg, actuation of the leg causing a change in elevation of the torso, the electronic display supported above the torso.

24. The mobile human interface robot of claim 23, wherein the robot further comprises:

a neck supported by the torso; and

a head supported by the neck, the neck capable of panning and tilting the head with respect to the torso, the electronic display detachably supported by the head.

25. A method of operating a mobile robot, the method comprising:

processing a display command on a central processing unit;

executing graphics computations on a general purpose graphics processing unit for displaying graphics on an electronic display of the robot;

processing a mobility command on the central processing unit; and

executing mobility computations on the general purpose graphics processing unit for issuing drive commands to a drive system of the robot.

26. The method of claim 25, wherein the central processing unit can process more than about 1000 million instructions per second (MIPS).

27. The method of claim 25, further comprising dedicating usage of the general purpose graphics processing unit for executing graphics computations while the electronic display is detached from the robot and dedicating usage of the general purpose graphics processing unit for executing mobility computations while the robot is driving.

28. The method of claim 25, further comprising displaying graphics on a tablet computer detachably supported above the drive system and in wireless communication with the central processing unit.

29. The method of claim 25, further comprising moving the electronic display with at least one degree of freedom while attached to the robot.

30. The method of claim 25, further comprising moving a camera supported above the drive system within at least one degree of freedom.

31. The method of claim 30, wherein the camera comprises a volumetric point cloud imaging device positioned to be capable of obtaining a point cloud from a volume of space adjacent the robot.

32. The method of claim 30, wherein the camera is movable to view an area proximate a front edge of the robot.

33. The method of claim 30, wherein the camera comprises a volumetric point cloud imaging device positioned at a height of greater than about one foot above a ground surface and directed to be capable of obtaining a point cloud from a volume of space that includes a floor plane in a direction of movement of the robot.

34. The method of claim 25, further comprising driving in a holonomic manner based on the drive commands received by the drive system, the drive system comprising first, second, and third driven drive wheels, each drive wheel trilaterally spaced about a vertical center axis of the base and each having a drive direction perpendicular to a radial axis with respect to the vertical center axis.

35. A method of operating a mobile robot, the method comprising:

receiving a layout map into memory of the robot, the layout map corresponding to an environment of the robot;

moving the robot in the environment to a layout map location on the layout map;

recording a robot map location on a robot map corresponding to the environment and produced by the robot using a sensor system of the robot;

executing a distortion routine on a general purpose graphics processing unit of the robot to determine a distortion between the robot map and the layout map using the recorded robot map locations and the corresponding layout map locations; and

applying the determined distortion to a target layout map location to determine a corresponding target robot map location.

36. The method of claim 35, further comprising producing the layout map using a graphical user interface electronically displayed by the robot using the general purpose graphics processing unit.

37. The method of claim 35, wherein the distortion routine comprises determining a scaling size, origin mapping, and rotation between the layout map and the robot map using existing layout map locations and recorded robot map locations.

38. The method of claim 37, further comprising applying an affine transformation to the determined scaling size, origin mapping, and rotation to resolve the target robot map location.

39. The method of claim 35, wherein the distortion routine comprises determining a triangulation between layout map locations that bound the target layout map location.

40. The method of claim 39, further comprising determining a scale, rotation, translation, and skew between a triangle mapped in the layout map and a corresponding triangle mapped in the robot map and applying the determined scale, rotation, translation, and skew to the target layout map location to determine the corresponding robot map point.

41. The method of claim 35, wherein the distortion routine comprises:

determining distances between all layout map locations and the target layout map location;

determining a centroid of the layout map locations;

determining a centroid of all recorded robot map locations; and

for each layout map location, determining a rotation and a length scaling to transform a vector running from the layout map centroid to the target layout location into a vector running from the robot map centroid to the target robot map location.

42. The method of claim 35, further comprising:

receiving a drive command to move a destination location on the layout map;

resolving a corresponding destination location on the robot map using the general purpose graphics processing unit; and

controlling a drive system of the robot to move to the destination location on the layout map based on the robot map.

43. A method of operating a mobile robot, the method comprising:

receiving video data;

executing a graphics routine on a general purpose graphics processing unit for displaying the video data on an electronic display;

receiving sensor data from a sensor system of the robot;

executing a mobility routine on the general purpose graphics processing unit for producing a local perceptual space corresponding to an environment around the robot; and

executing a drive command to move to a location in the environment based on the local perceptual space.

44. The method of claim 43, further comprising:

classifying portions of the local perceptual space corresponding to sensed objects located above a ground plane and below a height of the robot as obstacles;

classifying portions of the local perceptual space corresponding to sensed objects below the ground plane as obstacles;

classifying portions of the local perceptual space corresponding to unobstructed area on the ground plane as free space;

classifying all remaining unclassified local perceptual space as unknown; and

executing the drive command to move to a location in the environment corresponding to local perceptual space classified as at least of free space and unknown.

45. The method of claim 44, wherein the classifications decay over time unless persisted with updated sensor data.

46. The method of claim 44, further comprising evaluating predicted robot paths corresponding to feasible robot drive commands by rejecting robot paths moving to locations having a corresponding local perceptual space classified as obstacles or unknown.

47. The method of claim 43, further comprising producing a three-dimensional voxel grid using the sensor data and converting the three-dimensional voxel grid into a two-dimensional grid, each cell of the two-dimensional grid corresponding to a portion of the local perceptual space.

48. The method of claim 43, further comprising producing a grid corresponding to the local perceptual space, each grid cell having the classification of the corresponding local perceptual space, for each grid cell classified as an obstacle or unknown, retrieving a grid point within that grid cell and executing a collision evaluation.

49. The method of claim 48, wherein the collision evaluation comprises rejecting grid points located within a collision circle about a location of the robot.

50. The method of claim 48, wherein the collision evaluation comprises rejecting grid points located within a collision triangle centered about a location of the robot.

51. The method of claim 43, wherein the sensor data comprises three-dimensional depth image data provided by a volumetric point cloud imaging device positioned on the robot to be capable of obtaining a point cloud from a volume of space adjacent the robot.

52. The method of claim 51, further comprising orienting a field of view of the volumetric point cloud imaging device toward an area in the environment corresponding to local perceptual space classified as unknown.

53. The method of claim 51, further comprising rejecting a drive command that moves the robot to a robot position beyond a field of view of the volumetric point cloud imaging device.

54. The method of claim 51, further comprising accepting a drive command to move to a location in the environment having corresponding local perceptual space classified as unknown when the robot determines that a field of view of the volumetric point cloud imaging device covers the location before the robot reaches the location.

55. The method of claim 51, wherein the volumetric point cloud imaging device is positioned on the robot at a height of greater than about 2 feet above the ground and directed to be capable of obtaining a point cloud from a volume of space that includes a floor plane in a direction of movement of the robot.