Method and system for remote control of mobile robot

• US 20010037163A1
• Filed: 05/01/2001
• Published: 11/01/2001
• Est. Priority Date: 05/01/2000
• Status: Active Grant

• Current Assignee: iRobot Corporation
• Original Assignee: iRobot Corporation
• Inventors: Allard, James R.
• Granted Patent: US 6,535,793 B2
• Time in Patent Office: Days
• US Class Current: 700/245
• CPC Class Codes: B25J 9/1689 Teleoperation
• CPC Class Codes: G05B 2219/35506 Camera images overlayed with graphics, model
• CPC Class Codes: G05B 2219/40161 Visual display of machining, operation, remote viewing
• CPC Class Codes: G05B 2219/40169 Display of actual situation at the remote site
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G05D 1/0044 by providing the operator with a computer generated representation of the environment of the vehicle, e.g. virtual reality, maps maps used for automatic navigation G05D1/0274; flight directors G01C23/005
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/027 comprising intertial navigation means, e.g. azimuth detector inertial navigation G01C21/16; inertial navigation combined with non-inertial navigation instruments G01C21/165
• CPC Class Codes: G05D 1/0272 comprising means for registering the travel distance, e.g. revolutions of wheels measuring distance traversed on the ground by vehicles, e.g. using odometers G01C22/00
• CPC Class Codes: G05D 1/0282 generated in a local control room

Abstract

Methods of remote control of a mobile robot and an intuitive user interface for remotely controlling a mobile robot are provided. Using a point-and-click device (405), the user is able to choose a target location (430) within a heads-up display (400) toward which to move a mobile robot. Additional graphical overlays (410 & 412) are provided to aid the user in navigating even in systems with asynchronous communication.

Claims

1. A method for enabling a user to tele-operate a robot, the method comprising the steps of:

(a) providing image information representative of a region around the robot;

(b) providing, using the image information, a user-perceptible image representative of the region around the robot;

(c) enabling the user to designate in the image one or more targets toward which the robot should move; and

(d) enabling the robot to move toward the target.

2. The method of

claim 1

, wherein the step of providing of image information representative of a region around the robot further comprises the step of transmitting a video signal from the robot.

3. The method of

claim 2

, wherein the video signal is transmitted via the Internet.

4. The method of

claim 3

, wherein the user-perceptible image is provided on a computer monitor.

5. The method of

claim 3

, wherein the user-perceptible image is provided on a portable liquid-crystal display.

6. The method of

claim 4

, wherein the step of enabling the user to designate in the image a target toward which the robot should move is performed with a point-and-click selection device.

7. The method of

claim 6

, wherein the step of enabling the user to designate in the image a target toward which the robot should move further comprises the steps of:

(a) selecting an icon;

(b) transporting the icon to a target location on the user-perceptible image.

8. The method of

claim 2

further comprising the step of determining the location of the camera relative to the current location of the robot.

9. The method of

claim 8

wherein the step of enabling the user to designate one or more targets further comprises:

(a) determining the click location within the user-perceptible image selected by the user and (b) determining the target location by projecting said click location into said region around the robot.

10. The method of

claim 9

, wherein the user designates a plurality of targets, further comprising the step of enabling the robot to move toward each of the plurality of targets.

11. The method of

claim 10

, wherein the user designates a plurality of targets, further comprising the step of enabling the robot to move toward each of the plurality of targets in the temporal order in which the targets are designated.

12. The method of

claim 9

, further comprising the step of enabling the robot to detect obstacles in its path.

13. The method of

claim 12

wherein the robot ceases movement toward the target when an obstacle is detected.

14. The method of

claim 12

wherein the robot alters its path to avoid said obstacles while moving toward the target.

15. The method of

claim 9

, wherein the step of providing of image information representative of a region around the robot further comprises the step of transmitting video signal from the robot.

16. The method of

claim 15

, wherein the video signal is transmitted via the Internet.

17. The method of

claim 16

, wherein the user-perceptible image is provided on a computer monitor.

18. The method of

claim 16

, wherein the user-perceptible image is provided on a portable liquid-crystal display.

19. The method of

claim 17

, wherein the step of enabling the user to designate in the image a target toward which the robot should move is performed with a point-and-click selection device.

20. The method of

claim 19

, wherein the step of enabling the user to designate in the image a target toward which the robot should move further comprises the steps of:

(a) selecting an icon;

(b) transporting the icon to a target location on the user-perceptible image.

21. A system for tele-operating a robot comprising (a) a sensor means on a mobile robot for providing image data representative of a region around said robot;

(b) a transmitter means for transmitting said image data to a remote user;

(c) a means for providing a perceptible image representative of the region around the robot;

(d) a means for selecting a target within said perceptible image toward which the robot should move.

22. The system of

claim 21

, wherein the sensor means is a camera.

23. The system of

claim 22

, wherein said transmitter means comprises a wireless Internet connection.

24. The system of

claim 23

, wherein said means for providing a perceptible image comprises a computer monitor.

25. The system of

claim 23

, wherein said means for providing a perceptible image comprises a portable liquid-crystal display.

26. The system of

claim 24

, wherein said means for selecting a target comprises a point-and-click device.

27. A graphical user interface for remotely operating a robot, comprising:

(a) a window for displaying image information representative of a region around the robot;

(b) means for selecting a target location for the robot.

28. The graphical user interface for remotely operating a robot as in

claim 27

further comprising drive controls for the robot.

29. The graphical user interface for remotely operating a robot as in

claim 28

wherein said drive controls include a means for turning the robot to face the means for gathering image data.

30. The graphical user interface for remotely operating a robot as in

claim 27

, wherein said window further comprises superimposed graphical data regarding the orientation of the robot.

31. The graphical user interface for remotely operating a robot as in

claim 30

, wherein said superimposed graphical data comprises an image approximately parallel to the ground plane.

32. The graphical user interface for remotely operating a robot as in

claim 31

, wherein said superimposed graphical data comprises a grid approximately parallel to the ground plane.

33. The graphical user interface for remotely operating a robot as in

claim 30

, wherein said superimposed graphical data comprises a perspective box.

34. The graphical user interface for remotely operating a robot as in

claim 30

, wherein said superimposed graphical data comprises targeting circles.

35. The graphical user interface for remotely operating a robot as in

claim 30

, wherein said window further comprises a targeting reticle.

36. The graphical user interface for remotely operating a robot as in

claim 27

further comprising a plurality of buttons for performing preprogrammed tasks.

37. The graphical user interface for remotely operating a robot as in

claim 27

further comprising a plurality of windows for displaying sensor data.

38. The graphical user interface for remotely operating a robot as in

claim 27

, wherein said image information comprises video transmission from a camera on the robot.

39. The graphical user interface for remotely operating a robot as in

claim 38

further comprising means to control movement of said camera on the robot.

40. The graphical user interface for remotely operating a robot as in

claim 39

wherein said means to control movement of said camera comprises a two-dimensional selection grid.