Method and System for Providing Autonomous Control of a Platform

• US 20100234993A1
• Filed: 06/19/2009
• Published: 09/16/2010
• Est. Priority Date: 06/19/2008
• Status: Active Grant

• Current Assignee: John-David Yoder, Michael J. Seelinger
• Original Assignee: John-David Yoder, Michael J. Seelinger
• Inventors: Yoder, John-David, Seelinger, Michael J.
• Granted Patent: US 8,229,595 B2
• Time in Patent Office: Days
• US Class Current: 700/254
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/37567 3-D vision, stereo vision, with two cameras
• CPC Class Codes: G05B 2219/39397 Map image error directly to robot movement, position with relation to world, base not needed, image based visual servoing
• CPC Class Codes: G05B 2219/40003 Move end effector so that image center is shifted to desired position
• CPC Class Codes: G05B 2219/40298 Manipulator on vehicle, wheels, mobile
• CPC Class Codes: G05D 1/0248 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0251 extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593

Abstract

The present application provides a system for enabling instrument placement from distances on the order of five meters, for example, and increases accuracy of the instrument placement relative to visually-specified targets. The system provides precision control of a mobile base of a rover and onboard manipulators (e.g., robotic arms) relative to a visually-specified target using one or more sets of cameras. The system automatically compensates for wheel slippage and kinematic inaccuracy ensuring accurate placement (on the order of 2 mm, for example) of the instrument relative to the target. The system provides the ability for autonomous instrument placement by controlling both the base of the rover and the onboard manipulator using a single set of cameras. To extend the distance from which the placement can be completed to nearly five meters, target information may be transferred from navigation cameras (used for long-range) to front hazard cameras (used for positioning the manipulator).

Claims

1. A method for operating an autonomous vehicle that includes a manipulator and one or more sets of cameras on the autonomous vehicle, the method comprising:

calibrating the manipulator with the one or more sets of cameras to establish calibration parameters describing a relationship between a location of features of the manipulator in a two-dimensional image acquired by the one or more sets of cameras and a three-dimensional position of the features of the manipulator;

defining a relationship between a three-dimensional location of a target and a location of the target in a two-dimensional image acquired by the one or more sets of cameras;

using the calibration parameters and the relationship between the three-dimensional location of the target and the location of the target in the two-dimensional image acquired by the one or more sets of cameras to estimate a location of the target relative to the manipulator;

creating a trajectory for the autonomous vehicle and the manipulator to follow to position the autonomous vehicle and the manipulator such that the manipulator can engage the target; and

updating the trajectory as the autonomous vehicle and the manipulator traverse the trajectory.

2. The method of claim 1, wherein the autonomous vehicle further comprises a first set of cameras and a second set of cameras.

3. The method of claim 2, wherein the first set of cameras on the autonomous vehicle are long-range viewing cameras and the second set of cameras on the autonomous vehicle are short-range viewing cameras.

4. The method of claim 2, further comprising, as the autonomous vehicle approaches the target, the first set of cameras transferring information regarding the three-dimensional location of the target to the second set of cameras.

5. The method of claim 4, further comprising:

defining a relationship between the three-dimensional location of the target and a location of the target in a two-dimensional image acquired by the second set of cameras;

using the calibration parameters and the relationship between the three-dimensional location of the target and the location of the target in the two-dimensional image acquired by the second set of cameras to estimate the location of the target relative to the manipulator; and

creating an updated trajectory for the autonomous vehicle and the manipulator to follow to position the autonomous vehicle and the manipulator such that the manipulator can engage the target.

6. The method of claim 4, wherein transferring information regarding the three-dimensional location of the target to the second set of cameras comprises:

positioning a light source to emit a light substantially near the location of the target;

at least one of the second set of cameras acquiring an image including the light;

updating the three-dimensional location of the target based on a location of the light in the image.

7. The method of claim 1, further comprising determining a configuration of the system to position the manipulator approximately at the three-dimensional location of the target.

8. The method of claim 2, further comprising:

positioning a light source to emit a light substantially near the location of the target;

identifying the light within images produced by one of the first set of cameras and the second set of cameras; and

defining the location of the target in the images produced by one of the first set of cameras and the second set of cameras.

9. The method of claim 1, wherein calibrating comprises:

moving the manipulator through a series of positions;

at each positions, acquiring images from the cameras and recording corresponding angles and positions of the manipulator;

in each image, identifying a camera-space location of a feature of the manipulator;

establishing parameters describing a relationship between the camera-space location of the feature of the manipulator and a three-dimensional position of the feature of the manipulator.

10. The method of claim 1, wherein defining a relationship between a three-dimensional location of a target and a location of the target in a two-dimensional image acquired by the one or more sets of cameras comprises using the CAHVOR camera model.

11. The method of claim 1, wherein the CAHVOR camera model includes six vectors of three parameters each for a total of eighteen camera model parameters comprising c={c0,c1,c2}, a={a0,a1,a2}, h={h0,h1,h2}, v={v0,v1,v2}, o={o0,o1,o2}, r={r0,r1,r2}, where the location of the target in the two-dimensional image is described with coordinates (x, y) and the three-dimensional location of the target is described by vector p, and where

12. The method of claim 1, wherein creating the trajectory for the autonomous vehicle and the manipulator to follow comprises:

determining a position relative to the autonomous vehicle at which to move the robotic arm;

determining coordinates of the location of the target; and

determining a trajectory for the autonomous vehicle and instructions for following the trajectory.

13. A system for operating an autonomous vehicle, the system comprising:

a manipulator;

a first set of cameras on the autonomous vehicle for long-range viewing to specify a target;

a second set of cameras on the autonomous vehicle for short-range viewing of the target; and

a processor for executing machine language instructions to perform the functions of;

calibrating the manipulator with the first set of cameras and the second set of cameras to establish calibration parameters describing a relationship between a location of features of the manipulator in a two-dimensional image acquired by the first set of cameras and the second set of cameras and a three-dimensional position of the features of the manipulator;

defining a relationship between a three-dimensional location of a target and a location of the target in a two-dimensional image acquired by the first set of cameras;

using the calibration parameters and the relationship between the three-dimensional location of the target and the location of the target in the two-dimensional image acquired by the first set of cameras to estimate a location of the target relative to the manipulator;

creating a trajectory for the autonomous vehicle and the robotic arm to follow to position the autonomous vehicle and the manipulator such that the robotic arm can engage the target; and

updating the trajectory as the autonomous vehicle and the manipulator traverse the trajectory.

14. The system of claim 13, further comprising additional cameras on the autonomous vehicle for high-precision viewing of the target.

15. The system of claim 14, wherein the machine language instructions are further executable by the processor to perform the function of transferring information regarding a position of the target from one set of cameras to another set of cameras as the autonomous vehicle approaches the target.

16. The system of claim 13, wherein the machine language instructions are further executable by the processor to perform the function of transferring information from one set of cameras regarding the three-dimensional location of the target to another set of cameras.

17. The system of claim 13, further comprising a light source to emit a light substantially near the location of the target, and wherein the machine language instructions are further executable by the processor to perform the function of updating the three-dimensional location of the target based on a location of the light in images of the first set of cameras and the second set of cameras.

18. The system of claim 13, wherein the machine language instructions are further executable by the processor to perform the function of determining a configuration of the system to position the robotic arm approximately at the three-dimensional location of the target.

19. The system of claim 13, wherein the machine language instructions are further executable by the processor to perform the functions of:

positioning a light source to emit a light substantially near the location of the target;

identifying the light within images produced by one set of cameras and another set of cameras; and

defining the location of the target in the images produced by one set of cameras and another set of cameras.

20. A computer program product, comprising a computer usable medium having a computer readable program code embodied therein, the computer readable program code adapted to be executed to implement a method for operating an autonomous vehicle, said method comprising:

calibrating the manipulator with one or more sets of cameras to establish calibration parameters describing a relationship between a location of features of the manipulator in a two-dimensional image acquired by the one or more sets of cameras and a three-dimensional position of the features of the manipulator;

defining a relationship between a three-dimensional location of a target and a location of the target in a two-dimensional image acquired by the one or more sets set of cameras;

using the calibration parameters and the relationship between the three-dimensional location of the target and the location of the target in the two-dimensional image acquired by the one or more sets of cameras to estimate a location of the target relative to the manipulator;

creating a trajectory for the autonomous vehicle and the manipulator to follow to position the autonomous vehicle and the manipulator such that the manipulator can engage the target; and

updating the trajectory as the autonomous vehicle and the manipulator traverse the trajectory.