SYSTEMS AND METHODS FOR OPERATING ROBOTS USING VISUAL SERVOING

• US 20130041508A1
• Filed: 08/13/2012
• Published: 02/14/2013
• Est. Priority Date: 08/12/2011
• Status: Abandoned Application

• Current Assignee: Georgia Tech Research Corporation (University System of Georgia)
• Original Assignee: Georgia Tech Research Corporation (University System of Georgia)
• Inventors: HU, Ai-Ping, McMurray, Gary, Matthews, James Michael, Marshall, Matt
• Application Number:
  US13/584,594
• Publication Number:
  US 20130041508A1
• Time in Patent Office: Days
• US Class Current: 700/259
• CPC Class Codes: B25J 9/162 Mobile manipulator, movable base with manipulator arm mounted on it
• CPC Class Codes: B25J 9/1689 Teleoperation
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 19/427 Teaching successive positions by tracking the position of a joystick or handle to control the positioning servo of the tool head, master-slave control G05B19/423 takes precedence
• CPC Class Codes: G05B 2219/39397 Map image error directly to robot movement, position with relation to world, base not needed, image based visual servoing

Abstract

A system and method for providing intuitive, visual based remote control is disclosed. The system can comprise one or more cameras disposed on a remote vehicle. A visual servoing algorithm can be used to interpret the images from the one or more cameras to enable the user to provide visual based inputs. The visual servoing algorithm can then translate that commanded motion into the desired motion at the vehicle level. The system can provide correct output regardless of the relative position between the user and the vehicle and does not require any previous knowledge of the target location or vehicle kinematics.

Claims

1. A method for providing visual based, intuitive control comprising:

moving one or more elements on a device;

measuring the movement of the one or more elements physically with one or more movement sensors mounted on the one or more elements;

measuring the movement of the one or more elements visually with one or more visual based sensors;

comparing the measurement from the one or more movement sensors to the measurement from the one or more visual based sensors to create a control map; and

inverting the control map to provide visual based control of the device.

2. The method of claim 1, further comprising:

receiving a control input from a controller to move the device in a first direction with respect to the visual based sensor; and

transforming the control input to move the one or more elements of the device to move the device in the first direction.

3. The method of claim 2, wherein the controller comprises one or more joysticks.

4. The method of claim 1, wherein the one or more visual based sensors comprise a 2-D video camera.

5. The method of claim 1, wherein the one or more visual based sensors comprise stereoscopic 2-D video cameras.

6. The method of claim 1, wherein the device is a robotic arm;

wherein the one or more elements comprise one or more joints; and

wherein each of the one or more joints rotates, translates, or both.

7. The method of claim 1, wherein visually measuring the movement of the one or more elements comprises:

identifying one or more key objects in a first image captured by the visual based sensor;

moving one or more of the elements of the device;

reidentifying the one or more key objects in a second image captured by the visual based sensor; and

comparing the relative location of the one or more key objects in the first image and the second image.

8. A system for providing visual based, intuitive control comprising:

a device comprising one or more moveable elements each element capable of translation, rotation, or both, and each element comprising one or more movement sensors for physically measuring the movement of the element;

one or more image sensors for visually measuring the movement of the one or more elements; and

a computer processor for;

receiving physical movement data from the one or more movement sensors;

receiving visual movement data from the one or more image sensors;

comparing the physical movement data to the visual movement data to create a control map; and

inverting the control map to provide visual based control of the device.

9. The system of claim 8, the computer processor further:

receiving a control input from a controller to move the device in a first direction with respect to the visual based sensor; and

transforming the control input to move the one or more elements of the device to move the device in the first direction.

10. The system of claim 9, wherein the device comprises a robotic arm with one or more joints.

11. The system of claim 10, the robotic arm further comprising an end-effector.

12. The system of claim 8, wherein the one or more image sensors comprise one or more 3-D time-of-flight cameras.

13. The system of claim 8, wherein the one or more image sensors comprise one or more infrared cameras.