Means and method of camera space manipulation

• US 4,833,383 A
• Filed: 08/13/1987
• Issued: 05/23/1989
• Est. Priority Date: 08/13/1987
• Status: Expired due to Term

• Current Assignee: Iowa State University Research Foundation Incorporated (Iowa State University)
• Original Assignee: Iowa State University Research Foundation Incorporated (Iowa State University)
• Inventors: Skarr, Steven B., Brockman, William H.
• Primary Examiner(s): Ro, Bentsu
• Application Number:
  US07/085,018
• Time in Patent Office: 649 Days
• Field of Search: 318/568, 901/2, 901/9, 901/46, 901/47
• US Class Current: 318/568.16
• CPC Class Codes: B25J 19/023 including video camera means
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/39391 Visual servoing, track end effector with camera image feedback

Abstract

A means and method for camera space manipulation includes a manipulator arm extending from a base to an outward end. The arm is movable through a work space to accomplish various tasks. One or more cameras are movably oriented towards the arm and work space to capture the arm and work space in what will be called camera space or camera vision. A visual cue is associated with the outward end of the manipulator arm. Additionally, a visual cue is associated with an object which is desired to be engaged by the manipulator arm or by what is held by the manipulator arm. A control device is connected to the camera or cameras and to the manipulator. According to identification and tracking of the visual cues in the camera space, the control device instructs appropriate motors to move the manipulator arm according to estimations for engagement calculated by the control device.

Claims

1. A camera space manipulation control system, utilizing at least one camera means for engaging a manipulator means with an object where there is not required any known prior three-dimensional physical space relationship between the manipulator means the object, and the camera means, between physical space and the two dimensional image at the focal plane of the camera means, denoted as camera space, comprising:

an articulatable manipulator means of known nominal kinematics in physical space extending from a base to an outward end for movement of the outward end in a predefined physical work space in the physical space relative to the base, the manipulator means including a motor means for articulating the manipulator means in said physical space, and means for producing a signal identifying an approximate position and orientation of the manipulator means with respect only to the base, wherein the kinematic description of the manipulator means with respect to the base being known;

at least one camera means each positionable in physical space without any previously known relation and correlation to the manipulator means except that each camera means must be oriented towards the manipulator means for providing camera vision of at least the outward end of the manipulator means in at least a part of tee physical work space to view, at least intermittently, at least the outward end of the manipulator means in the camera space;

a first visual one means associated with the outward end of the manipulator means;

a second visual cue means associated with an object to be engaged by the manipulator means, the first and second cue means comprising means which are distinct and identifiable in said camera space from the remainder of the system and any surrounding environment, the first and second visual cue means providing descriptions of three-dimensional physical space maneuver objectives as admissible configurations of the visual cue means in the two-dimensional camera spaces of the camera means; and

a control means operatively connected to the manipulator means and the camera means, the control means including computing means for receiving the signal from the manipulator means identifying the approximate position and orientation of the manipulator means with respect to the base through the use of previously known kinematics, and signal processing means which identifies and tracks the visual cue means in the camera spaces and converts such into two-dimensional camera space cue position signals, the manipulator approximate position and orientation signal and the camera space cue position signals being used in the control means to estimate the relationship between the position and orientation of the manipulator means and the location in each camera space of the visual cue means placed on the manipulator means, and, using the current estimations of these relationships, select required movement and orientation of the manipulator means which will bring about admissible configurations of the visual cue means in each camera space ensure successful engagement of the object in physical space, and to control orientation of the manipulator means in physical space according to the selected movement and orientation commands resultant from the estimated relationship.

2. The system claim 1 wherein the manipulator means comprises an arm, the arm being rotatable in a plane about a rotational axis associated with the base of the manipulator means.

3. The system of claim 1 wherein said manipulator means comprises an arm movable in a plurality of directions relative to the base of the manipulator means.

4. The system of claim 1 wherein the outward end of the manipulator means comprises a grasping means.

5. The system of claim 1 wherein the outer end of the manipulator means comprises a receiving means.

6. The system of claim 1 wherein the first and second visual cue means comprise distinct and identifiable markings from the remainder of the system and the surrounding environment.

7. The system of claim 1 wherein the camera means being movable in orientation.

8. The system of claim 1 wherein the location of any of the camera means is not required to be known.

9. The system of claim 1 comprising two camera means.

10. The system of claim 1 wherein the camera means has zoom focusing capabilities.

11. The system of claim 1 further comprising visual cue means associated with the end of the manipulator means and adjacent to the base of the manipulator means.

12. The system of claim 1 wherein the first visual cue means is positioned on the outward end of the manipulator means.

13. The system of claim 1 wherein the second cue means is positioned on the object grasped by the outward end of the manipulator means.

14. The system of claim 1 wherein the object is fixed in the work space.

15. The system of claim 1 wherein the object is not fixed in the workspace.

16. The system of claim 1 wherein the object is interjected into the workspace.

17. The system of claim 1 wherein said control means controls said motor means which moves the manipulator means.

18. A method of camera space manipulation utilizing at least one camera means for engaging an articulatable ,amputator means with an object there is not required any known prior three-dimensional physical space relationship between the manipulator means the object, and the camera means and between physical space and the two-dimensional image at the focal plane of the camera means, denoted as camera space, comprising the steps of:

orienting the camera means to view the manipulator means which has an arm extending from a base to an outward end and being movable in a physical work space with known nominal kinematics relative to the base, the manipulator means including a motor means which articulates the manipulator means in said physical work space, and means for producing a signal identifying the approximate position and orientation of the manipulator means with respect only to the base in said physical work space, each camera means being positioned and oriented in said physical work space without any previously known relation and correlation to the manipulator means except that each camera means must provide, at least intermittently, camera vision of at least the outward end of the manipulator means in at least part of the physical work space to view at least the outward end of the manipulator means in the camera space;

placing a first visual cue means in association with the outward end of the arm means;

placing a second visual cue means in association with the object to be engaged by the manipulator means, the first and second visual cue means comprising means which are distinct and identifiable in said camera space from the remainder of the system and any surrounding environment, the first and second visual cue means providing descriptions of three-dimensional physical space maneuver objectives as admissible configurations of the visual cue means in the two-dimensional camera space of each camera;

receiving signals from the manipulator means and identifying the approximate position and orientation of the manipulator means with respect to the base through the use of the known nominal kinematics;

identifying and tracking the visual cue means in the two dimensional camera space of each camera means and continuously estimating the relationship between the position and orientation of the manipulator means and the location in each camera space the visual cue means placed on the manipulator means, and using the current estimations of these relationships to select the movement and to command the orientation of the manipulator means which will bring about admissible configurations of the visual cue means in each camera space which ensure successful engagement of the object; and

continuously controlling movement and orientation of the manipulator means according to such selected movement and orientation commands to achieve engagement of the manipulator means with the object in said physical work space.