Method and apparatus for remotely sensing orientation and position of objects

• US 5,943,476 A
• Filed: 06/13/1996
• Issued: 08/24/1999
• Est. Priority Date: 06/13/1996
• Status: Expired due to Fees

• Current Assignee: August Designs, Inc.
• Original Assignee: August Designs, Inc.
• Inventors: Dougherty, Edmond J., Simmons, George R. III
• Primary Examiner(s): Grant, William
• Assistant Examiner(s): THOMAS, CAROLYN D
• Application Number:
  US08/662,594
• Time in Patent Office: 1,167 Days
• Field of Search: 395/94, 395/97, 395/86, 364/513, 364/474.28, 318/568.11, 359/877, 345/158
• US Class Current: 700/259
• CPC Class Codes: B25J 9/1694 characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
• CPC Class Codes: G05B 2219/37288 Tracking lasers follow object, reflection gives 3-D position
• CPC Class Codes: G05B 2219/37572 Camera, tv, vision
• CPC Class Codes: G05B 2219/39391 Visual servoing, track end effector with camera image feedback
• CPC Class Codes: G05B 2219/39393 Camera detects projected image, compare with reference image, position end effector

Abstract

A system for determining position and orientation of a stationary or moving object projects beams of electromagnetic waves, such as light, onto the object. The beams make lines on the object. A camera senses reflections of the lines. The reflections are converted into pixel data which are used to determine the position and orientation of the object relative to a predetermined coordinate system. The system can also maneuver a robot end effector to the object'"'"'s position and orientation by interactively viewing representations of the object'"'"'s position and orientation, and the end effector on a computer screen.

Claims

1. An apparatus for determining position and orientation of an object from a position remote therefrom, comprising:

a. means for projecting beams of electromagnetic waves as lines onto said object;

b. means for directly sensing reflections of said lines from said remote object;

c. means for converting said sensed reflections into binary code referenced to a coordinate system; and

d. means for interpreting said binary code to determine position and orientation of said remote object with respect to said coordinate system.

2. An apparatus for determining position and orientation of an object located remotely from said apparatus, comprising:

a. a camera;

b. means for generating beams of visible electromagnetic radiation;

c. means for projecting said beams onto said remote object to form lines thereon which generally traverse one another;

d. means for sensing said lines reflected by said remote object;

e. means for converting said sensed reflected lines into pixels referenced to a coordinate system;

f. means for converting said pixels into binary code; and

g. means for interpreting said binary code to determine position and orientation of said remote object with respect to said coordinate system.

3. An apparatus for determining position and orientation of an object located remotely from said apparatus, comprising:

a. a frame;

b. a plurality of laser light generators mounted on said frame;

c. a plurality of oscillating mirrors mounted on said frame and paired with respective ones of said laser light generators, for reflectingly positioning laser light output from said generators onto said remote object as lines of laser light traversing one another;

d. camera means for receiving said laser light lines reflected by said remote object and creating an electrical signal indicative of reflected laser light lines;

e. a video-processing board for converting said electrical signal into pixels referenced to a coordinate system;

f. means for converting said pixels into binary code; and

g. means for mathematically processing said binary code to provide position and orientation of said remote object with respect to said coordinate system.

4. The apparatus of claim 3 further comprises means for positioning and orienting a robotic end effector to said position and orientation of said remote object based on said binary code.

5. The apparatus of claim 3 further comprises means for receiving reflections of said lines from said remote object and transmitting said reflections to a monitor which displays said reflections of said lines.

6. A method for remotely determining position and orientation of a remote object with respect to a reference coordinate system, comprising:

a. projecting electromagnetic beams onto said object to produce a pattern of lines thereon;

b. directly sensing the pattern of lines from a position remote from said object;

c. interpreting the sensed pattern of lines as mathematical data; and

d. processing said mathematical data to provide position and orientation of said object measured relative to said reference coordinate system.

7. The method of claim 6 further comprises positioning, orienting and moving a robotic end effector to the mathematically determined position and orientation of said object relative to said reference coordinate system.

8. The method of claim 6 wherein the projected beams in step a, produce either a visible or invisible pattern.

9. A method for remotely sensing and computing the position and orientation of a remote object having a surface, the method comprising:

a. projecting electromagnetic beams onto the surface of said remote object to produce a pattern of laser light lines thereon;

b. viewing reflections of said lines detected by a television camera;

c. transmitting said reflections to a video processing board in a computing device which converts said reflections to pixels and stores said pixels in memory;

d. accessing and converting said pixels into binary code; and

e. interpreting said binary code to determine position and orientation of said object relative to a predetermined coordinate system.

10. The method of claim 9 further comprises positioning, orienting and moving a robotic end effector to said determined position and orientation of said object relative to said predetermined reference coordinate system.

11. A method for remotely sensing and computing the position and orientation of a moving object relative to a predetermined coordinate system, comprising:

a. projecting electromagnetic beams onto respective oscillating mirrors which oscillate between two predetermined angles;

b. deflecting respective electromagnetic beams by oscillating said mirrors between two predetermined angles to create laser light lines on said moving object;

c. positioning said beams and said respective oscillating mirrors to create laser light lines which traverse one another on said object;

d. detecting a pattern of reflections of said laser light lines with a television camera located in a position removed from said object;

e. transmitting said reflections from said television camera to a video processing board of a computer and to a monitor;

f. converting said reflections into pixels and storing said pixels in memory of computer;

g. accessing and converting said pixels into binary code; and

h. calculating the position and orientation of said object relative to a predetermined coordinate system by using a laser scan angle, field of view angle, length of laser scan line, dimension of the monitor, and said binary code.

12. The method of claim 11 further comprises positioning, orienting and moving a robotic end effector to said calculated position and orientation of said object.

13. A method for remotely determining position and orientation of a remote object using a coordinate system as a reference and for maneuvering a robotic end effector to said position and orientation, comprising:

a. maneuvering said robotic end effector in surge and sway directions until pitch and roll laser light lines viewed on a computer screen are fully on said remote object;

b. maneuvering said robotic end effector in pitch direction until said pitch line is vertically positioned on said remote object;

c. maneuvering said robotic end effector in roll direction until said roll line is horizontally positioned on said remote object;

d. maneuvering said robotic end effector in surge direction until said roll line disappears or shortens and then maneuvering robotic end effector in an opposite direction in fine steps until the roll line reappears and returns to its correct length;

e. dithering the robotic end effector such that the roll line appears or lengthens and disappears or shortens;

f. maneuvering the robotic end effector in sway direction until the pitch line disappears or shortens and then maneuvering robotic end effector in an opposite direction in fine steps until the pitch line reappears or returns to its correct length;

g. dithering the robotic end effector such that the pitch line appears or lengthens to its correct length and disappears or shortens;

h. rotating the robotic end effector in yaw direction until said pitch line or said roll line is the correct length;

i. maneuvering the robotic end effector in the heave direction until said pitch line and said roll lines move out of field of view of camera; and

j. repeating steps a through i until the robotic end effector picks up said remote object.

14. A method for remotely determining position and orientation of a remote object using a rectangular X, Y and Z coordinate system in a camera'"'"'s field of view as a reference and for maneuvering a robotic end effector to said position and orientation, comprising:

a. mounting a television camera to electromagnetic beam generators and to oscillating mirrors so that a focus of lens system of the television camera and points of deflection of said electromagnetic beams from said oscillating mirrors are in the same X-Y plane and the focus of the lens system of the television camera is a point on the Z axis;

b. projecting a pitch laser light line of predetermined length in parallel with the Y axis and projecting a roll laser light line of predetermined length in parallel with the X axis onto an object at a predetermined distance therefrom, by projecting electromagnetic beams at respective oscillating mirrors through a predetermined angle, the pitch and roll lines traversing one another;

c. maneuvering a robotic end effector in +Y or -Y direction until the pitch line is fully on target;

d. maneuvering the robotic end effector in +X and -X direction until the roll line is fully on object;

e. computing a pitch angle;

f. maneuvering the robotic end effector clockwise about the X axis whenever the bottom point of pitch line is further from left edge of field of view than the top point;

g. maneuvering the robotic end effector counterclockwise about the X axis until the pitch line is vertical whenever top point of the pitch line is further from the left edge of field of view than the bottom point;

h. computing roll angle;

i. maneuvering the robotic end effector clockwise about the Y axis until the roll line is horizontal whenever right-most point of the roll line is further from the bottom of field of view than the left-most point;

j. maneuvering the robotic end effector in counterclockwise direction until the roll line is horizontal whenever left-most point of the roll line is further from bottom of field of view at the right-most point;

k. aligning the robotic end effector with the X axis by completing the following steps;

i. maneuvering grossly the robotic end effector in -Y direction until the roll line disappears or shortens as viewed by the camera,

ii. maneuvering the robotic end effector in fine steps in the +Y direction until the roll line reappears or returns to its correct length in the field of view of the camera, and

iii. dithering the robotic end effector in +Y and -Y directions such that the roll line appears or lengthens when moving in +Y direction and disappears or shortens when moving in -Y direction;

l. aligning the robotic end effector with the Y axis by completing the following steps;

i. maneuvering the robotic end effector grossly in -X direction until the pitch line disappears or shortens from the camera field of view,

ii. maneuvering the camera in fine steps in the positive X direction until the pitch line reappears or returns to its correct length in the camera field of view, and

iii. dithering the robotic end effector in +X and -X directions such as the pitch line appears or lengthens to its correct length when maneuvering in +X direction and disappears or shortens when maneuvering in -X direction;

m. performing yaw alignment by rotating the robotic end effector clockwise or counterclockwise about the Z axis;

n. computing distance between the robotic end effector and the camera; and

o. performing Z alignment by slowly maneuvering robotic end effector in Z direction until the pitch and roll lines move out of the camera field of view.

15. A method for remotely detecting the distance between the focus of a camera and a remote object, comprising:

a. mounting a camera having a lens system, electromagnetic beam generator and oscillating mirror so that the focus of the camera lens system is in the same plane as the point of deflection of an electromagnetic beam;

b. creating a laser light line of predetermined length projected onto an object at a distance therefrom, by projecting the electromagnetic beam at a mirror oscillating through a predetermined angle;

c. positioning the camera so that the remote object is within the field of view of the camera lens system;

d. digitizing a directly received reflection of the electromagnetic beam;

e. storing the digitized data in memory; and

f. computing the distance between the focus of said camera and the remote object by using;

i. distance between the center line of the camera lens system and edge of the field view,

ii. distance between the point where the laser light line intersects the plane of the object and the closest edge of the field of view, and

iii. predetermined distance between the point of deflection of the electromagnetic beam and the point of intersection of the deflected electromagnetic beam and field of view.

16. A method for remotely detecting the distance between the focus of a camera and a remote object, comprising:

a. mounting a camera having a lens system, electromagnetic beam generator and oscillating mirror so that the focus of the camera lens system is in the same plane as the point of deflection of an electromagnetic beam;

b. creating a laser light line of predetermined length projected onto an object at a distance therefrom, by projecting the electromagnetic beam at a mirror oscillating through a predetermined angle;

c. positioning the camera so that the remote object is within the field of view of the camera lens system;

d. digitizing a directly received reflection of the electromagnetic beam;

e. storing the digitized data in memory; and

f. computing the pitch angle of the object by using;

i. angle between vertical deflected electromagnetic beam and center line of camera lens system,

ii. distance between focus of camera and top point of object, and

iii. distance between focus of camera and bottom point of object.

17. A method for remotely detecting the position and orientation of a moving object relative to a selected reference, comprising:

a. projecting an electromagnetic beam onto the surface of an object whose position is to be determined to produce a pattern of lines thereon;

b. directly sensing the pattern of lines from a position removed from said object;

c. interpreting the sensed pattern of lines as mathematical data; and

d. processing the mathematical data to produce the position and orientation of said object relative to the selected reference.

18. Apparatus for determining the position and orientation of a remote object, comprising:

a. a camera;

b. means for generating an electromagnetic beam to which air is substantially transparent;

c. means for projecting said beam onto said remote object in the form of lines which are generally transverse one to another;

d. means for receiving electromagnetic beam reflections of said lines directly from said remote object; and

e. means for converting said lines into data elements referenced to an x,y coordinate system.