Image processing system for an optical seam tracker

• US 4,849,679 A
• Filed: 12/31/1987
• Issued: 07/18/1989
• Est. Priority Date: 12/31/1987
• Status: Expired due to Fees

• Current Assignee: Westinghouse Electric UK Holdings Limited (Fluor Corporation)
• Original Assignee: Westinghouse Electric Company LLC (Brookfield Asset Management Incorporated)
• Inventors: Taft, Jeffrey D., Ellison, James F., Breakey, Gerald A.
• Primary Examiner(s): Shaw, Clifford C.
• Application Number:
  US07/140,858
• Time in Patent Office: 565 Days
• Field of Search: 219/124.34, 219/130.01, 318/577, 901/42, 901/47
• US Class Current: 318/577
• CPC Class Codes: B23K 9/1274 Using non-contact, optical means, e.g. laser means
• CPC Class Codes: B23Q 35/128 Sensing by using optical means
• CPC Class Codes: G05B 19/425 Teaching successive positions by numerical control, i.e. commands being entered to control the positioning servo of the tool head or end effector
• CPC Class Codes: G05B 2219/36417 Programmed coarse position, fine position by alignment, follow line, path adaptive
• CPC Class Codes: G05B 2219/45104 Lasrobot, welding robot
• CPC Class Codes: G05B 2219/4701 Edge detector, project line, inclined camera detects discontinuity

Abstract

An image processing system uses the digital expression of the video image of a light stripe derived optically from a seam to create a 1-D signal representative of a one-pixel wide line. The target coordinates for the tool operating on the seam is converted directly into 3-D tool position and orientation from the 1-D signal. Matching with models is effected in real time by comparing symbolic attributes of the graphs built up with the 1-D signal and with standard seams. From such matching is selected the operative corner, or target for the tool.

Claims

1. A method of operating a robot system including:

a robot controller for iteratively controlling an effector end with an error compensating control signal in relation to a taught path;

a seam sensor vision system havign an optical sensor disposed ahead of the effector end for iteratively deriving successive 2-D light stripe images representing discrete successive transversed views of the seam as sensed at successive locations upon the seam ahead of the effector end, an image processor for treating each of said images to derive the relation to each image a first signal representative of the corresponding sensed location, and a control processor responsive to a second signal indicative of a present effector end position and to said first signal for generating said control signal as an indication of an error between effector end position and sensed location;

comprising the following operative steps by the image processor;

with the 2-D strip image having a plurality of lines in a raster and each line having a plurality of addressable pixels;

deriving from said 2-D stripe image a digital representation thereof by scanning said lines successively to provide a series of digital signals each representative of a pixel, said digital signals being characteristic of the raster;

processing said digital signals by converting to a predetermined white pixel characteristic digital signal a derived digital signal bordering a selected one of the edges of said 2-D stripe image, and by converting to a predetermined black pixel characteristic digital signal derived digital signals from non bordering said selected edge, whereby said derived digital signals together represent a one-pixel wide line characteristic of said light stripe image;

processing said pixel line to identify thereupon a point characteristic of the location sensed on the seam by said optical sensor; and

deriving from said characteristic point 3-D coordinates thereof as an indication of the sensed seam location.

2. The method of claim 1, with the 2-D stripe image being a camera video image, the image being obtained by interlacing video frames;

whereby said derived digital signals represent a two-pixel wide line;

an additional step being provided consisting in cancelling every occurring white pixel characteristic digital signal when followed by another white pixel characteristic signal for the same line scanning position, whereby a one-pixel wide line is obtained.

3. The method of claim 2, with said additional step comprisng storing every derived digital signal in relation to an address representing a scanning line position, and substituting a second derived digital signal for a first stored such derived digital signal at the same address, whereby said 2-D image is converted into a 1-D representation thereof as a one-pixel line.

4. The method of claim 3, with said pixel line processing step including matching said 1-D pixel line representation with a reference 1-D pixel line model.

5. The method of claim 4, with said 1-D pixel line representation being converted into a graph having numerical and symbolic attributes, matching being obtained in relation to similar symbolic attributes of the reference model and used to identify said characteristic point, and said numerical attributes being selected according to the symbolic attributes of the matching reference model and applied to said identified characteristic point;

the selected numerical attributes being used in said 3-D coordinates deriving step.

6. The method of claim 5, with said 3-D coordinates deriving step performing conversion from 1-D coordinates to 3-D coordinates directly.

7. The method of claim 6, with the steps of transforming the seam coordinates derived in relation to the characteristic point into coordinates matching the effector end coordinates for control thereof by the control processor.

8. The method of claim 7, with the coordinates derived in relation to a sensed location being classified in relation to elapsed distance since the start of the seam sensing process.

9. The method of claim 8, with the control processor performing the following operative steps:

calculating and storing iteratively, in response to said control signal and to said second signal, the elapsed distance of the effector end position;

calculating and storing iteratively, in response to said control signal and to said second signal, the elapsed distance from the first down to the present sensed location by correlation with the distance between effector end and the sensor;

ordering said seam coordinates to, and correlating said successive sensed locations with, said sensed location elapsed distances to provide a look-up table thereof;

deriving iteratively with said second signal for a present effector end position and with said control signal for a present positioning of the effector end successive positions representing the taught path as recovered therefrom;

extrapolating by a predetermined amount from past positions on said recovered path to an anticipated position thereon;

extrapolating by said predetermined amount from past elapsed distances to an anticipated elapsed distance for said effector end;

determining with said look-up table the anticipated sensed location corresponding to said anticipated elapsed distance;

comparing said corresponding sensed location with said anticipated recovered taught path position for deriving a present control signal in relation to said effector end position; and

controlling by feedforward the robot controller in response to said present control signal.

10. The method of claim 9, with said corresponding sensed location determining step being performed by interpolation between two stored sensed locations embracing in said look-up table said anticipated elapsed distance.

11. The method of claim 9, with said look-up table containing said sensed locations stored as a modeling function of elapsed distances, and with said corresponding sensed location determining step being performed with said anticipated elapsed distance used as a reference to derive from said modeling function the corresponding sensed location.

12. The method of claim 9 with said second signal providing for each present effector end control the effector end position in world coordinates;

the control processor providing the following additional steps;

determining in world coordinates the chord of said predetermined distance extending across the seam path between a present effector end position and a present sensed seam location;

deriving from said effector end present position world coordinates an indication of the effector end direction of travel along the seam path; and

determining with said chord and direction of travel an angle therebetween;

means being provided for moving the sensor by rotation about said effector end by the amount of said determined angle in order to position the sensor upon the seam prior to sensing another seam location thereupon.

13. The method of claim 12, with said control signal defining axes in world coordinates for the effector end and orientation angles in world coordinates therefor, said chord and direction angle being applied as an additional angle to a corresponding one of said orientation angles.

14. The method of claim 9, with said second and control signals being provided cyclically upon each of a series of tick cycles, whereby the effector end is being moved, under a present control signal and upon each tick, from a present position to a next position along successive sensed locations of the seam.

15. The method of claim 14, with a present control signal being determined by the control processor in relation to an anticipated effector and location situated three-ticks ahead of a present effector end position.

16. An optical seam tracker system generating a video signal representative of a light stripe image defined as a raster of scanning lines having white pixels when located on the light stripe and black pixels as a background thereof, the seam tracker using image processing comprising the following:

means for converting said video signal into digital signal representing pixels on the scanning lines;

sliding window means for establishing a running mask along said scanning lines allowing said digital signals to pass through;

a plurality of reference signals each representing one of successive and possible mask situations corresponding to pixels bordering a selected one of the edges of said light stripe;

means responsive to the mask allowed digital signals and to said reference signals for passsing a white pixel digital signal when the digital signal matches one of said reference signals and for passing a black pixel digital signal otherwise, whereby the stripe image is converted into a one-pixel wide representative line.

17. The system of claim 16, with said video signal being obtained by interlacing, whereby one-pixel wide said line is derived to each half frame of the interlacing;

with the provision of;

means for storing said passed white and black pixel signals as a discrete function of the line position, each stored white signal being cancelled upon storing another passed white signal for the same line position, whereby a single one-pixel wide line is derived per video image represented as a 1-D characteristic signal.

18. The system of claim 17, with said mask representing a 3×

3 pixel neighborhood.

19. The system of claim 18, with said plurality of reference signals corresponding to ten of said situations.

20. The system of claim 19, with a digital pipeline embodying all of said means is responsive to said video signal and outputting said 1-D characteristic signal.

21. The system of claim 20, with said digital pipeline being supervised, monitored and controlled by a microcomputer.

22. The system of claim 21, with the microcomputer identifying with said 1-D signal a target point on said pixel line characteristic of a sensed seam location on said light stripe.

23. The system of claim 21, with said microcomputer deriving from said 1-D signal and storing in RAM numerical data relative to said pixel line, one of said data being the coordinates of said target point.

24. The system of claim 23, with means controlled by said microcomputer for converting said 1-D signal into a 3-D representation of said target point.

25. The system of claim 24, with the seam tracker including a projector of light directed along an axis toward the seam, an optical lens system about an optical axis z, the lens being in normal plane (x,y), the projector axis being at an angle φ

to the z-axis and in the plane (y,z), with the projector being a distance b from the z-axis, the video image being received in a plane (x,y) a distance f from the plane of the lens, the conversion from 1-D to 3-D is performed by the computer according to the following equations;

##EQU29## where j is the ordinate of the discrete function f(i) characterizing said 1-D signal, kj is the vertical interpixel distance, Jm the maximum vertical resolution of the sensor, ki the horizontal interpixel distance and Im the maximum horizontal resolution of the sensor.

26. The system of claim 25, with an image slope signal representative of a characteristic slope of said pixel line being derived from said 1-D signal by reference to a standard seam geometry, said microcomputer converting said image slope signal into a surface slope signal characterizing the inclination m_zx of the 3-D seam surface of the y-axis.

27. The system of claim 26, with m_zx being computed according to the following:

##EQU30## where j/i is the image slope, the projector being aligned so as to place lines parallel to the i-axis when the surface is normal to the lens axis.

28. The system of claim 24, with said microcomputer identifying from said 1-D signal by slope rate changes corners characterizing the geometry of said one pixel line, numerical and symbolic attributes being stored in a corner RAM as characteristics for each of said identified corners.

29. The system of claim 28, with the identification of said corners and the ordering of said corner RAM'"'"'s being performed continuously and successively along said one pixel line.

30. The system of claim 29, with the numerical attributes stored in a corner RAM including the coordinates (i,j) of the corresponding corner.

31. The sytem of claim 30, with the numerical attributes stored in a corner RAM including the distance from one corner to a line joining two adjacent corners, a predetermined minimum such distance being established below which the corresponding such one corner is being cancelled as a duplication of an adjacent corner.

32. The system of claim 30, with the symbolic attributes stored in a corner RAM including the geometry of the link to an adjacent corner.

33. The system of claim 32, with at least one standard seam being selected as a model and the light stripe image thereof being characterized by a corresponding 1-D signal, a model graph representative of the standard seam being formed with such corresponding 1-D signal by storing symbolic attributes thereof;

an actual graph representative of the sensed seam being formed in relation to the associated derived 1-D signal by storing symbolic attributes thereof;

the symbolic attributes being compared for matching between said model and actual graphs;

said target point being determined with an identified corner selected by reference to said model graph when said graph matching has occurred.

34. The system of claim 33, with the gap distance to the seam applicable for the effector end being determined by reference to a matching model graph.

35. The system of claim 34, with a refined determination of said target point being effected with said 1-D signal by reference to a matching model graph before conversion into a 3-D target point representation.