Method and apparatus for automatic application and monitoring of a structure to be applied onto a substrate

• US 8,400,503 B2
• Filed: 07/16/2004
• Issued: 03/19/2013
• Est. Priority Date: 12/23/2003
• Status: Active Grant

• Current Assignee: QUISS Qualitaets InspektionsSysteme und Service GmbH (Atlas Copco AB)
• Original Assignee: Quiss GmbH
• Inventors: Linnenkohl, Jan Anders, Berger, Mirko, Raab, Roman, Tomtschko, Andreas
• Primary Examiner(s): Anyikire, Chikaodili E
• Application Number:
  US10/584,120
• Publication Number:
  US 20080024602A1
• Time in Patent Office: 3,168 Days
• Field of Search: 348125-134, 382141-152
• US Class Current: 348/125
• CPC Class Codes: B05B 12/126 responsive to target velocity, e.g. to relative velocity between spray apparatus and target B05B9/06 takes precedence
• CPC Class Codes: B05C 5/0216 by relative movement of article and outlet according to a predetermined path
• CPC Class Codes: G01N 21/88 Investigating the presence of flaws or contamination
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: Y10T 156/10 Methods of surface bonding and/or assembly therefor

Abstract

A method and apparatus are provided for automatic application and monitoring of a structure to be applied onto substrate. A plurality of cameras positioned around an application facility are utilized to monitor the automatic application of a structure on a substrate by means of a stereometry procedure. Three-dimensional recognition of a reference contour position results in the overlapping area to be used for gross adjustment of the application facility prior to applying the structure.

Claims

1. A method for automatic application and monitoring of a structure to be applied onto a substrate, the method comprising:

determining a reference contour by utilizing a first camera in a leading direction to record a plurality of images;

regulating the progression of the structure to be applied according to the reference contour;

guiding an application facility based on the images recorded by the first camera;

applying the structure onto the substrate by the application facility according to the reference contour determined by the first camera; and

monitoring the structure applied onto the substrate by the application facility utilizing a second camera in trailing direction;

wherein the method is performed such that the cameras have an overlapping area on a circular line, a segment of the circular line is assigned to images of each camera, at least one of the reference contour and the applied structure progresses relative to the cameras from one camera to the next camera, and an automatic switch is made when the applied structure or the reference contour progresses relative to the cameras from the segment of the circular line of one camera via the overlapping area to the segment of the circular line of another camera.

2. The method according to claim 1, wherein the structure comprises at least one of an adhesive line and an adhesive trail.

3. The method according to claim 1, wherein the reference contour comprises a reference edge of a component that is to be joined to the substrate.

4. The method according to claim 1, wherein the two cameras record an image of at least one of the substrate, a section of a component to be joined to the substrate, and one or more components, the image including at least one of an image strip, a full image, and a large image.

5. The method according to claim 2, wherein the overlapping area is in the leading direction.

6. The method according to claim 5, wherein the overlapping area recognizes the reference contour as a three-dimensional reference contour, and a position of the reference contour is used to adjust the application facility prior to applying the structure.

7. The method according to claim 1, wherein a projection is made onto an area of the reference contour for three-dimensional analysis.

8. The method according to claim 7, wherein the projection comprises at least one laser line applied onto the substrate.

9. The method according to claim 1, wherein at least one camera records each of one or more images as an image strip for at least one of an online regulation of the application structure and an online monitoring of the applied structure.

10. The method according to claim 1, wherein the second camera records an image strip, and the second camera uses the image strip for online monitoring of the applied structure.

11. The method according to claim 1, wherein the method is performed such that the first and second cameras each record images as image strips, and the image strips of the two cameras are recorded to form a single sequence of images.

12. The method according to claim 9, wherein an image recording rate is increased in proportion with data reduction achieved by recording each image as an image strip.

13. The method according to claim 9, wherein a particular image comprises a plurality of picture lines, each camera utilizing at least one of a third, a fourth, and a fifth of the picture lines as a strip of the image.

14. The method according to claim 13, wherein an image recording rate of the cameras is multiplied by at least one of a three-fold, a four-fold and a five-fold factor based on the plurality of picture lines utilized.

15. The method according to claim 1, wherein a parameterization and a recording of an application track proceed in a single image recording run, whereby the images of all cameras are stored in a sequence of images.

16. The method according to claim 15, wherein the parameterization is determined by the stored sequence of images of at least a robot travel path, a robot travel time, robot coordinates, a position of the applied structure, a contrast of the applied structure, a gray scale value of the applied structure, a color value of the applied structure, a width of the applied structure, and a quality of the applied structure.

17. The method according to claim 16, wherein the parameterization is stored as a vector chain.

18. The method according to claim 17, wherein the vector chain comprises a plurality of images recorded at a high image rate.

19. The method according to claim 18, wherein the vector chain comprises a plurality of images recorded as short partial sections, the sections being less than 4 mm in length.

20. The method according to claim 19, wherein the vector chain comprises sections between 0.5 mm and 4 mm in length.

21. The method according to claim 19, wherein the vector chain comprises sections between 1 mm and 3 mm in length.

22. The method according to claim 11, wherein the method is performed with a third camera, each camera being configured to regulate in a leading direction according to the reference contour and being configured for monitoring the applied structure in a trailing direction.

23. The method according to claim 22, wherein the three cameras each comprise an overlapping area to an adjacent camera on the circular line.

24. The method according to claim 23, wherein angle values of the circular line range from 0 to 360°

and form a global coordinate system, and wherein a segment of the circular line is assigned to images of the third camera.

43. The method of claim 1 wherein the structure comprises a sealing agent.

44. The method of claim 1 wherein the reference contour is determined by performing a three-dimensional positional correction for the application facility by means of a stereometry procedure utilizing the first and second cameras.

25. A method for automatic application and monitoring of a structure to be applied onto a substrate, the method comprising:

determining a reference contour of a component to be connected to the substrate by utilizing a first camera in a leading direction to record a plurality of images;

regulating the progression of the structure to be applied according to the reference contour;

guiding an application facility based on the images recorded by the first camera;

applying the structure onto the substrate by the application facility according to the reference contour determined by the first camera; and

monitoring the structure applied onto the substrate by the application facility utilizing a second camera in trailing direction;

wherein the method is performed with a third camera, each camera being configured to regulate in a leading direction according to the reference contour and to monitor the applied structure in a trailing direction, and the three cameras each comprise an overlapping area to an adjacent camera on a circular line, and wherein the method is performed such that at least one of the reference contour and the applied structure progresses relative to the cameras from one camera to the next camera, and an automatic switch is made when the applied structure or the reference contour progresses relative to the cameras from the segment of the circular line of one camera via the overlapping area to the segment of the circular line of another camera.

26. An apparatus for automatic application and monitoring of a structure to be applied onto a substrate, the apparatus comprising:

at least one illumination module;

an application facility for applying the structure onto the substrate; and

a sensor unit having two cameras provided around the application facility, the two cameras including a first camera configured in a leading direction for regulation of the application facility by means of a reference contour and a second camera configured in a trailing direction for simultaneous online monitoring of the structure applied onto the substrate;

wherein the cameras are configured such that the cameras have an overlapping area on a circular line, a segment of the circular line is assigned to images of each camera, and an automatic switch is made when the applied structure or the reference contour progresses relative to the cameras from the segment of the circular line of one camera via the overlapping area to the segment of the circular line of another camera.

27. The apparatus according to claim 26, wherein each camera has an optical axis, and the optical axes of the cameras are configured to intersect along a direction of view.

28. The apparatus according to claim 27, wherein the optical axes are configured to be at least one of the following:

directed along a longitudinal axis of the application facility, parallel to each other, and perpendicular to the substrate.

29. The apparatus according to claim 26, wherein the sensor unit further includes a third camera.

30. The apparatus according to claim 29, wherein the first, second and third cameras are arranged at equal distances from each other in a circle.

31. The apparatus according to claim 26, wherein the cameras are configured to interact with each other such that the images of the cameras are stored in a sequence of images.

32. The apparatus according to claim 31, wherein each camera is configured to record each associated image as an image strip to form a part of the sequence of images.

33. The apparatus according to claim 26, wherein a recording rate is increased according to data reduction achieved by recording each image as a single strip.

34. The apparatus according to claim 26 further comprising a projection facility provided on the application facility, the projection facility being configured to project a plurality of image strips onto the substrate for a three-dimensional analysis.

35. The apparatus according to claim 34, wherein the projection facility is configured to emit at least one laser line for a three-dimensional profile analysis.

36. The apparatus according to claim 26 further comprising at least two projection facilities arranged around the application facility, the at least two projection facilities being configured to project a plurality of image strips onto the substrate for a three-dimensional analysis.

37. The apparatus according to claim 26, wherein the cameras are directed at a circle whose center coincides with a center of the application facility, and wherein the cameras are configured to perform at least an essentially circular edge scan in the form of a circular caliper.

38. The apparatus according to claim 37, wherein the sensor unit further includes a third camera and each individual camera is configured to image at least one of an overlapping area between 30°

to 90° and

an overlapping area of approximately 60°

relative to the next camera.

39. The apparatus according to claim 26, wherein the at least one illumination module comprises light emitting diodes (LEDs), and the LEDs comprise at least one of an infrared LED, an ultra-violet (UV) LED, and a red-green-blue (RGB)LED, and wherein the LEDs are configured to flash utilizing pulses of current ranging between 1.0 to 0.01 ms.

40. The apparatus according to claim 29, wherein the apparatus further comprises a calibrating device having individual form elements spaced apart at an angle distance of 10°

to calibrate the individual cameras for assignment of an angle assignment.

41. The apparatus according to claim 40, wherein the calibrating device further comprises at least three marker sites utilized to calibrate the three cameras, the marker sites being arranged in a circular arc of the calibrating device and located at approximately 0°

, 120°

, and 240°

.

42. The apparatus according to claim 41, wherein the marker sites are formed by at least two form elements, and the marker sites are located on the circular arc extending to an angle range of approximately 10°

.

45. An apparatus for automatic application and monitoring of a structure to be applied onto a substrate, the apparatus comprising:

at least one illumination module;

an application facility for applying the structure onto the substrate;

a sensor unit having multiple cameras provided around the application facility, the multiple cameras including a first camera configured in a leading direction for regulation of the application facility by means of a reference contour, a second camera configured in a trailing direction for simultaneous online monitoring of the structure applied onto the substrate, and a third camera; and

a calibrating device having individual form elements spaced apart at an angle distance of 10°

to calibrate the individual cameras for assignment of an angle assignment, wherein the calibrating device further comprises at least three marker sites for calibrating the three cameras, the marker sites being arranged in a circular arc and located at approximately 0°

, 120° and

240°

.