System and method for assembly inspection

• US 8,131,055 B2
• Filed: 01/31/2008
• Issued: 03/06/2012
• Est. Priority Date: 01/31/2008
• Status: Expired due to Fees

• Current Assignee: Caterpillar Incorporated
• Original Assignee: Caterpillar Incorporated
• Inventors: Clarke, Burton Roland, Norlin, John Edward
• Primary Examiner(s): Toatley, Gregory J
• Assistant Examiner(s): Underwood, Jarreas C
• Application Number:
  US12/010,956
• Publication Number:
  US 20090198464A1
• Time in Patent Office: 1,496 Days
• Field of Search: 382141-152
• US Class Current: 382/141
• CPC Class Codes: G05B 19/41875 characterised by quality surveillance of production
• CPC Class Codes: G05B 2219/37205 Compare measured, vision data with computer model, cad data
• CPC Class Codes: Y02P 90/02 Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]

Abstract

A method for assembly inspection is disclosed. The system may include obtaining a digital image of an assembled product, extracting images of one or more objects from the digital image of the assembled product, and recognizing each of the one or more objects as a component based on its extracted image and a library of standard components. The method may further include identifying one or more features of each recognized component, comparing each of the one or more identified features with a corresponding standard feature of the corresponding standard component, and determining an assembly fault if at least one of the one or more identified features does not match the corresponding standard feature.

Claims

1. A method for assembly inspection, the method comprising:

obtaining a digital image of an assembled product;

extracting, by at least one processor, images of one or more objects from the digital image of the assembled product, the extracting including;

extracting, from the digital image, images of one or more rigid objects;

growing images of one or more flexible objects based on the extracted images of the one or more rigid objects; and

extracting images of the one or more flexible objects;

recognizing each of the one or more rigid or flexible objects as a component based on its extracted image and a library of standard components;

identifying one or more features of each recognized component;

comparing each of the one or more identified features with a corresponding standard feature of the corresponding standard component; and

determining an assembly fault if at least one of the one or more identified features does not match the corresponding standard feature.

2. The method of claim 1, wherein extracting images of one or more rigid objects further includes:

detecting a contour of each component;

segmenting the digital image into a plurality of intensity areas; and

combining each intensity area with the corresponding contour to obtain an extracted image.

3. The method of claim 1, wherein growing images of one or more flexible objects is implemented with at least one of clustering, region growing, or filtering algorithms, or any combination thereof.

4. The method of claim 1, wherein growing images of one or more flexible objects includes growing an image of a flexible object between two coupling rigid objects.

5. The method of claim 1, wherein growing images of one or more flexible objects includes growing an image of a flexible object from a terminating rigid object.

6. The method of claim 1, wherein the library of standard components is generated by:

converting a design chart of the assembled product to a library of standard components; and

identifying one or more standard features for each standard component.

7. The method of claim 1, wherein obtaining the digital image of the assembled product includes:

applying a first optical signal for illuminating the assembled product;

receiving a second optical signal containing the structure information of the assembled product; and

converting the second optical signal to a digital image, wherein each pixel value of the digital image is proportional to a signal intensity received at each pixel.

8. The method of claim 7, wherein obtaining the digital image of the assembled product further includes:

compressing the digital image;

transferring the compressed digital image; and

uncompressing the compressed digital image.

9. The method of claim 1, wherein determining an assembly fault further includes:

diagnosing the assembly fault;

determining a type and a location of the assembly fault; and

providing an inspection report having the type and the location of the assembly fault.

10. The method of claim 1, wherein recognizing each of one or more objects includes:

performing a border-fitting comparison between the extracted image with each standard component in the library of standard components;

performing parallelism judgment based a result of the border-fitting comparison;

calculating a correlation between the extracted image and each standard component; and

labeling the extracted image as a component of a type of the standard component corresponding to a highest correlation.

11. The method of claim 10, wherein the correlation is determined based on at least one of a color and a shape of the component defined by the extracted image.

12. The method of claim 1, wherein the one or more features includes at least one of an orientation of the component and a dimension of the component.

13. A non-transitory computer readable storage medium having stored thereon instructions that, when executed by a processor associated with an assembly inspection system, cause the assembly inspection system to perform a method for assembly inspection, the method comprising:

obtaining a digital image of an assembled product;

extracting images of one or more objects from the digital image of the assembled product, the extracting including;

extracting, from the digital image, images of one or more rigid objects;

growing images of one or more flexible objects based on the extracted images of the one or more rigid objects; and

extracting images of the one or more flexible objects based on the grown images of the one or more flexible objects;

recognizing each of the one or more rigid or flexible objects as a component based on its extracted image and a library of standard components;

identifying one or more features of each recognized component;

comparing each of one or more identified features with a corresponding standard feature of the corresponding standard component; and

determining an assembly fault if at least one of the one or more identified features does not match the corresponding standard feature.

14. The computer readable medium of claim 13, wherein the library of standard components is generated by:

converting a design chart of the assembled product to a library of standard components; and

identifying one or more standard features for each standard component.

15. The computer readable medium of claim 13, wherein determining an assembly fault includes:

diagnosing the assembly fault;

determining a type and a location of the assembly fault; and

providing an inspection report having the type and the location of the fault.

16. A system for assembly inspection, comprising:

an imaging device configured to obtain a digital image of an assembled product;

a storage device configured to store an assembly inspection tool, a library of standard components, and one or more standard features of each standard component; and

a processor configured to execute the assembly inspection tool to;

extract, from the digital image, images of one or more rigid objects from the digital image of the assembled product

grow images of one or more flexible objects based on the extracted images of the one or more rigid objects;

recognize each rigid or flexible object as a component based on its image and a library of standard components;

identify one or more features of each recognized component;

compare each identified feature with the corresponding standard feature of the corresponding standard component;

determine an assembly fault if at least one of the one or more identified features does not match the corresponding standard feature; and

determine at least one of a type or a location of the assembly fault.

17. The system of claim 16, wherein the imaging device includes:

an optical emitter configured to applying a first optical signal for illuminating the assembled product;

an optical detector configured to receive a second optical signal containing the structure information of the assembled product; and

a converter configured to convert the second optical signal to a digital image, wherein each pixel value of the digital image is proportional to a signal intensity received at each pixel.

18. The system of claim 16, further including a display device configured to display an inspection report having the type and the location of the assembly fault.