Inspection apparatus, method, and computer program product for machine vision inspection

• US 9,747,680 B2
• Filed: 11/27/2013
• Issued: 08/29/2017
• Est. Priority Date: 11/27/2013
• Status: Active Grant

• Current Assignee: Industrial Technology Research Institute
• Original Assignee: Industrial Technology Research Institute
• Inventors: Tai, Chung-Li, Chiu, Chih-Kai, Tsai, Yao-Yang
• Primary Examiner(s): Lefkowitz, Sumati
• Assistant Examiner(s): PERLMAN, DAVID S
• Application Number:
  US14/092,146
• Publication Number:
  US 20150146964A1
• Time in Patent Office: 1,371 Days
• Field of Search: None
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06T 2219/2012 Colour editing, changing, or manipulating; Use of colour codes
• CPC Class Codes: G06T 7/001 using an image reference approach

Abstract

A method for vision machine inspection comprises providing depth information of a target acquired by an image capturing system, determining real-time three-dimensional information of a target object in a predetermined inspecting area based on depth information of at least one real-time image of the target. The method further comprises projecting color pixel information of a real-time color image of the target object to a three-dimensional virtual model based on the real-time three-dimensional information. The real-time color image may be acquired by a color camera system. The method further comprises generating a color three-dimensional virtual model. The color three-dimensional virtual model may comprise the color pixel information.

Claims

1. A method comprising:

receiving depth information of a target object acquired by an image capturing system;

determining real-time three-dimensional information of the target object in a predetermined inspecting area based on the depth information of at least one real-time image of the target object;

determining a position and/or orientation of the target object relative to at least one color camera of a color camera system based on the depth information of the at least one real-time image of the target object and position information of the at least one color camera of the color camera system relative to at least one image capturing camera of the image capturing system, wherein the position information of the at least one color camera of the color camera system relative to the at least one image capturing camera of the image capturing system is fixed and predetermined; and

projecting color pixel information of a real-time color image of the target object onto a three-dimensional virtual model based on the determined position and/or orientation of the target object relative to the at least one color camera of the color camera system to generate a color three-dimensional virtual model comprising the color pixel information, wherein the color pixel information is based on the real-time three-dimensional information of the target object and the real-time color image of the target object acquired by the color camera system.

2. The method of claim 1, further comprising identifying a defect of the target object in the predetermined inspecting area based on a reference.

3. The method of claim 1, further comprising providing one of a real-time three-dimensional virtual model of the target object by the image capturing system, a predetermined three-dimensional virtual model of the target object by the image capturing system, and a predetermined three-dimensional virtual model of the target object by a three-dimensional scanner.

4. The method of claim 1, further comprising comparing assembly information of the target object in the predetermined inspecting area to reference assembly information in the corresponding inspecting area to identify an assembly defect.

5. The method of claim 1, further comprising acquiring position and viewpoint of the image capturing system.

6. The method of claim 1, further comprising providing a plurality of reference images of the target object in the predetermined inspecting area and storing the reference images in a computer-readable storage medium.

7. The method of claim 1, further comprising changing a viewpoint of the three-dimensional virtual model to a predetermined viewpoint.

8. An apparatus for vision machine inspection comprising:

a processor configured to;

receive depth information of a target object acquired by an image capturing system;

determine real-time three-dimensional information of the target object in a predetermined inspecting area based on the depth information of at least one real-time image of the target object;

determine a position and/or orientation of the target object relative to at least one color camera of the color camera system based on the depth information of the at least one real-time image of the target object and position information of the at least one color camera of the color camera system relative to at least one image capturing camera of the image capturing system, wherein the position information of the at least one color camera of the color camera system relative to the at least one image capturing camera of the image capturing system is fixed and predetermined; and

project color pixel information of the at least one real-time color image of the target object onto a three-dimensional virtual model based on the determined position and/or orientation of the target object relative to the at least one color camera of the color camera system to generate a color three-dimensional virtual model comprising the color pixel information, wherein the color pixel information is based on the real-time three-dimensional information of the target object and the at least one real-time color image of the target object acquired by the color camera system.

9. The apparatus of claim 8, wherein the processor is configured to identify a defect of the target object in the predetermined inspecting area based on a reference.

10. The apparatus of claim 8, wherein the processor is configured to compare assembly information of the target object in the predetermined inspecting area to reference assembly information in the corresponding inspecting area to identify an assembly defect.

11. The apparatus of claim 8, wherein the processor is configured to acquire position and viewpoint of the image capturing system.

12. The apparatus of claim 8 further comprising a storage medium to store a plurality of reference images of the predetermined inspecting area.

13. The apparatus of claim 8, wherein the processor is configured to change a viewpoint of the three-dimensional virtual model to a predetermined viewpoint.

14. A computer program product comprising a non-transitory computer readable storage medium and computer program instructions stored therein, the computer program instructions comprising program instructions configured to:

provide depth information of a target object acquired by an image capturing system;

determine real-time three-dimensional information of the target object in a predetermined inspecting area based on the depth information of at least one real-time image of the target object;

determine a position and/or orientation of the target object relative to at least one color camera of a color camera system based on the depth information of the at least one real-time image of the target object and position information of the at least one color camera of the color camera system relative to at least one image capturing camera of the image capturing system, wherein the position information of the at least one color camera of the color camera system relative to the at least one image capturing camera of the image capturing system is fixed and predetermined; and

project color pixel information of a real-time color image of the target object onto a three-dimensional virtual model based on the determined position and/or orientation of the target object relative to the at least one color camera of the color camera system to generate a color three-dimensional virtual model comprising the color pixel information, wherein the color pixel information is based on the real-time three-dimensional information of the target object and the real-time color image of the target object acquired by the color camera system.

15. The computer program product of claim 14, further comprising program instructions configured to identify a defect of the target object in the predetermined inspecting area based on a reference of the predetermined inspecting area.

16. The computer program product of claim 14, further comprising program instructions configured to acquire position and viewpoint of the image capturing system.

17. The computer program product of claim 14, further comprising program instructions configured to provide a plurality of reference images of the target object in the predetermined inspecting area and store the plurality of reference images in the storage medium.

18. The computer program product of claim 14, further comprising program instructions configured to change a viewpoint of the three-dimensional virtual model to a predetermined viewpoint.