Information search and retrieval system

• US 6,463,426 B1
• Filed: 10/26/1998
• Issued: 10/08/2002
• Est. Priority Date: 10/27/1997
• Status: Expired due to Term

• Current Assignee: Massachusetts Institute of Technology
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Sinha, Pawan, Lipson, Pamela R.
• Primary Examiner(s): Jung, David
• Application Number:
  US09/179,541
• Time in Patent Office: 1,443 Days
• Field of Search: 707/1-206, 345/433-443, 345/144-151, 382/154-285
• US Class Current: 1/1
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 2207/30141 Printed circuit board [PCB]
• CPC Class Codes: G06T 2207/30148 Semiconductor; IC; Wafer
• CPC Class Codes: G06T 2207/30152 Solder
• CPC Class Codes: G06T 7/0006 using a design-rule based approach
• CPC Class Codes: G06T 7/001 using an image reference approach
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: G06V 10/754 involving a deformation of the sample pattern or of the reference pattern; Elastic matching
• CPC Class Codes: G06V 2201/06 Recognition of objects for industrial automation
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99936 Pattern matching access

Abstract

An image search and retrieval system and method of operation within such a system are disclosed. The image search and retrieval system includes a search engine coupled to an image analyzer, an input system, an output system and an image database. The image search and retrieval system includes one or more feature modules which define particular regions of an image and particular measurements to make on pixels within the defined image region as well as the measurements to make on neighboring pixels in neighboring image regions for a given application and communicate this application specific information to the image analyzer thereby allowing rapid comparisons of images.

Claims

1. An image processing system comprising:

(a) a search engine;

(b) an image analyzer coupled to receive images from said search engine and to compare a first image in a first field of use to a second image in the first field of use provided thereto from said search engine; and

(c) one or more feature modules, each of said one or more feature modules coupled to at least one of said search engine and said image analyzer with each individual one of said one or more feature modules specifying information relevant to a particular field of use and wherein a particular one of the one or more feature modules provides to said image analyzer, application specific information which is used by said image analyzer to compare the first and second images, wherein the information corresponds to at least one of an image parameter, an image characteristic and a constraint in the first field of use.

2. The system of claim 1 further comprising an input system coupled to one of said search engine and said image analyzer, said input system comprising at least one of:

a graphical user interface;

a facsimile system;

a camera system;

a scanner;

a network connection; and

a video system.

3. The system of claim 2 wherein each of said one or more feature modules defines at least one measurement to make on one or more pixels in an image region neighboring the one particular image region.

4. The system of claim 1 wherein each of said one or more feature modules defines at least one particular region of an image and at least one particular measurement to make on one or more pixels within each of the at least one particular image regions.

5. The system of claim 3 further comprising a storage device having at least one image stored therein coupled to one of said a search engine, said image analyzer and said feature module.

6. A process for comparing two images comprising the steps of:

(a) providing a compact primary image by providing application specific information to collapse at least one primary image subregion;

(b) aligning a target image having a first predetermined characteristic and the compact primary image having the first predetermined characteristic;

(c) dividing the compact primary image into a plurality of image regions each of the image regions of the primary image having a set of image region characteristics;

(d) selecting a primary image region of the compact primary image;

(e) dividing the target image into a plurality of target image regions;

(f) selecting one of the plurality of target image regions; and

(g) comparing one or more image region characteristics of the selected one of the primary image region of the compact primary image to corresponding image region characteristics in the selected one of the plurality of target image regions.

7. The process of claim 6, further comprising the steps of:

(g) assigning a score indicating the difference between the set of image characteristics in the primary image region of the compact primary image and the corresponding set of image characteristics in the selected one of the plurality of target image regions;

(h) selecting a next one of the plurality of image regions in the target image;

(i) comparing one or more image region characteristics of the selected next one of the primary image region of the compact primary image to corresponding image region characteristics in the selected next one of the plurality of target image regions; and

(j) assigning a score indicating the difference between the image region characteristics in the primary image region of the compact image and the corresponding image region characteristics in the next target image region.

8. The process of claim 7 further comprising the step of repeating steps (h)-(j) for each of a predetermined number of regions in the target image.

9. The process of claim 8 further comprising the step of finding the best match between the primary image region of the compact image and one of the predetermined regions in the target image.

10. The process of claim 9, further comprising the steps of:

selecting a next primary image region in the compact primary image; and

repeating steps (h)-(j) for each desired primary image region.

11. The process of claim 10, further comprising the steps of:

computing a total match score; and

providing said total match score as output.

12. The process of claim 6, further comprising the step of:

combining the set of image region characteristics in first ones of the plurality of image regions with the set of image region characteristics in second ones of the plurality of image regions of the selected image to provide a plurality of combined image regions which represent a compact primary image wherein each of the combined image regions are provided having a set of image region characteristics which are a combination of the image region characteristics of at least a first and a second image region of the primary image.

15. The process of claim 11, wherein the step combining the set of primary image region characteristics in first ones of the plurality of primary image regions with the set of image region characteristics in second ones of the plurality of primary image regions to provide a plurality of combined image regions which compactly represent the primary image as a compact primary image further comprises the step of combining primary image region characteristics in neighboring primary image regions to provide combined image regions.

13. A method of manufacturing a printed circuit board comprising the steps of:

(a) performing a manufacturing operation on a printed circuit board;

(b) obtaining an electronic image of the manufacturing operation wherein the image includes at least a portion of the printed circuit board and wherein the image corresponds to a primary image having a first application specific predetermined characteristic;

(c) aligning the primary image to a target image of the manufacturing operation wherein the target image includes at least the portion of the printed circuit board included in the primary image and wherein the target image has the first predetermined characteristic;

(d) dividing the primary image into a plurality of primary image regions each having a set of application specific primary image region characteristics;

(e) combining the set of primary image region characteristics in first ones of the plurality of primary image regions with the set of image region characteristics in second ones of the plurality of primary image regions to provide a plurality of combined image regions which represent the primary image wherein each of the combined image regions are provided having a set of image region characteristics which are a combination of the image region characteristics of at least first and second primary image regions;

(f) selecting one of the plurality of combined image regions;

(g) dividing the target image into a plurality of target image regions;

(h) selecting one of the plurality of target image regions; and

(i) inspecting the result of the manufacturing operation by comparing one or more image region characteristics of the selected one of the plurality of combined image regions to corresponding image region characteristics in the selected one of the plurality of target image regions.

14. The method of claim 13 further comprising the steps of:

capturing an image of a portion of the printed circuit board wherein said captured image portion includes an image of the printed circuit board having the manufacturing operation applied thereto;

selecting a region of the captured image;

aligning a target image and the selected region of the captured wherein the target image corresponds to an image of a printed circuit board which has been properly manufactured in during that particular portion of the manufacturing process and wherein each of the target and selected images has at least one property;

dividing the selected image into a plurality of image regions each of the images regions having at least one property;

combining properties in predetermined ones of the plurality of image regions;

selecting a primary image region within the selected image;

selecting a target image region;

comparing one or more properties of the primary image region within the selected image to a corresponding one or more properties in the target image region; and

in response to the primary image being substantially similar to the target image in said comparing step, utilizing, as the target images, images of like portions of the printed circuit board having the same manufacturing operation applied thereto as the primary image.

16. The process of claim 13 wherein for each of a predetermined number of regions in the target image the method further comprises the step of repeating the steps of:

(1) selecting a next region in the target image;

(2) comparing one or more image region characteristics of the compact image region to a corresponding one or more image region characteristics in the next target image region; and

(3) assigning a score indicating the difference between the one or more image region characteristics in the compact image region and the corresponding one or more image region characteristics in the next target image region.

17. The process of claim 16 further comprising the step of finding the best match between the compact image region and one of the predetermined regions in the target image.

18. The process of claim 13, wherein the target image corresponds to an image of a printed circuit board portion which has failed a printed circuit board inspection process.

19. The process of claim 13 wherein the manufacturing operation corresponds to a solder paste application step.

20. The process of claim 13 wherein said manufacturing operation corresponds to a component placement operation.

21. The process of claim 13 wherein said manufacturing operation corresponds to a solder reflow operation.

22. The process of claim 18 wherein said manufacturing operation corresponds to a solder operation and the method further includes the step of utilizing relative brightness between predetermined regions at and proximate to a solder joint to verify the acceptability of the solder joint.

23. A memory for storing a model for access by an image processing system, the model comprising:

a plurality of image regions each of the image regions from a primary image having a set of application specific image region characteristics; and

a set of image region characteristics wherein the set of region characteristics are provided from a combination of the set of image region characteristics in first ones of the plurality of image regions with second ones of the plurality of image regions to provide a plurality of combined image regions which represent a compact primary image wherein each of the combined image regions are provided having a set of image region characteristics which are a combination of the image region characteristics of at least a first and a second image region of the primary image.

24. The memory of claim 23 further wherein at least one of the combined image regions includes a combined spatial and a combined photometric relationship between a first one of the plurality of image patches and a second one of the plurality of image patches within the model.

25. The memory of claim 24 wherein the least one of the combined image regions includes the characteristics of a third one of the plurality of image regions.

26. The memory of claim 24 wherein each of the plurality of image regions correspond to a subset of an entire image.