Method and apparatus for training a probe model based machine vision system

• US 8,457,390 B1
• Filed: 10/10/2008
• Issued: 06/04/2013
• Est. Priority Date: 10/10/2008
• Status: Active Grant

• Current Assignee: Cognex Corporation
• Original Assignee: Cognex Corporation
• Inventors: Wagman, Adam, Wallack, Aaron, Michael, David J., Barker, Simon
• Primary Examiner(s): Park, Edward
• Application Number:
  US12/249,318
• Time in Patent Office: 1,698 Days
• Field of Search: 382/155, 382/161, 382/159, 700/47, 700/48
• US Class Current: 382/159
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30208 Marker matrix
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/243 by compensating for image skew or non-uniform image deformations
• CPC Class Codes: G06V 30/1914 Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries, e.g. user dictionaries
• CPC Class Codes: G06V 30/2504 Coarse or fine approaches, e.g. resolution of ambiguities or multiscale approaches

Abstract

A method for training a pattern recognition algorithm for a machine vision system that uses models of a pattern to be located, the method comprising the steps of training each of a plurality of models using a different training image wherein each of the training images is a version of a single image of the pattern at a unique coarse image resolution, using the models to identify at least one robust image resolution where the image resolution is suitable for locating the pattern within an accuracy limit of the actual location of the pattern in the image and storing the at least one robust image resolution for use in subsequent pattern recognition processes.

Claims

1. A method for training a pattern recognition algorithm for a machine vision system that uses models of a pattern to be located, the method comprising the steps of:

training each of a plurality of models using a different training image wherein each of the training images is a version of at least one image of the pattern at a unique coarse image resolution, wherein each model includes a plurality of probes that are each a measure of similarity of at least one of a run-time image feature and a run time image region to at least one of a pattern feature and a pattern region, respectively, at a specific location;

using the models to identify at least one robust image resolution where the robust image resolution is suitable for locating the pattern within an accuracy limit of the actual location of the pattern in the at least one image of the pattern;

storing the at least one robust image resolution for use in subsequent pattern recognition processes during run time image processing;

applying the plurality of probes at a plurality of poses to a run-time image resulting in a plurality of score spaces at each pose where the score spaces include score-space peaks, the score-space peaks indicating likely locations of occurrences of the pattern in the run-time image at each pose;

and comparing the score-space peaks to an accept threshold where the results of the comparison are used to identify poses of instances of the model in the image.

2. The method of claim 1 wherein the pattern recognition algorithm searches for the pattern in a translation degree of freedom and at least one further degree of freedom.

3. The method of claim 1 further including the step of identifying at least one image resolution and scan code combination where the image resolution is suitable for locating the pattern within the accuracy limit of the actual location of the pattern in the at least one image of the pattern and the scan code is robust at the image resolution in the image resolution and scan code combination, the step of storing the at least one robust image resolution including storing at least one robust image resolution and scan code combination.

4. The method of claim 3 further including the steps of identifying the known location of the pattern within an image and wherein the step of using the models to identify includes using the models to identify a fastest combination by using a first model associated with a finest image resolution to generate a first score space for a first image, examining the first score space to identify a repeating pattern frequency, for each of the other models associated with different resolutions, using the model to generate an additional score space and a location error, the location error indicating the difference between the known location and a location determined using the additional score space, when the location error is less than a desired accuracy distance, identifying the image resolution as a possible image resolution for the fastest image resolution and scan code combination.

5. The method of claim 4 wherein the step of using the model to generate an additional score space includes generating a set of lower resolution images at different coarse subpixel phases and using the model associated with the image resolution to generate a separate additional score space and a location error for each of the lower resolution images and wherein the step of identifying the image resolution as a possible image resolution for the fastest image resolution and scan code combination includes identifying the image resolution as a possible image resolution for the fastest image resolution and scan code combination when each of the location errors generated using the model is less than a threshold percentage of the repeating pattern frequency.

6. The method of claim 4 wherein the step of using the models to identify the fastest image resolution and scan code combination further includes the steps of, for each of the possible resolutions, identifying the sparsest robust scan code.

7. The method of claim 6 wherein the step of identifying the sparsest robust scan code for a particular image resolution includes examining the additional score space generated using the model associated with the image resolution using at least one additional scan code that is coarser than the full scan code to determine if the additional scan code is robust at the particular image resolution.

8. The method of claim 7 wherein the step of using the additional scan code to examine the score space includes identifying a peak score in the score space, identifying a worse case peak score in a scan region centered on the peak score in the score space, comparing the worse case peak score to a highest other score within the score space outside the scan region and, where the worse case peak score is greater than a threshold percentage of the highest other score, identifying the additional scan code as a possible sparsest robust scan code for the image resolution.

9. The method of claim 8 wherein the threshold percentage is greater than 100 percent.

10. The method of claim 8 wherein the step of using the half scan code to examine the score space includes identifying a worse case half scan peak score in a one-by-three region within the score space and centered on the peak score in the score space, comparing the worse case half scan peak value to a highest other score within the score space outside the one-by-three region and, where the worse case half scan peak score is greater than a threshold percentage of the highest other score outside the one-by-three region, identifying the half scan code as the sparsest robust scan code for the image resolution.

11. The method of claim 10 wherein, when the half scan and quarter scan codes are not robust, the full scan code is identified as the sparsest robust scan code for the image resolution.

12. The method of claim 7 wherein the additional scan code is a quarter scan code, the scan region centered on the peak score includes a three-by-three region within the score space and wherein, when the worse case peak score is greater than a threshold percentage of the highest other score, the step of identifying the additional scan code as a possible sparsest robust scan code for the image resolution includes identifying the additional scan code as the sparsest robust scan code for the image resolution.

13. The method of claim 12 wherein the step of using at least one additional scan code includes, when the quarter scan code is not robust, using a half scan code to examine the score space.

14. The method of claim 1 wherein the pattern includes repeating elements and wherein the step of using the models to identify at least one robust image resolution includes using the models to identify at least one image resolution where the image resolution is suitable for locating the pattern within a fraction of one pitch of the repeating elements.

15. The method of claim 1 wherein the step of using the models to identify at least one robust image resolution that is suitable for locating the pattern within an accuracy limit of the actual location of the pattern in the at least one image of the pattern includes using the models to identify at least one image resolution usable to find the pattern within an accuracy limit that at least one of (i) is a function of the capabilities of further refinement steps of the recognition algorithm to be able to refine the location of the pattern to a higher level of accuracy;

(ii) is specified within a spatial, a rotation, a scale, a shear and an aspect degree of freedom;

(iii) is a function of the frequency content of the training pattern and (iv) any combination of (i) through (iii) above.

16. The method of claim 1 wherein the step of using the models to identify at least one robust image resolution where the robust image resolution is suitable for locating the pattern within an accuracy limit of the actual location of the pattern in the at least one image of the pattern includes using different models associated with different image resolutions to examine images until at least one robust image resolution is identified and wherein the order of models used includes one of (i) starting with a coarse resolution model and linearly progressing to finer resolution models via integers;

(ii) starting with one resolution and progressing in a non-linear fashion to other resolutions; and

(iii) using at least some non-integer resolutions.

17. The method of claim 1 wherein the step of using the models to identify at least one robust image resolution where the robust image resolution is suitable for locating the pattern within an accuracy limit of the actual location of the pattern in the at least one image of the pattern includes generating a plurality of test images at the robust image resolution, each test image characterized by a different combination of degrees of freedom where the degrees of freedom include at least a sub-set of rotation, shear, aspect and scale values and applying the model to each of the plurality of test images to determine if pattern location is identified within the accuracy limit of the actual location of the pattern in the images.

18. The method of claim 1 further including the step of providing the robust image resolution for manual entry into a run-time pattern recognition system.

19. The method of claim 1 further including the step of providing the robust image resolution and scan code combination for manual entry into a run-time pattern recognition system.

20. A method for training a pattern recognition algorithm for a machine vision system that uses models of a pattern to be located wherein the pattern includes repeating elements, the method comprising the steps of:

training each of a plurality of models using a different training image wherein each of the training images is a version of at least one image of the pattern at a unique coarse image resolution;

using the models to identify at least one robust image resolution where the robust image resolution is suitable for locating the pattern within a fraction of one pitch of the repeating elements;

storing the at least one robust image resolution for use in subsequent pattern recognition processes during run time image processing;

wherein the steps of training, using and storing include, starting with a coarse image having a coarse image resolution where the coarse image resolution is a test image resolution;

(a) using the image at the test image resolution to train a model;

(b) running the model on the image at the test image resolution to generate a score space and a location error;

(c) using the location error to determine if the test image resolution is suitable for locating the pattern within a fraction of one pitch of the repeating elements;

(d) where the test image resolution is suitable for locating the pattern within a fraction of one pitch of the repeating elements, using the score space to identify a sparsest robust scan code for the test image resolution where the test image resolution and sparsest robust scan code comprise a test image resolution and scan code combination;

(e) storing the test image resolution and scan code combination for use in subsequent pattern recognition processes as a stored image resolution/scan code combination;

(f) determining if a less coarse image resolution and scan code combination could be faster than the stored image resolution/scan code combination;

(g) where a less coarse image resolution and scan code combination cannot be faster than the stored image resolution/scan code combination, ending the process; and

(h) where a less coarse image resolution and scan code combination could be faster than the stored image resolution/scan code combination, repeating steps (a) through (g) using a less coarse test image resolution and only scan codes that could be faster at the less coarse test image resolution than the stored image resolution/scan code combination.

21. The method of claim 20 further including the steps of identifying the known location of the pattern within a finest image resolution image, using an image at the finest image resolution to train a model, using the model associated with a finest image resolution to generate a score space and examining the score space to identify a repeating pattern frequency, the step of using the location error to determine if the test image resolution is suitable for locating the pattern within a fraction of one pitch of the repeating elements including comparing the location error to a threshold percentage of the repeating pattern frequency and when the location error is less than the threshold percentage of the repeating pattern frequency, determining that the test image resolution is suitable for locating the pattern within a fraction of one pitch of the repeating elements.

22. The method of claim 20 wherein the scan codes include a full scan code, a half scan code and a quarter scan code.

23. The method of claim 22 wherein the steps of using the score space to identify a sparsest robust scan code for the test image resolution and the step of determining if a less coarse image resolution and scan code combination could be faster than the stored image resolution/scan code combination include testing the quarter scan code first and, where the quarter scan code is robust, determining that a less coarse image resolution and scan code combination cannot be faster than the stored image resolution/scan code combination.

24. The method of claim 20 further including the steps of specifying a minimum number of probes that are required in each model and, after a model is trained and prior to running the model on an image, determining if the model has at least the minimum number of probes and, where the model does not have at least the minimum number of probes, skipping to step (f).

25. An apparatus for training a pattern recognition algorithm for a machine vision system that uses models of a pattern to be located, the apparatus comprising:

a processor programmed to perform the steps of;

training each of a plurality of models using a different training image wherein each of the training images is a version of a single image of the pattern at a unique coarse image resolution, each model including a plurality of probes that are each a measure of similarity of at least one of a run-time image feature and a run time image region to at least one of a pattern feature and a pattern region, respectively, at a specific location;

using the models to identify at least one robust image resolution where the image resolution is suitable for locating the pattern within an accuracy limit of the actual location of the pattern in the single image of the pattern; and

storing the at least one robust image resolution for use in subsequent pattern recognition processes during run time image processing;

applying the plurality of probes at a plurality of poses to a run-time image resulting in a plurality of score spaces at each pose where the score spaces include score-space peaks, the score-space peaks indicating likely locations of occurrences of the pattern in the run-time image at each pose; and

comparing the score-space peaks to an accept threshold where the results of the comparison are used to identify poses of instances of the model in the image.

26. The apparatus of claim 25 wherein the pattern recognition algorithm searches for the pattern in a translation degree of freedom and at least one further degree of freedom.

27. The apparatus of claim 25 wherein the processor is further programmed to perform the step of identifying a fastest image resolution and scan code combination where the image resolution is suitable for locating the pattern within the accuracy limit of the actual location of the pattern in the single image and the scan code is robust at the image resolution in the image resolution and scan code combination, the processor programmed to perform the step of storing the at least one robust image resolution by storing at least one robust image resolution and scan code combination.

28. The apparatus of claim 27 wherein the pattern includes repeating elements and wherein the processor is programmed to perform the step of using the models to identify at least one robust image resolution by using the models to identify at least one image resolution where the image resolution is suitable for locating the pattern within a fraction of one pitch of the repeating elements.

29. The apparatus of claim 28 wherein the processor is further programmed to perform the step of identifying the known location of the pattern within an image and wherein the processor is programmed to perform the step of using the models to identify includes using the models to identify a fastest combination by using a first model associated with a finest image resolution to generate a first score space for a first image, examining the first score space to identify a repeating pattern frequency, for each of the other models associated with different resolutions, using the model to generate an additional score space and a location error, the location error indicating the difference between the known location and a location determined using the additional score space, when the location error is less than a threshold percentage of the repeating pattern frequency, identifying the image resolution as a possible image resolution for the fastest image resolution and scan code combination.

30. The apparatus of claim 28 wherein the processor is programmed to perform the step of using the model to generate an additional score space by generating a set of lower resolution images at different coarse subpixel phases and using the model associated with the image resolution to generate a separate additional score space and a location error for each of the lower resolution images, the processor programmed to perform the step of identifying the image resolution as a possible image resolution for the fastest image resolution and scan code combination by identifying the image resolution as a possible image resolution for the fastest image resolution and scan code combination when each of the location errors generated using the model is less than a threshold percentage of the repeating pattern frequency.

31. The apparatus of claim 30 wherein the processor is programmed to perform the step of using the models to identify the fastest image resolution and scan code combination by, for each of the possible resolutions, identifying the sparsest robust scan code.

32. The apparatus of claim 31 wherein the processor is programmed to perform the step of identifying the sparsest robust scan code for a particular image resolution by examining the additional score space generated using the model associated with the image resolution using at least one additional scan code that is coarser than the full scan code to determine if the additional scan code is robust at the particular image resolution.

33. The method of claim 31 wherein the processor is programmed to perform the step of using the additional scan code to examine the score space by identifying a peak score in the score space, identifying a worse case peak score in a scan region centered on the peak score in the score space, comparing the worse case peak score to a highest other score within the score space outside the scan region and, where the worse case peak score is greater than a threshold percentage of the highest other score, identifying the additional scan code as a possible sparsest robust scan code for the image resolution.

34. The apparatus of claim 28 wherein the additional scan code includes at least one of a quarter scan code and a half scan code.

35. The apparatus of claim 25 wherein the processor is programmed to perform the step of using the models to identify at least one robust image resolution that is suitable for locating the pattern within an accuracy limit of the actual location of the pattern in the single image by using the models to identify at least one image resolution usable to find the pattern within an accuracy limit that at least one of (i) is a function of the capabilities of further refinement steps of the recognition algorithm to be able to refine the location of the pattern to a higher level of accuracy;

(ii) is specified within a spatial, a rotation, a scale, a shear and an aspect degree of freedom;

(iii) is a function of the frequency content of the training pattern and (iv) any combination of (i) through (iii) above.

36. The apparatus of claim 25 wherein the processor performs the step of using the models to identify at least one robust image resolution where the robust image resolution is suitable for locating the pattern within an accuracy limit of the actual location of the pattern in the single image by using different models associated with different image resolutions to examine images until at least one robust image resolution is identified and wherein the order of models used includes one of (i) starting with a coarse resolution model and linearly progressing to finer resolution models via integers;

(ii) starting with one resolution and progressing in a non-linear fashion to other resolutions; and

(iii) using at least some non-integer resolutions.

37. The apparatus of claim 25 wherein the processor performs the step of using the models to identify at least one robust image resolution where the robust image resolution is suitable for locating the pattern within an accuracy limit of the actual location of the pattern in the single image by generating a plurality of test images at the robust image resolution, each test image characterized by a different combination of degrees of freedom where the degrees of freedom include at least a sub-set of rotation, shear, aspect and scale values and applying the model to each of the plurality of test images to determine if pattern location is identified within the accuracy limit of the actual location of the pattern in the images.

38. The apparatus of claim 25 wherein the processor is further programmed to provide the robust image resolution for manual entry into a run-time pattern recognition system.

39. The apparatus of claim 25 wherein the processor is further programmed to providing the robust image resolution and scan code combination for manual entry into a run-time pattern recognition system.

40. The apparatus for training a pattern recognition algorithm for a machine vision system that uses models of a pattern to be located wherein the pattern includes repeating elements, the apparatus comprising:

a processor programmed to perform the steps of;

training each of a plurality of models using a different training image wherein each of the training images is a version of a single image of the pattern at a unique coarse image resolution;

using the models to identify at least one robust image resolution where the image resolution is suitable for locating the pattern within a fraction of one pitch of the repeating elements in the single image of the pattern;

storing the at least one robust image resolution for use in subsequent pattern recognition processes during run time image processing;

wherein the processor is further programmed to perform the step of identifying a fastest image resolution and scan code combination where the image resolution is suitable for locating the pattern within the accuracy limit of the actual location of the pattern in the single image and the scan code is robust at the image resolution in the image resolution and scan code combination, the processor programmed to perform the step of storing the at least one robust image resolution by storing at least one robust image resolution and scan code combination; and

wherein the processor is programmed to perform the steps of training, using and storing by, starting with a coarse image having a coarse image resolution where the coarse image resolution is a test image resolution;

(a) using the image at the test image resolution to train a model;

(b) running the model on the image at the test image resolution to generate a score space and a location error;

(c) using the location error to determine if the test image resolution is suitable for locating the pattern within a fraction of one pitch of the repeating elements;

(d) where the test image resolution is suitable for locating the pattern within a fraction of one pitch of the repeating elements, using the score space to identify a sparsest robust scan code for the test image resolution where the test image resolution and sparsest robust scan code comprise a test image resolution and scan code combination;

(e) storing the test image resolution and scan code combination for use in subsequent pattern recognition processes as a stored image resolution/scan code combination;

(f) determining if a less coarse image resolution and scan code combination could be faster than the stored image resolution/scan code combination;

(g) where a less coarse image resolution and scan code combination cannot be faster than the stored image resolution/scan code combination, ending the process; and

(h) where a less coarse image resolution and scan code combination could be faster than the stored image resolution/scan code combination, repeating steps (a) through (g) using a less coarse test image resolution and only scan codes that could be faster at the less coarse test image resolution than the stored image resolution/scan code combination.

41. The apparatus of claim 40 wherein the processor is further programmed to perform the steps of identifying the known location of the pattern within a finest image resolution image, using an image at the finest image resolution to train a model, using the model associated with a finest image resolution to generate a score space and examining the score space to identify a repeating pattern frequency, the processor programmed to perform the step of using the location error to determine if the test image resolution is suitable for locating the pattern within a fraction of one pitch of the repeating elements by comparing the location error to a threshold percentage of the repeating pattern frequency and when the location error is less than the threshold percentage of the repeating pattern frequency, determining that the test image resolution is suitable for locating the pattern within a fraction of one pitch of the repeating elements.