Method and apparatus for analyzing an image to detect and identify patterns

• US 6,650,779 B2
• Filed: 03/26/1999
• Issued: 11/18/2003
• Est. Priority Date: 03/26/1999
• Status: Expired due to Fees

• Current Assignee: Georgia Tech Research Corporation (University System of Georgia)
• Original Assignee: Georgia Tech Research Corporation (University System of Georgia)
• Inventors: Wang, Peng, Echauz, Javier, Dorrity, Lewis J., Vachtesvanos, George J., Mufti, Muid
• Primary Examiner(s): Dastouri, Mehrdad
• Application Number:
  US09/280,145
• Publication Number:
  US 20020054694A1
• Time in Patent Office: 1,698 Days
• Field of Search: 382/190, 382/191, 382/192, 382/195, 382/218, 382/224, 382/111, 382/279, 382/156, 382/157, 382/228, 375/240.19, 706/15, 706/25, 348/88, 348/91, 348/92, 348/128
• US Class Current: 382/228
• CPC Class Codes: G01N 21/898 Irregularities in textured or patterned surfaces, e.g. textiles, wood
• CPC Class Codes: G01N 21/956 Inspecting patterns on the surface of objects contactless testing of electronic circuits G01R31/308; testing currency G07D; manufacturing processes per se of semiconductor devices implementing a measuring step H01L22/10
• CPC Class Codes: G06T 2207/20064 Wavelet transform [DWT]
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30124 Fabrics; Textile; Paper
• CPC Class Codes: G06T 7/0006 using a design-rule based approach
• CPC Class Codes: G06V 10/421 by analysing segments intersecting the pattern
• CPC Class Codes: G06V 10/52 Scale-space analysis, e.g. wavelet analysis multi-scale boundary representations G06V10/42
• CPC Class Codes: G06V 2201/06 Recognition of objects for industrial automation

Abstract

A method and apparatus is provided which analyzes an image of an object to detect and identify defects in the object utilizing multi-dimensional wavelet neural networks. “The present invention generates a signal representing part of the object, then extracts certain features of the signal. These features are then provided to a multidimensional neural network for classification, which indicates if the features correlate with a predetermined pattern. This process of analyzing the features to detect and identify predetermined patterns results in a robust fault detection and identification system which is computationally efficient and economical because of the learning element contained therein which lessens the need for human assistance.”

Claims

1. An apparatus for analyzing a 2-D representation of an object, said apparatus comprising:

at least one sensor disposed to capture a 2-D representation of at least a portion of an object;

a memory that stores at least a portion of said 2-D representation received from said sensor;

a processor containing a program module operative to;

receive said stored portion of said 2-D representation;

derive a plurality of features from said stored portion of said representation;

provide said features to a multi-dimensional wavelet neural network, said multi-dimensional wavelet neural network incorporating a learning technique from the following group consisting of structure learning, parameter learning, or combined parameter and structure learning for classification performance, wherein said features are compared to a predetermined fault pattern to determine if the features represent a defect; and

produce a classification output indicative of whether said stored portion of said representation comprises a defect; and

a decision logic unit which receives said features and said classification output and determines if comparing said features to said predetermined fault patterns results in a classification output which is potentially indicative of a predetermined fault pattern, wherein said decision logic unit resolves said classification output by referencing said multi-dimensional wavelet neural network to provide a final declaration as to whether the classification output should be classified as a defect.

2. An apparatus according to claim 1 wherein said at least one sensor comprises a CCD array camera that captures said 2-D representation.

3. An apparatus according to claim 1 further comprising a light source that produces signals having a frequency in either the infared or visible range and wherein said at least one sensor comprises a vision camera responsive to said signals in the infared or visible frequency range.

4. An apparatus according to claim 3 further comprising a light diffuser disposed between said light source and said object for controlling the uniformity and intensity of the signals being projected onto the object.

5. An apparatus according to claim 1 comprising an analog to digital converter that converts said 2-D representation to a digital representation and provides said digital representation to said memory, and wherein said processor derives said at least one signal from said digital representation.

6. An apparatus according to claim 1 wherein said processor derives said at least one signal by projecting said 2-D representation into a 1-D signal.

7. An apparatus according to claim 6 wherein said processor derives said 1-D signal from said 2-D representation by utilizing a projection from the group consisting of vertical scanning, horizontal scanning and fractal scanning.

8. An apparatus according to claim 6 wherein said processor generates some of said features by deriving values from said 1-D signal.

9. An apparatus according to claim 6 wherein said processor generates a feature vector by storing said feature values in an array.

10. An apparatus according to claim 1 wherein said processor determines whether at least one of said features is within a range indicative of a predetermined fault pattern and does not provide said features to said multi-dimensional wavelet neural network if said at least one feature is not within said range.

11. An apparatus according to claim 1 further comprising a decision logic module that receives said classification output from said multi-dimensional wavelet neural network and said features to determine the degree of certainty that said pattern recognition system determines that said predetermined pattern was detected.

12. An apparatus according to claim 1 wherein the object is textile fabric being manufactured by warp knitting.

13. An apparatus according to claim 1 wherein the object is greige fabric.

14. An apparatus according to claim 1 wherein the object is textile fabric being inspected on a device selected from a group consisting of a circular loom, loom, an off-line inspection station or a weaving machine.

15. The apparatus of claim 1, the program module further comprising instructions to provide for a learning element wherein extraction is based on features present in the multi-dimensional wavelet neural network which features are updated based on the resolution of conflicts in the classification output concerning prior derived signals from said logic unit.

16. A method for recognizing defects in an object, comprising:

generating a 2-D digital representation of at least part of an object, the digital representation comprising a plurality of pixels;

extracting a plurality of features from said 2-D digital representation based on classification characteristics;

providing said features to a multi-dimensional wavelet neural network, said multidimensional wavelet neural network incorporating a learning technique from the following group consisting of structure learning, parameter learning, or combined parameter and structure learning for classification performance;

comparing said features to a predetermined fault pattern to determine if the feature represents a defect, and

providing said features and said classification output from said multi-dimensional wavelet neural network to a logic unit to determine if comparing said features to said predetermined fault pattern results in an uncertain classification output, wherein said logic unit resolves uncertainty in said classification output by referencing said multi-dimensional wavelet neural network to determine whether the uncertain classification output should be classified as a defect.

17. A method according to claim 16 wherein said predetermined fault pattern is one of a group of predetermined fault patterns and said multi-dimensional wavelet neural network comprises a plurality of wavelet neurons, wherein the step of providing from said multi-dimensional wavelet neural network comprises a step of determining what classification output to provide by providing an output of said neurons to a competition function, wherein said competition functions provides said classification output.

18. A method according to claim 16 further comprising a step of deriving a 1-D signal from said 2-D representation and wherein the step of extracting a plurality of features from said 2-D digital representation comprises extracting a plurality of features from said 1-D signal.

19. A method according to claim 18 wherein said step of extracting said plurality of features comprises deriving at least one feature of said plurality of features from said 1-D signal.

20. A method according to claim 18 wherein said step of extracting said plurality of features comprises a step convolving said 1-D signal with a plurality of wavelet functions utilizing fast wavelet transforms to produce a plurality of fast wavelet coefficients corresponding to at least one feature of said plurality of features.

21. A method according to claim 18 wherein said step of extracting said plurality of features comprises the steps of:

deriving at least one feature of said plurality of feature values from said 1-D signal;

convolving said 1-D signal with a plurality of wavelet functions utilizing fast wavelet transforms to produce a plurality of wavelet coefficients corresponding to at least one feature of said plurality of features; and

arranging said plurality of features into a feature vector.

22. The method of claim 16 further comprising a step of determining whether at least one of said plurality of features comprise a value indicative of a predetermined pattern, and if said value is not indicative of a predetermined pattern not providing said features to said multi-dimensional wavelet neural network.

23. The method of claim 16 wherein said object comprises a textile material.

24. The method of claim 16, wherein extracting a feature comprises a learning element wherein extraction is based on features present in the multi-dimensional wavelet network which are updated based on the resolution of conflicts in the classification output concerning prior derived signals from said logic unit.

25. A computer readable medium containing instructions for a computer comprising:

means for instructing the computer to read at least a portion of a 2-D digital image, said digital image comprising a plurality of pixels;

means for instructing the computer to generate a feature vector from said digital image based on classification characteristics;

means for instructing the computer to provide said feature vector to a multidimensional wavelet neural network;

means for instructing the computer to provide a classification output indicative of whether said feature vector corresponds to a predetermined pattern; and

means for resolving any conflicts arising from providing such classification output by referencing said multi-dimensional wavelet neural network to determine which one of a plurality of potentially identified classification outputs should be classified as a defect.

26. The computer readable medium of claim 25 further comprising means for instructing the computer to generate a 1-D representation of said 2-D digital image, wherein said feature vector is generated from said 1-D representation.

27. The computer readable medium of claim 25 wherein said means for instructing the computer to generate a feature vector comprises means for instructing the computer to derive said features from said 1-D signal.

28. A computer readable medium of claim 25 wherein said means for instructing the computer to generate a feature vector comprises means for instructing the computer to convolve said 1-D signal with a plurality of fast wavelet functions.

29. A computer readable medium according to claim 25 further comprising means for instructing said computer to generate a feature library comprising a plurality of selected features, wherein said computer is instructed by said means for instructing the computer to generate said feature vector so that the components of said feature vector each correspond to a value of one of said selected features.

30. A computer readable medium according to claim 25 wherein said multi-dimensional wavelet neural network comprises a plurality of wavelet functions, the computer readable medium further comprising means for instructing said computer to adjust said wavelet functions based upon known feature vectors of said predetermined patterns.

31. A computer readable medium according to claim 25 further comprising means for instructing the computer to determine whether said feature vector comprise at least one value indicative of a predetermined pattern, and if said at least one value is not indicative of a predetermined pattern instructing said computer not provide said feature vector to said multi-dimensional wavelet neural network.

32. The computer readable medium of claim 25, wherein the means for instructing the computer to generate a feature vector further comprises a learning element wherein extraction is based on features present in the multi-dimensional wavelet network which are updated based on the resolution of conflicts in the classification output concerning prior derived signals from said logic unit.

33. An apparatus for pattern recognition comprising:

an input that receives a 2-D representation of at least part of an object;

a memory that stores at least a portion of said 2-D representation; and

a processor that generates a plurality of feature values representing features of said at least one signal and that provides each of said feature values to a perceptron neural network comprising a plurality of neurons each defined by the function ψ

_a,b={square root over (diag(a))}(diag(a)(x−

b)) where x is a vector comprising said feature values, a is a squashing matrix for the neuron and b is the translation vector for that neuron, said perceptron neural network providing a classification output indicative of whether said representation contains a predetermined pattern.

34. An apparatus according to claim 33 wherein said processor derives at least one 1-D signal from said 2-D representation and said processor derives said features from said at least one 1-D signal.

35. An apparatus according to claim 34 wherein said processor derives said 1-D signal from said 2-D representation by utilizing a projection from the group consisting of vertical scanning, horizontal scanning and fractal scanning.

36. An apparatus according to claim 34 wherein said processor generates some of said features by deriving values from said 1-D signal.

37. An apparatus according to claim 34 wherein said processor generates some of said features by convolving said 1-D signal with a plurality of fast wavelet functions.

38. An apparatus according to claim 34 wherein said processor generates a feature vector by storing said features in an array.

39. An apparatus according to claim 33 wherein said processor determines whether at least one of said features is within a range indicative of a predetermined pattern and does not provide said features to said multi-dimensional wavelet neural network if said at least one feature is not within said range.

40. An apparatus according to claim 33 further comprising decision logic module that receives said classification signal from said perceptron neural network to determine the degree of certainty of the detection of said predetermined pattern.

41. An apparatus according to claim 33 wherein said object comprises a textile fabric.

42. An apparatus for analyzing a 2-D representation of an object, said apparatus comprising

at least one sensor disposed to capture a 2-D representation of at least a portion of an object;

a memory that stores at least a portion of the 2-D representation; and

a processor that derives at least one signal from said 2-D representation that generates a feature representing a characteristic fault signature of at least one signal and that provides said feature to a multi-dimensional wavelet neural network which provides a classification output indicative of whether said representation comprises a predetermined pattern;

wherein said multi-dimensional wavelet neural network comprises a plurality of wavelet neurons each defined by ψ

_a,b={square root over (diag(a)|)}ψ

(diag(a)(x−

b)

where x is a vector comprising said feature, a is a squashing matrix for that neuron and b is the translation vector for that neuron.