Recognition of patterns in images

• US 5,313,532 A
• Filed: 10/28/1991
• Issued: 05/17/1994
• Est. Priority Date: 01/23/1990
• Status: Expired due to Fees

• Current Assignee: Massachusetts Institute of Technology
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Heinemann, Karl G., DiCaprio, Paul N., Harvey, Robert L.
• Primary Examiner(s): Razavi, Michael T.
• Assistant Examiner(s): KLOCINSKI, STEVEN
• Application Number:
  US07/784,634
• Time in Patent Office: 932 Days
• Field of Search: 382/21, 382/22, 382/6, 382/36, 382/48, 382/14, 382/15, 964/413.1
• US Class Current: 382/156
• CPC Class Codes: G01N 15/1433 using image recognition
• CPC Class Codes: G01N 15/1468 with spatial resolution of the texture or inner structure of the particle
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G06V 20/69 Microscopic objects, e.g. biological cells or cellular parts

Abstract

A pattern (e.g., the normal or abnormal characteristics of a biological cell) within an image is recognized based on visual characteristics of the pattern, the image being represented by signals whose values correspond to the visual characteristics, using a location channel which determines the location of the pattern within the image based on the signal values, and a classification channel which categorizes the pattern based on the signal values, the location channel and the classification channel operating in parallel and cooperatively to recognize the pattern. In other aspects, there the orientations of edges of the pattern within subwindows of the image are analyzed as are the strengths of edges of the pattern near the periphery of portions of the image; an unsupervised classifier defines internal representation classes of objects, and a supervised classifier maps the classes to user-specified categories; and there is a feedback path from the classification channel to the location channel to cause the location channel to adapt to classification results generated by the classification channel.

Claims

1. Apparatus for recognizing, as one of at least two possible patterns, a pattern within an image based on visual characteristics of the pattern, said image being represented by signals whose values correspond to said visual characteristics, comprising

a location channel which determines the location of the pattern within the image based on the signal values, and

a classification channel which categorizes the pattern based on the signal values and generates an indication of the pattern'"'"'s object identity from among a set of possible identities, said classification channel comprising

addition means for summing said signal values to generate an indication of the overall size of said pattern,

orientation means for sensing edges of said pattern and generating indications of orientations of said edges, and

an unsupervised classifier which processes a classification spectrum comprised of said size indication and said orientation indications are based on said processing, automatically selects and associates characteristics from the spectrum to define categories,

said location channel and said classification channel operating simultaneously in parallel and cooperatively to recognize said pattern as said one of said patterns.

2. The apparatus of claim 1 wherein said location channel comprises

a coarse locator which makes a coarse determination of the existence and location of the pattern within the image, and

a fine locator, responsive to the coarse, which makes a fine determination of the location of the pattern within the image.

3. The apparatus of claim 2 wherein said coarse locator comprises a neural network which processes said image with interconnecting traces whose weights are chosen based on general shape of interest.

4. The apparatus of claim 3 wherein said coarse locator operates with respect to a field of view within said image and a feedback path from said classification channel to said location channel controls the position of the field of view within the image.

5. The apparatus of claim 2 wherein said fine locator includes circuitry for responding to feedback from said classification channel in order to adjust the position of a field of view within said image in order to determine the fine location of a pattern within the image.

6. The apparatus of claim 4 wherein said coarse locator provides a feedforward signal to said fine locator to control the fine position of said field of view.

7. The apparatus of claim 1 wherein said classification channel further comprises

calibration means for calibrating said signal values so that said values are evenly distributed across a predefined range of values before being processed by said addition means and said orientation means, and

a supervised classifier which associates categories defined by said unsupervised classifier with user-specified object identities.

8. The apparatus of claim 1 wherein said orientation means is adapted to generate measures of the strengths of edges in predetermined orientations within portions of said image.

9. The apparatus of claim 1 wherein said orientation means is adapted to generate indications of orientations of edges at the periphery of a portion of said image.

10. The apparatus of claim 8 wherein said predetermined orientations include vertical, horizontal, and 45°

.

11. The apparatus of claim 9 wherein said orientation means is adapted to generate measures of the existence of said edges at the top, bottom, and each side of the image.

12. The apparatus of claim 1 wherein said indications which correspond to coarser features appear in the lower end of said spectrum and measures which correspond to finer features appear in the upper end of said spectrum.

13. The apparatus of claim 7 wherein said classification channel includes a feedback path for providing said classification spectrum information to said location channel.

14. Apparatus for recognizing a pattern within an image based on visual characteristics of said pattern, said pattern having edges, said image being represented by signals whose values correspond to said visual characteristics, comprising

an orientation analyzer adapted to analyze the orientations of edges of the pattern within subwindows of said image,

a strength analyzer adapted to analyze the strengths of edges of the pattern near the periphery of a portion of said image,

a classifier for processing the outputs of said orientation and strength analyzers as part of a spectrum, and

a mapper for causing outputs of subwindows of said image to be treated in said spectrum in an order such that outputs of subwindows nearer to the center of the image are treated as appearing higher on the spectrum than outputs of subwindows near the periphery of the image.

15. The apparatus of claim 14 wherein said orientation analyzer comprises detectors for detecting the strengths of orientation of edges in four different possible orientations.

16. The apparatus of claim 15 wherein said four different possible orientations comprises 0, 45, 90, and 135 degrees, respectively.

17. The apparatus of claim 15 wherein each said analyzer comprises neural networks.

18. The apparatus of claim 15 wherein said strength analyzer comprises

an averaging module for averaging elements of a subwindow to derive an averaged subwindow and

four neural networks for processing said averaged subwindow to determine the strength of edges at the north, south, east, and west peripheries of said subwindow.

19. Apparatus for recognizing, as one of at least two possible patterns, a pattern within an image based on visual characteristics of the pattern, said image being represented by signals whose values correspond to said visual characteristics, comprising

a location channel which determines the location of the pattern within the image based on the signal values, and

a classification channel which categorizes the pattern based on the signal values and generates an indication of the pattern'"'"'s object identity from among a set of possible identities, said classification channel comprising

addition means for summing said signal values to generate an indication of the overall size of said pattern,

orientation means for sensing edges of said pattern and generating indications of orientations of said edges, and

an unsupervised classifier which processes a classification spectrum comprised of said size indication and said orientation indications are based on said processing, automatically selects and associates characteristics from the spectrum to define categories, and

a feedback path from said classification channel to said location channel to cause said location channel to adapt to classification results generated by said classification channel,

said location channel and said classification channel operating simultaneously in parallel and cooperatively to recognize said pattern as said one of said patterns.

20. Apparatus for recognizing, as one of at least two health conditions, the health condition of a biological cell within an image based on visual characteristics of the cell, said image being represented by signals whose values correspond to said visual characteristics, comprising

a location channel which determines the location of the cell within the image based on the signal values, and

a classification channel which categorizes the cell based on the signal values and generates an indication of the cell'"'"'s object health conditions from among a set of possible health conditions, said classification channel comprising

addition means for summing said signal values to generate an indication of the overall size of said cell,

orientation means for sensing edges of said cell and generating indications of orientations of said edges, and

an unsupervised classifier which processes a classification spectrum comprised of said size indication and said orientation indications and based on said processing, automatically selects and associates characteristics from the spectrum to define health condition categories,

said location channel and said classification channel operating simultaneously in parallel and cooperatively to recognize the health condition of said cell as said one of said health conditions.

21. The apparatus of claim 20 wherein said classification channel further comprises storage for said classification spectrum information about the visual characteristics of said cell for use in categorizing said health condition of said cell.

22. The apparatus of claim 20 wherein said location channel comprises

a coarse locator which makes a coarse determination of the existence and location of the cell within the image, and

a fine locator, responsive to the coarse, which makes a fine determination of the location of the cell within the image.

23. The apparatus of claim 22 wherein said coarse locator comprises a neural network which processes said image with interconnecting traces whose weights are chosen based on cell shapes of interest.

24. The apparatus of claim 23 wherein said coarse locator operates with respect to a field of view within said image and a feedback path from said classification channel to said location channel controls the position of the field of view within the image.

25. The apparatus of claim 24 wherein said coarse locator provides a feedforward signal to said fine locator to control the fine position of said field of view.

26. The apparatus of claim 22 wherein said fine locator includes circuitry for responding to feedback from said classification channel in order to adjust the position of a field of view within said image in order to determine the fine location of the cell within the image.

27. The apparatus of claim 20 wherein said classification channel further comprises

calibration means for calibrating said signal values so that said values are evenly distributed across a predefined range of values before being processed by said addition means and said orientation means, and

a supervised classifier which associates health condition categories defined by said unsupervised classifier with user-specified health conditions.

28. The apparatus of claim 27 wherein, said classification channel includes a feedback path for providing said classification spectrum information to said location channel.

29. The apparatus of claim 20 wherein said orientation means is adapted to generate indications of orientations of edges at the periphery of a portion of said image.

30. Apparatus for recognizing the health condition of a biological cell within an image based on visual characteristics of said cell, said cell having visible edges, said image being represented by signals whose values correspond to said visual characteristics, comprising

an orientation analyzer adapted to analyze the orientations of edges of the cell within subwindows of said image,

a strength analyzer adapted to analyze the strengths of edges of the cell near the periphery of a portion of said image,

a classifier for processing the outputs of said orientation and strength analyzers as part of a spectrum, and

a mapper for causing outputs of subwindows of said image to be treated in said spectrum in an order such that outputs of subwindows nearer to the center of the image are treated as appearing higher on the spectrum than outputs of subwindows near the periphery of the image.

31. The apparatus of claim 30 wherein said orientation analyzer comprises detectors for detecting the strengths of orientation of edges in four different possible orientations.

32. The apparatus of claim 31 wherein each said analyzer comprises neural networks.

33. The apparatus of claim 31 wherein said strength analyzer comprises

an averaging module for averaging elements of a subwindow to derive an averaged subwindow and

four neural networks for processing said averaged subwindow to determine the strength of edges at the north, south, east, and west peripheries of said subwindow.

34. Apparatus for recognizing, as one of at least two health conditions, the health condition of a biological cell within an image based on visual characteristics of the cell, said image being represented by signals whose values correspond to said visual characteristics, comprising

a location channel which determines the location of the cell within the image based on the signal values,

a classification channel which categorizes the cell based on the signal values and generates an indication of the cell'"'"'s health condition from among a set of possible health conditions, said classification channel comprising

addition means for summing said signal values to generate an indication of the overall size of said cell,

orientation means for sensing edges of said cell and generating indications of orientations of said edges, and

an unsupervised classifier which processes a classification spectrum comprised of said size indication and said orientation indications and based on said processing, automatically selects and associates characteristics from the spectrum to define health condition categories, and

a feedback path from said classification channel to said location channel to cause said location channel to adapt to classification results generated by said classification channel,

said location channel and said classification channel operating simultaneously in parallel and cooperatively to recognize the health condition of said cell as said one of said health conditions.