Method and apparatus for classifying objects in sonar images

• US 5,612,928 A
• Filed: 05/28/1992
• Issued: 03/18/1997
• Est. Priority Date: 05/28/1992
• Status: Expired due to Term

• Current Assignee: Northrop Grumman Systems Corporation (Northrop Grumman Corporation)
• Original Assignee: Northrop Grumman Corporation
• Inventors: Oravec, James J., Petlevich, Walter J. Jr., Haley, Paul H., Schweizer, Philipp F.
• Primary Examiner(s): Pihulic, Daniel T.
• Application Number:
  US07/847,512
• Time in Patent Office: 1,755 Days
• Field of Search: 367/7, 367/131, 367/87, 367/135, 367/11, 382/36, 382/14, 382/16, 382/39, 382/43, 382/54, 382/103, 382/199, 382/205, 382/291, 382/156, 358/112, 348/163
• US Class Current: 367/11
• CPC Class Codes: G06V 10/40 Extraction of image or video features

Abstract

A method and apparatus for classifying objects in images utilize means for selecting portions of those images which contain objects and means for classifying those objects based upon parameters of the selected portions, which parameters are useful for classifying the objects. The selecting means preferably is a shadow and highlight detector, a statistical window detector and a neural network window detector whose output is combined in a combined cuer. The parameters are determined from the greylevels and positions of pixels using one or more modules which perform certain mathematical operations on this data. Such parameters include edge parameters, smoothness, clutter, presence and characteristics of highlights and shadows, and texture. The invention is particularly useful for classifying objects in sonar images as natural or man-made.

Claims

1. A scene classifier for classifying objects in images consisting of a matrix of pixels, each pixel having a known greyness level comprising:

(a) snippet selection means for selecting portions of images for classification;

(b) a scene classifier connected to and receiving input from the snippet selection means, said scene classifier containing

(i) at least two parameter extraction modules selected from the group consisting of edge parameter module, smoothness module, frame parameter module, cuer detection module, highlight and shadow module and texture module, and

(ii) a Bayesian Classifier connected to and receiving input from said at least two parameter extraction modules.

2. The scene classifier of claim 1 wherein the snippet selection means is comprised of at least one detector selected from the group consisting of shadow and highlight detectors, statistical window detectors and neural network window detectors.

3. The scene classifier of claim 1 wherein the shipper selection means is comprised of at least two detectors selected from the group consisting of shadow and highlight detectors, statistical window detectors and neural network window detectors and also comprising means for combining outputs received from the detectors for transmission to the scene classifier.

4. The scene classifier of claim 1 also comprising means for displaying the images with indicia classifying objects within the image, said means being connected to the Bayesian Classifier.

5. The scene classifier of claim 4 wherein the means for displaying the images is one of a cathode ray tube, a printer and a screen.

6. The scene classifier of claim 4 wherein the indicia is comprised of at least one line encircling at least one classified object.

7. The scene classifier of claim 4 wherein the indicia is comprised of at least one word identifying at least one classified object.

8. The scene classifier of claim 1 wherein the images are sonar images and objects are classified as man-made or natural.

9. The scene classifier of claim 1 wherein the snippet selection means is comprised of a processing unit and memory for storing programs and intermediate results.

10. The scene classifier of claim 1 wherein the edge parameter module performs for each snippet at least one of edge detecting, edge thinning, searching for straight lines, recording characteristics of each straight line found, and determining straightness of the snippet.

11. The scene classifier of claim 1 wherein the smoothness module determines smoothness of a snippet by measuring pixel uniformity in selected directions using selected position patterns.

12. The scene classifier of claim 1 wherein the frame parameter module determines for each snippet at least one of clutter, image mean, image standard deviation and mean shift.

13. The scene classifier of claim 1 wherein the cuer detection module determines at least one of a quantity of object detections from the shipper selection means, areas of each shipper, average area of snippets, distances between centroids of nearby snippets, standard deviation based upon said distances, a matrix of second order moments of centroid point distribution and a ratio of eigenvalues derived from the matrix.

14. The scene classifier of claim 1 wherein the highlight and shadow module determines for each shipper at least one of highlight area, shadow area, highlight extent, shadow extent, average greylevel of highlights, average greylevel of shadows, highlight perimeter, shadow perimeter, highlight blobs, shadow blobs, shadow depth, shadow quality, shadow compactness, number of shadow blobs and area of a shadow blob.

15. The scene classifier of claim 1 wherein the texture module performs for each shipper convolution with templates to detect dots, lines and edges, computation of co-occurrence matrices, and application of Fourier transforms.

16. The scene classifier of claim 1 wherein the Bayesian Classifier contains at least one equation for determining that an object is man-made or natural.

17. A method of classifying into at least one category, objects in an image comprised of a matrix of pixels, each pixel having a known greyness level comprising the steps of:

a) defining a set of windows, each window comprised of a set of adjacent pixels such that each pixel of the image is included in at least one window;

b) classifying each window as a highlight, shadow or background according to the greyness levels of the pixels in each window;

c) selecting those windows which have been classified as containing at least one of a highlight and a shadow;

d) grouping the selected windows into snippets, each snippet containing at least one selected window;

e) determining at least one parameter which is useful for classifying objects in the image and which can be obtained from the greyness levels and position of selected pixels in the image;

f) establishing a probability of objects found in an image being within the at least one category based upon the determined parameters;

g) examining each shipper for the presence and values of the determined parameters; and

h) classifying objects within each snippet by comparing the values to the established probability.

18. The method of claim 17 also comprising the step of displaying the image with indicia that classifies objects in the image.

19. The method of claim 17 wherein the image is a sonar image and the objects are classified as natural or man-made.

20. The method of claim 17 wherein the at least one parameter is selected from the group of parameters consisting of straight line characteristics, straightness of the snippet, smoothness of the snippet, clutter, image mean, image standard deviation, mean shift, snippet area, average area of snippets, distances between centroids of nearby snippets, standard deviation based upon said distances, a ratio of eigenvalues derived from a matrix of second order moments of centroid point distribution, highlight area, shadow area, highlight extent, shadow extent, average greylevel of highlights, average greylevel of shadows, highlight perimeter, shadow perimeter, highlight blobs, shadow blobs, shadow depth, shadow quality, shadow compactness, number of shadow blobs, area of shadow blobs, and texture.