Method and apparatus for machine vision classification and tracking

• US 5,434,927 A
• Filed: 12/08/1993
• Issued: 07/18/1995
• Est. Priority Date: 12/08/1993
• Status: Expired due to Term

• Current Assignee: 3M Company
• Original Assignee: 3M Company
• Inventors: Cerny, Darin G., Brady, Mark J., Million, Belayneh W., Granholm, Michelle C.
• Primary Examiner(s): COUSO, YON JUNG
• Application Number:
  US08/163,820
• Time in Patent Office: 587 Days
• Field of Search: 382/22, 382/28, 382/36, 382/14, 382/1, 395/3, 395/21, 395/905, 395/61, 348/148, 348/149
• US Class Current: 382/104
• CPC Class Codes: G06V 10/443 by matching or filtering
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G08G 1/015 with provision for distinguishing between two or more types of vehicles, e.g. between motor-cars and cycles
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors
• CPC Class Codes: Y10S 706/905 Vehicle or aerospace

Abstract

A method and apparatus for classification and tracking objects in three-dimensional space is described. A machine vision system acquires images from roadway scenes and processes the images by analyzing the intensities of edge elements within the image. The system then applies fuzzy set theory to the location and angles of each pixel after the pixel intensities have been characterized by vectors. A neural network interprets the data created by the fuzzy set operators and classifies objects within the roadway scene. The system can also track objects within the roadway scene, such as vehicle, by forecasting potential track regions and then calculating match scores for each potential track region based on how well the edge elements from the target track regions match those from the source region as weighted by the extent the edge elements have moved.

Claims

1. A machine vision system comprising:

image acquisition means for acquiring images from three-dimensional space;

means for determining the magnitude of vertical and horizontal edge element intensity components of each pixel of said image;

means for converting said vertical and horizontal edge element intensity components to a first vector with a magnitude of total edge element intensity and a direction for each said pixel within said image;

means for defining regions of interest within said image;

means for fuzzifying said first vectors in said regions of interest to create a second vector characterizing each said region of interest;

a neural network for interpreting said second vector of each said region of interest to determine a classification of said object;

means for generating a signal indicative of said classification of said object; and

interface means for providing an interface with external devices.

2. The machine vision system according to claim 1 wherein said image acquisition means comprises a plurality of video cameras.

3. The machine vision system according to claim 2, further comprising switching means electrically interconnected between said plurality of video cameras and said interface means for selectively choosing one of said plurality of video cameras.

4. The machine vision system according to claim 1, wherein said image processing means further comprises traffic interpreting means for analyzing signals indicative of classification generated by said image processing means and for converting said signals to alarm and statistical information.

5. The machine vision system according to claim 1, wherein said image processing means further comprises means for mapping pixel space to real world measurements.

6. The machine vision system according to claim 4, wherein said interface means comprises:

display means for graphically displaying;

real-time video output from said image acquisition means;

said statistical information; and

said alarm information; and

input means for conveying user requests to said system.

7. The machine vision system according to claim 1, wherein said interface means comprises a traffic control device system.

8. The machine vision system according to claim 1, wherein said interface means comprises a database.

9. The machine vision system according to claim 1, further comprising digitizing means for digitizing said images acquired by said image acquisition means.

10. The machine vision system according to claim 6, wherein said image processing means is further programmed to execute the step of placing an icon on said object displayed on said display means based on said signal indicative of classification of said object.

11. A method for classifying objects in an image, comprising the computer implemented steps of:

determining the magnitude of vertical and horizontal edge element intensity components of each pixel of said image;

converting said vertical and horizontal edge element intensity components to a first vector with a magnitude of total edge element intensity and a direction for each said pixel within said image;

defining regions of interest within said image;

applying fuzzy set theory to said first vectors in regions of interest to create a second vector characterizing each said region of interest;

interpreting said second vector of each said region of interest with a neural network whereby said neural network determines a classification of said object based on said second vector; and

generating a signal indicative of said classification of said object.

12. The method for classifying objects in an image according to claim 11, further comprising the computer implemented steps of:

dividing said image into overlapping regions; and

selecting said regions of interest from said overlapping regions by selecting regions which meet predetermined criteria.

13. The method for classifying objects in an image according to claim 12, wherein said overlapping regions comprise overlapping, substantially trapezoidal regions.

14. The method for classifying objects in an image according to claim 12, wherein said image comprises a first portion and a second portion, said overlapping regions only covering said first portion of said image.

15. The method for classifying objects in an image according to claim 11, further comprising the computer implemented step of mapping said pixel space to real world measurements.

16. The method for classifying objects in an image according to claim 15, further comprising the computer implemented step of dynamically dividing said image into appropriately sized and shaped overlapping regions based on said step of mapping said pixel space.

17. The method for classifying objects according to claim 12, wherein selecting said regions of interest which meet predetermined criteria comprises the computer implemented steps of:

calculating an average total edge dement intensity of a candidate region based on the average of said magnitude of total edge element intensity of all pixels within said candidate region;

comparing said average total edge element intensity of said candidate region with a baseline average edge element intensity value; and

selecting said candidate region to be a region of interest if said total edge element intensity of said candidate region exceeds said baseline edge element intensity value.

18. The method for classifying objects according to claim 12, wherein selecting region of interest which meet predetermined criteria comprises the computer implemented steps of:

calculating an average total edge element intensity of a candidate region based on the average of said magnitude of total edge element intensity of all pixels within said candidate region; and

selecting said candidate region to be a region of interest if said average total edge element intensity of said candidate region is a local maxima.

19. The method for classifying objects in an image according to claim 11, wherein determining the magnitude of vertical and horizontal edge element intensity components of each pixel of said image comprises the computer implemented steps of:

defining a first kernel for convolution;

defining a second kernel for convolution;

applying said first kernel to each pixel within said region of interest, thereby computing said horizontal edge element intensity value;

applying said second kernel to each pixel within said region of interest, thereby computing said vertical edge element intensity value; and

assigning each pixel within said region of interest said horizontal and vertical edge element intensity values.

20. The method for classifying objects in an image according to claim 19, wherein computing said degree of horizontal and vertical edge element intensity comprises evaluating pixel intensities in an area surrounding each pixel of said plurality of pixels.

21. The method for classifying objects in an image according to claim 20, wherein evaluating pixel intensities in an area surrounding each pixel comprises determining the degree of contrast of the pixels surrounding the evaluated pixel.

22. The method for classifying objects in an image according to claim 11, wherein said image is provided by real-time video.

23. The method for classifying objects in an image according to claim 22, further comprising the computer implemented step of stabilizing said image provided by real-time video.

24. The method for classifying objects in an image according to claim 23, wherein stabilizing said image comprises the computer implemented steps of:

providing at least one reference marker in said image;

comparing said image containing said at least one reference marker to a reference image;

calculating a compensation translation and rotation for said image; and

warping said image, based on said translation and rotation calculation.

25. The method for classifying objects in an image according to claim 11, wherein applying fuzzy set theory to said first vectors in each said region of interest comprises computer implemented steps of:

applying fuzzy set theory according to the location of each said pixel in said image;

applying fuzzy set theory according to the angle of each said pixel; and

weighting the angle associated with each said pixel according to said magnitude of intensity.

26. The method for classifying objects in an image according to claim 11, further comprising the computer implemented step of segmenting said image into object related pixels and non-object related pixels.

27. The method for classifying objects in an image according to claim 25, wherein applying fuzzy set theory according to the location of each said pixel in said image comprises the computer implemented steps of:

placing overlapping tent functions over each said region of interest, said tent functions substantially pyramidal in shape and each said tent function weighting each said pixel beneath said tent function according to the height of said tent function at the location of said pixel;

apportioning said intensity of each said pixel to generalized locations corresponding to the locations of said tent functions, said apportionment according to said weighting of said pixels.

28. The method for classifying objects in an image according to claim 27, wherein each said tent function has a unit height at its vertex and wherein the sum of the heights of said overlapping tent functions at any pixel location is equal to said unit height.

29. The method for classifying objects in an image according to claim 27, wherein said step of placing overlapping tent functions over each said region of interest comprises placing nine overlapping tent functions over each said region of interest.