Method and apparatus for machine vision classification and tracking

• US 5,761,326 A
• Filed: 04/27/1995
• Issued: 06/02/1998
• Est. Priority Date: 12/08/1993
• Status: Expired due to Term

• Current Assignee: 3M Company
• Original Assignee: 3M Company
• Inventors: Cerny, Darin G., Granholm, Michelle C., Brady, Mark J., Million, Belayneh W.
• Primary Examiner(s): Johns, Andrew W.
• Assistant Examiner(s): Davis, Monica S.
• Application Number:
  US08/429,943
• Time in Patent Office: 1,132 Days
• Field of Search: 382/103, 382/104, 382/160, 382/199, 348/148, 395/905, 395/21, 340/937
• US Class Current: 382/103
• CPC Class Codes: G06V 10/443 by matching or filtering
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G08G 1/015 with provision for distinguishing between two or more types of vehicles, e.g. between motor-cars and cycles
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors
• CPC Class Codes: Y10S 706/905 Vehicle or aerospace

Abstract

A method and apparatus for classification and tracking objects in three-dimensional space is described. A machine vision system acquires images from roadway scenes and processes the images by analyzing the intensities of edge elements within the image. The system then applies fuzzy set theory to the location and angles of each pixel after the pixel intensities have been characterized by vectors. A neural network interprets the data created by the fuzzy set operators and classifies objects within the roadway scene. The system can also track objects within the roadway scene, such as vehicles, by forecasting potential track regions and then calculating match scores for each potential track region based on how well the edge elements from the target track regions match those from the source region as weighted by the extent the edge elements have moved.

Claims

1. A machine vision system for tracking movement of objects, comprising:

image acquisition means for acquiring images from three-dimensional space;

processing means connected to receive said images, for processing said images to track an object having a first track point in a first image, said processing means comprising;

means for determining a magnitude of vertical and horizontal edge element intensity components of each pixel of said first image from a first time;

means for determining a magnitude of vertical and horizontal edge element intensity components of each pixel of a second image from a second time;

means for computing a magnitude of total edge element intensity for each said pixel within said first image and for each said pixel within said second image;

means for forecasting a plurality of potential subsequent track points in said second image;

means for determining which one of the plurality of potential subsequent track points corresponds to the movement of the object by assigning a value to each said potential subsequent track point, said value being determined by comparing said magnitudes of total edge element intensity from a first region surrounding said first track point and with corresponding pixels in a second region surrounding each said potential subsequent track point and;

means for selecting one of said plurality of potential subsequent track points as said subsequent track point based on said values assigned.

2. The machine vision system of claim 1, wherein said processing means further comprises:

means for computing a first vector with a magnitude of total edge element intensity and angle for each said pixel within said first image;

means for applying fuzzy set theory to said first vectors in regions of interest to create a second vector characterizing each said region of interest;

a neural network for interpreting said second vector of each said region of interest to determine a classification and a location of said object based on said second vector; and

means for generating a second signal indicative of said location of said object, said location constituting said first track point.

3. The machine vision system according to claim 1, wherein said image acquisition means comprises a plurality of video cameras.

4. The machine vision system according to claim 3, further comprising traffic interpreting means for automatically selecting one of said plurality of video cameras when said signal indicative of said subsequent track point of said object conforms to predetermined criteria.

5. The machine vision system according to claim 3, wherein said processing means further comprises means for prioritizing said signals indicative of said subsequent track points from each of said plurality of video cameras and automatically selecting the video camera with the highest priority.

6. The machine vision system according to claim 1, wherein said image processing means further comprises means for placing an icon on said object displayed on said display means based on said signal indicative of said subsequent track point.

7. The machine vision system according to claim 1, further comprising transmission means electrically interconnected to said image acquisition means for transmitting images acquired by said image acquisition means.

8. The machine vision system according to claim 1, wherein said means for assigning a value to each said potential subsequent track point further comprises weighting means for assigning a weight to each said pixel based on evidence of motion.

9. The machine vision system according to claim 8, wherein said weighting means comprises:

means for creating a difference image having difference pixels by determining the difference between said magnitudes of total edge element intensity between corresponding pixels within said first image and said second image;

means for identifying pixels in said difference image corresponding to said first region;

means for identifying pixels in said difference image corresponding to said second region; and

means for producing said weights based on a comparison of said difference pixels in said difference image corresponding to said first region with said difference pixels in said difference image corresponding to said second region.

10. The machine vision system according to claim 9, wherein said weighting means further comprises means for applying a shrinking function to said difference pixels in said difference image corresponding to said first region at said first time.

11. The machine vision system according to claim 1, wherein said processing means further comprises means for applying a shrinking function to said pixels corresponding to said first region.

12. The machine vision system according to claim 1, wherein said means for forecasting a plurality of potential subsequent track points further comprises intermittent tracking means for tracking intermittently when said objects have not moved significantly.

13. The machine vision system according to claim 12, wherein said intermittent tracking means comprises:

means for creating a difference image having difference pixels by determining the difference between said magnitudes of total edge element intensity in said first region in said second image with said magnitudes of total edge element intensity in said first region in said second image;

means for summing values of said difference pixels to produce a summed difference;

means for comparing said summed difference with a threshold value; and

means for generating a signal indicating that tracking of said object is necessary for said second time when said summed difference exceeds said threshold value.

14. The machine vision system according to claim 1, wherein said means for forecasting a plurality of potential subsequent track points further comprises means for tracking said inertial history of said object for predicting next potential locations of said objects.

15. The machine vision system according to claim 1, further comprising means for determining a centroid of said object in said first image.

16. The machine vision system according to claim 15, further comprising means for repositioning said first track point over said centroid.

17. The machine vision system according to claim 15, further comprising means for adjusting the size of said first region surrounding said first track point based on said centroid of said object.