DYNAMIC BAYESIAN NETWORKS FOR VEHICLE CLASSIFICATION IN VIDEO

• US 20140085475A1
• Filed: 05/21/2012
• Published: 03/27/2014
• Est. Priority Date: 05/19/2011
• Status: Active Application

• Current Assignee: Regents of the University of California (University of California)
• Original Assignee: Regents of the University of California (University of California)
• Inventors: Bhanu, Bir, Kafai, Mehran
• Granted Patent: US 9,466,000 B2
• Time in Patent Office: Days
• US Class Current: 348/148
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights

Abstract

A system and method for classification of passenger vehicles and measuring their properties, and more particularly to a stochastic multi-class vehicle classification system, which classifies a vehicle (given its direct rear-side view) into one of four classes Sedan, Pickup truck, SUV/Minivan, and unknown, and wherein a feature pool of tail light and vehicle dimensions is extracted which feeds a feature selection algorithm to define a low-dimensional feature vector, and the feature vector is then processed by a Hybrid Dynamic Bayesian Network (HDBN) to classify each vehicle.

Claims

1. A method for vehicle classification comprising:

performing a Bayesian network analysis for vehicle classifications, which are known, wherein the Bayesian network is defined as a directed acyclic graph G=(V, E), where nodes represent random variables from a domain of interest and arcs symbolize direct dependencies between the random variables.

2. The method of claim 1, wherein for a Bayesian network with n nodes X₁, X₂ . . . X_n, a full joint distribution is defined as:

3. The method of claim 2, further comprising using a K2 algorithm to determine a sub-optimal structure.

4. The method of claim 3, comprising:

generating a manually structured network and comparing the sub-optimal structure to the manually structured network.

5. The method of claim 4, comprising:

adding a temporal dimension to the Bayesian network to create a Dynamic Bayesian Network (DBN structure).

6. The method of claim 5, comprising:

replicating the Bayesian network with time-dependent random variables over T time slices, and wherein a new set of arcs defining the transition model is used to determine how various random variables are related between time slices.

7. The method of claim 6, wherein the DBN structure is defined as following:

for each time slice t_{i, i=1, 2, . . . , 5},

each feature X_i^t is the parent of X_i^t+1;

C^t is the parent of C^t+1; and

all intra slice dependencies also hold as inter time slices except for arcs from time slice t hidden nodes to time slice t+1 observed nodes.

8. The method of claim 7, comprising:

generating a Hybrid Dynamic Bayesian Network (HDBN), which consists of discrete and continuous nodes, and wherein parameters of the HDBN are required before classification is performed.

9. The method of claim 8, wherein the classifications include Sedan, Pickup, and SUV or Minivan.

10. The method of claim 1, comprising:

generating a complete system framework, which includes data collection, feature extraction, feature selection, and classification.

11. The method of claim 10, comprising:

extracting features from images of a rear view of a vehicle.

12. The method of claim 11, comprising:

extracting tail light features, wherein the tail light features include separately for each tail light a width, a distance from the license plate, and an angle between the tail light and the license plate.

13. The method of claim 12, comprising:

extracting license plate features, the license plate features including the license plate location and a size thereof is used as a reference to enable comparison and normalize the tail light features and the vehicle size values.

14. The method of claim 1, comprising:

detecting three subcomponents, wherein the three subcomponents are vehicle detection, license plate extraction, and tail light extraction.

15. The method of claim 14, comprising:

using a Gaussian mixture model approach for detection of a moving object.

16. The method of claim 15, wherein the Gaussian mixture model comprises Gaussian distributions to determine if a pixel is more likely to belong to a background model or not, and an AND approach, which determines a pixel as background only if the pixel falls within three standard deviations for all the components in all three R, G, and B color channels.

17. The method of claim 16, comprising:

validating detected moving objects by using a simple frame differencing approach.

18. The method of claim 17, comprising:

removing shadows and erroneous pixels by finding a vertical axis of symmetry using an accelerated version of Loy'"'"'s symmetry and readjusting a bounding box containing a mask with respect to an axis of symmetry.

19. The method of claim 18, wherein if the shadow is behind the vehicle, removing the shadow using geometrical assumptions such as camera location, object geometry, and ground surface geometry, and wherein given the vehicle rear mask, measuring a height and width of the bounding box, and area of the mask.

20. The method of claim 19, comprising:

inputting the license plate corner coordinates into an algorithm for license plate extraction, and adding a Gaussian noise with constant mean 0 and variance 0.2 time width to the license plate width measurement.

21. The method of claim 20, comprising:

performing a tail light extraction by fusing two methods, a first method wherein an image is converted to hue, saturation and value (HSV) color space and classifying pixels into three main color groups red, green, and blue, and a second method, which defines the red level of each pixel as

22. The method of claim 21, comprising:

generating a bounding box surrounding each tail light by combining results from both methods and checking if the regions with high redness can be a tail light.

23. The method of claim 22, wherein if the vehicle body color is red itself, estimating a vehicle color using a hue, saturation and value (HSV) color space histogram analysis approach, which determines if the vehicle is red or not, and if a red vehicle is detected, the tail light detection component is enhanced by adding an extra level of post-processing which includes thresholding, color segmentation, removing large and small regions, and symmetry analysis, and after the tail lights are detected, separately computing width, centroid, and distance and angle with the license plate for both left and right tail lights.

24. The method of claim 23, comprising:

extracting one or more features for each image frame, and normalizing each distance with respect to the license plate width, the one or more features comprising;

perpendicular distance from license plate centroid to a line connecting two tail light centroids, right tail light width, left tail light width, right tail light license plate angle, left tail light license plate angle, right tail light license plate distance, left tail light license plate distance, bounding box width, bounding box height, license plate distance to bounding box bottom side, and/or vehicle mask area.

25. The method of claim 24, comprising:

optimizing an evaluation criterion to improve classification accuracy, shorten computational time, reduce measurements costs, and/or eliminate use of dimensionality.

26. The method of claim 25, comprising:

performing a Sequential Floating Forward Selection (SFFS), which is a deterministic statistical pattern recognition (SPR) feature selection method and returning a single suboptimal solution.

27. The method of claim 26, wherein SFFS starts from an empty set and adding features that increase accuracy and removing the least significant features during a conditional exclusion.

28. The method of claim 27, comprising:

a stopping condition, which halts the SFFS algorithm to limit the number of feature selection iterative steps to 2^n-1, wherein n is the number of features and defining a correct classification rate (CCR) threshold of b %, where b is greater than the CCR of the case when all features are used.

29. The method of claim 28, wherein the classification is a two stage approach, which consists of estimating the Gaussian distribution parameters of the distance to the nearest neighbor for all vehicles in the training dataset and determining if a vehicle test case is known or unknown by initially computing the distance to its nearest neighbor.

30. The method of claim 29, comprising:

if the distance does not lie within 4 standard deviations of a mean (μ

±

4σ

), classifying it as unknown, and if the vehicle is classified as known it is a candidate for a second stage of classification.

31. A stochastic multi-class vehicle classification system, which classifies a vehicle given its direct rear-side view into one of four classes Sedan, Pickup truck, SUV/Minivan, and unknown, the system comprising:

extracting a feature pool of low-level tail light and vehicle dimension features;

feeding the features into a selection algorithm to define a low-dimensional feature vector; and

processing the feature vector with a Hybrid Dynamic Bayesian Network (HDBN) to classify each vehicle.

32. A system for classification of vehicles comprising:

a camera configured to capture images of at least one moving object; and

a computer processing unit configured to perform a Bayesian network analysis for vehicle classifications, which are known, wherein the Bayesian network is defined as a directed acyclic graph G=(V, E), where nodes represent random variables from a domain of interest and arcs symbolize direct dependencies between random variables.

33. The system of claim 32, wherein the computer processing unit further comprises:

a memory arrangement, a processing unit and an optional display unit configured to display data and/or classification of the distinct structures in the class.

34. The system of claim 32, wherein the camera is configured to capture video images.

35. A computer program product comprising a non-transitory computer usable medium having a computer readable code embodied therein for classification of passenger vehicles and measuring their properties from a rear view video frame, the computer readable program code is configured to execute a process, which performs a Bayesian network analysis for vehicle classifications, which are known, wherein the Bayesian network is defined as a directed acyclic graph G=(V, E), where nodes represent random variables from a domain of interest and arcs symbolize direct dependencies between random variables.