3-D MODEL BASED METHOD FOR DETECTING AND CLASSIFYING VEHICLES IN AERIAL IMAGERY

• US 20120106800A1
• Filed: 10/28/2010
• Published: 05/03/2012
• Est. Priority Date: 10/29/2009
• Status: Active Grant

• Current Assignee: SRI International, Inc.
• Original Assignee: SRI International, Inc.
• Inventors: Cheng, Hui, Matthies, Dennis Lee, Sawhney, Harpreet Singh, Jung, Sang-Hack, Matei, Bogdan Calin Mihai, Divakaran, Ajay, Khan, Saad Masood, Broaddus, Chris
• Granted Patent: US 8,913,783 B2
• Time in Patent Office: Days
• US Class Current: 382/104
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06F 18/24323 Tree-organised classifiers
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 20/182 Network patterns, e.g. roads or rivers
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats

Abstract

A computer implemented method for determining a vehicle type of a vehicle detected in an image is disclosed. An image having a detected vehicle is received. A number of vehicle models having salient feature points is projected on the detected vehicle. A first set of features derived from each of the salient feature locations of the vehicle models is compared to a second set of features derived from corresponding salient feature locations of the detected vehicle to form a set of positive match scores (p-scores) and a set of negative match scores (n-scores). The detected vehicle is classified as one of the vehicle models models based at least in part on the set of p-scores and the set of n-scores.

Claims

1. A computer implemented method for determining a vehicle type of a vehicle detected in an image, comprising the steps of:

receiving an image comprising a detected vehicle;

projecting a plurality of vehicle models comprising salient feature locations on the detected vehicle, wherein each vehicle model is associated with a vehicle type;

comparing a first set of features derived from each of the salient feature locations of the vehicle models to a second set of features derived from corresponding salient feature locations of the detected vehicle to form a plurality of positive match scores (p-scores) and a plurality of negative match scores (n-scores); and

classifying the detected vehicle as one of the plurality of vehicle models based at least in part on the plurality of p-scores and the plurality of n-scores.

2. The method of claim 1, wherein one or more of the salient feature locations are located at a corner or a junction of the vehicle model.

3. The method of claim 1, wherein the p-score is a measure of similarity between a projected vehicle model and the detected vehicle in the image.

4. The method of claim 1, wherein the descriptors are histogram of oriented gradients (HoG) feature descriptors.

5. The method of claim 4, wherein the step of comparing further comprises matching HoG features of salient feature locations obtained from a vehicle model type and corresponding salient feature locations on the detected vehicle in the image.

6. The method of claim 5, wherein the step of matching further comprises the step of measuring a distance between a HoG feature of a salient feature location obtained from the vehicle model type and corresponding salient feature location of the detected vehicle in the image.

7. The method of claim 6, wherein the distance is a Euclidean distance between two HoG features.

8. The method of claim 5, further comprising repeating the step of matching HoG features for a remainder of the vehicle model types to form a salient feature match distribution matrix (SMD).

9. The method of claim 8, wherein the p-score is derived from the SMD.

10. The method of claim 8, wherein the p-score is a weighted mean of distances in a row of the SMD corresponding to a salient feature for each of the compared model types.

11. The method of claim 1, wherein the n-score is a measure of patterns of mismatch between rendered appearance of one of the plurality of vehicle models and an appearance of the detected vehicle in the image.

12. The method of claim 8, further comprising the step of performing a model-to-model match study in which patterns of inter-model differences are learned to form a plurality of model-to-model mismatch distributions.

13. The method of claim 12, further comprising the step of forming the n-score for each of the plurality of vehicle model types based on an RMS distance between model-to-model mismatch distributions and a canonical distribution of mismatches derived from the SMD.

14. The method of claim wherein the step of classifying the detected vehicle further comprises the steps of:

forming an N dimensional feature vector of n-scores and p-scores, wherein N is a size of the predetermined plurality of vehicle models,

training an N/2 set of specific vehicle-type SVM classifiers,

comparing the detected vehicle to each of the N/2 trained classifiers, and,

associating the detected vehicle with a trained classifier which produces the highest confidence value.

15. A method for detecting a presence and location of a vehicle part in at least one image, comprising the steps of:

training a multi-class classifier based on a plurality of predetermined landmarks corresponding to portions of a plurality of images of vehicle models in a plurality of canonical poses;

selecting a region of interest (ROI) in the at least one image;

for each pixel in the ROI, computing a set of descriptors corresponding to at least one image scale;

processing each of the set of descriptors with the multi-class classifier to obtain a plurality of likelihood scores;

summing the plurality of likelihood scores to produce a set of likelihood image maps each containing a probability value for having a particular vehicle part at a particular pixel location in the ROI; and

determining a particular vehicle part is located at a particular pixel location of the ROI if the probability value associated with a likelihood image map is greater than or equal to a threshold value.

16. The method of claim 15, wherein the multiclass classifier is a random forest of a plurality of random tree classifiers.

17. The method of claim 16, each of the random tree classifiers is a binary support vector machine (SVM).

18. The method of claim 15, further comprising the step of, for each landmark and for each canonical pose, collecting positive samples of landmark regions and negative samples surrounding the landmark regions to form positive and negative feature vectors, respectively.

19. The method of claim 18, wherein the positive and negative feature vectors are used to train the multi-class classifier.

20. The method of claim 19, wherein the negative feature vectors include background negative samples of random images that do not contain vehicles.

21. The method of claim 15, wherein each of the set of descriptors is a HoG descriptor.

22. A method for determining a pose of a vehicle detected in at least one image, comprising the steps of

selecting a plurality of landmarks corresponding to a plurality of images of vehicle models in a plurality of canonical poses;

for each canonical pose;

sampling random configurations of vehicle parts,

applying vehicle parts relationship constraints to the random configuration of, vehicle parts, and

fitting a two-dimensional (2D) deformable model of a vehicle to the random and constrained configuration of parts;

computing a plurality of poses of the vehicle based on a plurality of likelihood scores obtained from the fitted 2D deformable model; and

selecting a pose of the vehicle corresponding to a highest likelihood score.

23. The method of claim 22, wherein the plurality of likelihood scores are computed using Oriented Chamfer distances between the fitted model and edges in the at least one image.