3-D model based method for detecting and classifying vehicles in aerial imagery

• US 8,913,783 B2
• Filed: 10/28/2010
• Issued: 12/16/2014
• Est. Priority Date: 10/29/2009
• Status: Expired due to Fees

• Current Assignee: SRI International, Inc.
• Original Assignee: SRI International, Inc.
• Inventors: Khan, Saad Masood, Cheng, Hui, Matthies, Dennis Lee, Sawhney, Harpreet Singh, Jung, Sang-Hack, Broaddus, Chris, Matei, Bogdan Calin Mihai, Divakaran, Ajay
• Primary Examiner(s): Bitar, Nancy
• Application Number:
  US12/913,861
• Publication Number:
  US 20120106800A1
• Time in Patent Office: 1,510 Days
• Field of Search: None
• US Class Current: 382/103
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06F 18/24323 Tree-organised classifiers
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 20/182 Network patterns, e.g. roads or rivers
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats

Abstract

A computer implemented method for determining a vehicle type of a vehicle detected in an image is disclosed. An image having a detected vehicle is received. A number of vehicle models having salient feature points is projected on the detected vehicle. A first set of features derived from each of the salient feature locations of the vehicle models is compared to a second set of features derived from corresponding salient feature locations of the detected vehicle to form a set of positive match scores (p-scores) and a set of negative match scores (n-scores). The detected vehicle is classified as one of the vehicle models models based at least in part on the set of p-scores and the set of n-scores.

Claims

1. A computer implemented method for determining a vehicle type of a vehicle detected in an image, comprising the steps of:

receiving an image comprising a detected vehicle;

projecting a plurality of vehicle models comprising salient feature locations on the detected vehicle in the image, wherein each vehicle model is associated with a vehicle type;

comparing a first set of features derived from each of the salient feature locations of the vehicle models to a second set of features derived from corresponding salient feature locations of the detected vehicle to form a plurality of positive match scores (p-scores) and a plurality of negative match scores (n-scores); and

classifying the detected vehicle as one of the plurality of vehicle models by;

forming a feature vector of n-scores and p-scores;

training a set of specific vehicle-type (SVM) classifiers;

comparing the detected vehicle to each of the trained classifiers; and

associating the detected vehicle with a selected trained classifier based on a confidence value produced by the selected trained classifier.

2. The method of claim 1 wherein the feature vector is N dimensional, wherein N is a size of the plurality of vehicle models and the set of SVM classifiers contains N/2 classifiers.

18. The method of claim 1 wherein a size of the set of SVM classifiers is based on the number of different vehicle models.

19. The method of claim 1, wherein the p-score is a measure of similarity between a projected vehicle model and the detected vehicle in the image.

20. The method of claim 1, wherein the features are histogram of oriented gradients (HoG) feature descriptors and the step of comparing further comprises matching HoG features of salient feature locations obtained from a vehicle model type and corresponding salient feature locations on the detected vehicle in the image.

3. A computer implemented method for determining a vehicle type of a vehicle detected in an image, comprising the steps of:

receiving an image comprising a detected vehicle;

projecting a plurality of three-dimensional vehicle models comprising salient feature locations on the detected vehicle in the image, wherein each vehicle model is associated with a vehicle type;

comparing a first set of features derived from each of the salient feature locations of the vehicle models to a second set of features derived from corresponding salient feature locations of the detected vehicle to form a plurality of positive match scores (p-scores) and a plurality of negative match scores (n-scores); and

classifying the detected vehicle as one of the plurality of vehicle models based at least in part on the plurality of p-scores and the plurality of n-scores by;

training a set of specific vehicle-type SVM classifiers;

comparing the detected vehicle to each of the trained classifiers; and

associating the detected vehicle with a selected trained classifier based on a confidence value produced by the selected trained classifier.

4. The method of claim 3, wherein the features are histogram of oriented gradients (HoG) feature descriptors.

5. The method of claim 4, wherein the step of comparing further comprises matching HoG features of salient feature locations obtained from a vehicle model type and corresponding salient feature locations on the detected vehicle in the image.

6. The method of claim 3 wherein a size of the set of SVM classifiers is based on the number of different vehicle models.

7. The method of claim 3, wherein one or more of the salient feature locations are located at a corner or a junction of the vehicle model.

8. The method of claim 3, wherein the p-score is a measure of similarity between a projected vehicle model and the detected vehicle in the image.

9. The method of claim 3, wherein the features are histogram of oriented gradients (HoG) feature descriptors.

10. The method of claim 9, wherein the step of comparing further comprises matching HoG features of salient feature locations obtained from a vehicle model type and corresponding salient feature locations on the detected vehicle in the image.

11. The method of claim 10, wherein the step of matching further comprises the step of measuring a distance between a HoG feature of a salient feature location obtained from the vehicle model type and corresponding salient feature location of the detected vehicle in the image.

12. The method of claim 3, wherein the n-score is a measure of patterns of mismatch between rendered appearance of one of the plurality of vehicle models and an appearance of the detected vehicle in the image.

13. An apparatus for determining a vehicle type of a vehicle detected in an image, comprising:

an image capture device for receiving an image comprising a detected vehicle; and

a digital processing system configured for;

projecting a plurality of three-dimensional vehicle models comprising salient feature locations on the detected vehicle in the image, wherein each vehicle model is associated with a vehicle type;

comparing a first set of features derived from each of the salient feature locations of the vehicle models to a second set of features derived from corresponding salient feature locations of the detected vehicle to form a plurality of positive match scores (p-scores) and a plurality of negative match scores (n-scores); and

classifying the detected vehicle as one of the plurality of vehicle models based at least in part on the plurality of p-scores and the plurality of n-scores by;

training a set of specific vehicle-type SVM classifiers;

comparing the detected vehicle to each of the trained classifiers; and

associating the detected vehicle with a selected trained classifier based on a confidence value produced by the selected trained classifier.

14. The apparatus of claim 13, wherein the features are histogram of oriented gradients (HoG) feature descriptors.

15. The apparatus of claim 14, wherein the step of comparing further comprises matching HoG features of salient feature locations obtained from a vehicle model type and corresponding salient feature locations on the detected vehicle in the image.

16. The apparatus of claim 13 wherein a size of the set of SVM classifiers is based on the number of different vehicle models.

17. The apparatus of claim 13, wherein the p-score is a measure of similarity between a projected vehicle model and the detected vehicle in the image.