Passenger vehicle make and model recognition system

• US 10,223,609 B2
• Filed: 10/03/2016
• Issued: 03/05/2019
• Est. Priority Date: 10/02/2015
• Status: Active Grant

• Current Assignee: Regents of the University of California (University of California)
• Original Assignee: Regents of the University of California (University of California)
• Inventors: Bhanu, Bir, Thakoor, Ninad
• Primary Examiner(s): Yentrapati, Avinash
• Application Number:
  US15/283,869
• Publication Number:
  US 20170097944A1
• Time in Patent Office: 883 Days
• CPC Class Codes: G06V 10/32 Normalisation of the pattern dimensions
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/754 involving a deformation of the sample pattern or of the reference pattern; Elastic matching
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats

Abstract

A system, a method, and a non-transitory computer readable recording medium for recognizing a passenger vehicle make and model are disclosed, which includes a make and model recognition (MMR) database, which is constructed from rear view videos of passenger vehicles.

Claims

1. A method for recognizing a make and a model of passenger vehicles, the method comprising:

constructing a make and model recognition (MMR) database from rear view images of passenger vehicles;

generating a universal view for each type of passenger vehicle (MMRDB) from the make and model recognition database by averaging images from the make and model recognition database;

generating a database of universal views of the passenger vehicles from the generated universal view for the each type of passenger vehicle (“

UVaMMRDB”

);

generating a database of showcase views of the passenger vehicles, wherein each showcase view represents a visually distinct type of passenger vehicle within the generated database of universal views of the passenger vehicles (“

SVaMMRDB”

);

detecting a region of interest (ROI) of a passenger vehicle for classification, and aligning landmarks in the ROI with the database of universal views of the passenger vehicles to obtain a universal view aligned ROI (“

UVaROI”

);

comparing the UVaROI to the UVaMMRDB for the ROI to find nearest neighbors in the MMRDB, and based on the nearest neighbors, assigning a type of passenger vehicle to the ROI;

aligning the landmarks in the ROI with the database of showcase views of the passenger vehicles to obtain a showcase view aligned ROI (“

SVaROI”

); and

comparing the SVaROI to the SVaMMRDB and assigning the make and the model of the passenger vehicle to the ROI.

3. The method of claim 1, further comprising:

receiving a video steam of images of the passenger vehicle; and

generating the ROI from the video stream of images of the passenger vehicle.

4. The method of claim 1, further comprising:

performing illumination normalization on the detected ROI of the passenger vehicle.

5. The method of claim 1, further comprising:

iteratively improving the universal view by aligning the MMR database to a current universal view and averaging the aligned universal images.

6. The method of claim 1, further comprising:

generating the database of showcase views of the passenger vehicles by averaging images of a particular vehicle type in the MMR database; and

iteratively improving the database of showcase views by aligning the images of the particular vehicle type with a current showcase view and averaging the aligned images.

7. The method of claim 1, wherein the alignment of the ROI to obtain the universal view aligned ROI comprises a two-step alignment process of first aligning rows and secondly aligning columns;

and

wherein the alignment steps are interchangeable.

2. A non-transitory computer readable recording medium stored with a computer readable program code for recognizing a make and model of passenger vehicles, the computer readable program code configured to execute a process comprising:

constructing a make and model recognition (MMR) database from rear view images of passenger vehicles;

generating a universal view for each type of passenger vehicle (MMRDB) from the make and model recognition database by averaging images from the make and model recognition database;

generating a database of universal views of the passenger vehicles from the generated universal view for the each type of passenger vehicle (“

UVaMMRDB”

);

generating a database of showcase views of the passenger vehicles, wherein each showcase view represents a visually distinct type of passenger vehicle within the generated database of universal views of the passenger vehicles (“

SVaMMRDB”

);

detecting a region of interest (ROI) of a passenger vehicle for classification, and aligning landmarks in the ROI with the database of universal views of the passenger vehicles to obtain a universal view aligned ROI (“

UVaROI”

);

comparing the UVaROI to the UVaMMRDB for the ROI to find nearest neighbors in the MMRDB, and based on the nearest neighbors, assigning a type of passenger vehicle to the ROI;

aligning the landmarks in the ROI with the database of showcase views of the passenger vehicles to obtain a showcase view aligned ROI (“

SVaROI”

); and

comparing the SVaROI to the SVaMMRDB and assigning the make and the model of the passenger vehicle to the ROI.

8. A method for recognizing a make and a model of passenger vehicles, the method comprising:

detecting a region of interest (ROI) of a passenger vehicle for classification, and aligning landmarks in the ROI with the database of universal views of the passenger vehicles to obtain a universal view aligned ROI (“

UVaROI”

);

comparing the UVaROI to a database of universal views of passenger vehicles (“

UVaMMRDB”

) for the ROI to find nearest neighbors in a database of universal views for passenger vehicles (MMRDB);

assigning a type of passenger vehicle to the ROI based on the nearest neighbors;

aligning landmarks in the ROI with the database of showcase views of the passenger vehicles to obtain a showcase view aligned ROI (“

SVaROI”

); and

comparing the SVaROI to a database of showcase views of the passenger vehicles (SVaMMRDB), wherein each showcase view represents a visually distinct type of passenger vehicle within the generated database of universal views of the passenger vehicles and assigning the make and the model of the passenger vehicle to the ROI.

9. The method of claim 8, further comprising:

receiving a video steam of images of the passenger vehicle;

generating the ROI from the video stream of images of the passenger vehicle; and

performing illumination normalization on the detected ROI of the passenger vehicle.

10. The method of claim 8, wherein the images of the passenger vehicles are rear view images, side view images, or front view images.

11. A system for recognizing a make and a model of passenger vehicles, the system comprising:

a computer processor configured to;

construct a make and model recognition (MMR) database from images of passenger vehicles;

generate a universal view for each type of passenger vehicle (MMRDB) from the make and model recognition database by averaging images from the make and model recognition database;

generate a database of universal views of the passenger vehicles from the generated universal view for the each type of passenger vehicle (“

UVaMMRDB”

);

generate a database of showcase views of the passenger vehicles, wherein each showcase view represents a visually distinct type of passenger vehicle within the generated database of universal views of the passenger vehicles (“

SVaMMRDB”

);

detect a region of interest (ROI) of a passenger vehicle for classification, and aligning landmarks in the ROI with the database of universal views of the passenger vehicles to obtain a universal view aligned ROI (“

UVaROI”

);

compare the UVaROI to the UVaMMRDB for the ROI to find nearest neighbors in the MMRDB, and based on the nearest neighbors, assigning a type of passenger vehicle to the ROI;

align landmarks in the ROI with the database of showcase views of the passenger vehicles to obtain a showcase view aligned ROI (“

SVaROI”

); and

compare the SVaROI to the SVaMMRDB and assign the make and the model of the passenger vehicle to the ROI.

12. The system of claim 11, wherein the images of the passenger vehicles are rear view images.

13. The system of claim 11, wherein the computer processor is configured to:

receive a video steam of images of the passenger vehicle; and

generate the ROI from the video stream of images of the passenger vehicle.

14. The system of claim 11, wherein the computer processor is configured to:

perform illumination normalization on the detected ROI of the passenger vehicle.

15. The system of claim 11, wherein the computer processor is configured to:

iteratively improve the universal view by aligning the MMR database to a current universal view and averaging the aligned universal images.

16. The system of claim 11, wherein the computer processor is configured to:

generate the database of showcase views of the passenger vehicles by averaging images of a particular vehicle type in the MMR database.

17. The system of claim 16, wherein the computer processor is configured to:

iteratively improve the database of showcase views by aligning the images of the particular vehicle type with a current showcase view and averaging the aligned images.

18. The system of claim 11, wherein the alignment of the ROI to obtain the universal view aligned ROI comprises a two-step alignment process of first aligning rows and secondly aligning columns;

and

wherein the alignment steps are interchangeable.

19. The system of claim 11, wherein the computer processor is configured to:

identify the make and model of the passenger vehicle on a road;

identify the make and the model of the passenger vehicle in a parking garage or parking lot;

identify the make and the model of the passenger vehicle for vehicle fingerprinting and security related applications ensuring limited/controlled access to vehicles; and

/or

identify the make and the model of the passenger vehicle in conjunction with automatic license plate recognition and a vehicle registration database to avoid license plate spoofing.

20. The system of claim 11, wherein the computer processor is configured to:

generate the make and model recognition (MMR) database from front and/or side images of passenger vehicles.