Method and system of detecting and recognizing a vehicle logo based on selective search

• US 10,255,691 B2
• Filed: 03/14/2017
• Issued: 04/09/2019
• Est. Priority Date: 10/20/2016
• Status: Active Grant

• Current Assignee: Guangdong Fundway Technology Co., Sun Yat-Sen University
• Original Assignee: Guangdong Fundway Technology Co., Sun Yat-Sen University
• Inventors: Li, Xiying, Lv, Shuo, Jiang, Qianyin, Luo, Donghua, Yuan, Minxian, Yu, Zhi
• Primary Examiner(s): Le, Vu
• Assistant Examiner(s): Rivera-Martinez, Guillermo M
• Application Number:
  US15/458,186
• Publication Number:
  US 20180114337A1
• Time in Patent Office: 756 Days
• CPC Class Codes: G06F 18/2136 based on sparsity criteria, e.g. with an overcomplete basis
• CPC Class Codes: G06F 18/2431 Multiple classes
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/464 using a plurality of salient features, e.g. bag-of-words [BoW] representations
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/7715 Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle

Abstract

The invention discloses a method and a system of detecting and recognizing a vehicle logo based on Selective Search, the method comprising: positioning a vehicle plate on an original image of a vehicle to obtain a vehicle plate position; coarsely positioning a vehicle logo on the original image to obtain a coarse positioning image of the vehicle logo; selecting vehicle logo candidate areas in the coarse positioning image; performing target positioning in the vehicle logo candidate areas with the Selective Search to obtain a set of target regions; training a vehicle logo location classifier with Spatial Pyramid Matching based on Sparse Coding (ScSPM) to determine the vehicle logo from the set of target regions to obtain a vehicle logo position; and training a multi-class vehicle logo recognition classifier with the ScSPM to conduct a specific type-recognition for the vehicle logo to obtain a vehicle logo recognition result.

Claims

1. A method of detecting and recognizing a vehicle logo based on Selective Search, comprising the steps of:

positioning a license plate on an original image of a vehicle to obtain a license plate position;

coarsely positioning a vehicle logo on the original image of the vehicle according to the license plate position, spatial structure between the license plate and the vehicle logo, and vehicle window edge feature, to obtain a coarse positioning image of the vehicle logo;

selecting vehicle logo candidate areas in the coarse positioning image of the vehicle logo based on a central axis of the vehicle;

performing target positioning on the vehicle logo candidate areas with a Selective Search, to obtain a set of target regions, wherein region combination performed with the Selective Search is based on color similarity, texture similarity, size similarity and fill similarity comprehensively;

training a vehicle logo location classifier with Spatial Pyramid Matching based on Sparse Coding (ScSPM), to determine the vehicle logo from the set of target regions, to obtain a vehicle logo position; and

training a multi-class vehicle logo recognition classifier with the Spatial Pyramid Matching based on Sparse Coding (ScSPM) to conduct specific type-recognition for the vehicle logo, to obtain a vehicle logo recognition result;

determining a position of a down boundary for vehicle logo coarse positioning according to a position of an up boundary for vehicle logo coarse positioning, wherein the coordinate of the down boundary, Y_down, is determined by a formula Y_down=y_up, and y_up refers to the coordinate of the up boundary;

coarsely positioning a vehicle window according to vertical projection of the original image of the vehicle, to obtain the vehicle window edge feature, and determining a position of the up boundary according to the vehicle window edge feature, the coordinate Y_up of the up boundary being determined by a formula;

2. The method of claim 1, wherein the step of selecting vehicle logo candidate areas further comprises the steps of:

determining the central axis of the vehicle on the coarse positioning image of the vehicle logo; and

selecting an area having a set width and a set height as a first vehicle logo candidate area according to the central axis of the vehicle.

3. The method of claim 2, wherein the step of performing target positioning further comprises the steps of:

performing target detection in the first vehicle logo candidate area using the Selective Search; and

determining if the target detected is a single character or a character string logo or a symbol logo.

4. The method of claim 3, wherein the step of performing target positioning further comprises the step of transversely amplifying the first vehicle logo candidate area with a transversely amplified width set to form a second vehicle logo candidate area, and then performing target detection for vehicle logo in the second vehicle logo candidate area using the Selective Search.

5. The method of claim 4, wherein the step of performing target positioning further comprises the step of abandoning the first vehicle logo candidate area and continuing to amplify the second vehicle logo candidate area with a new transversely amplified width, until a target is detected in the second vehicle logo candidate area.

6. The method of claim 3, wherein the step of performing target positioning further comprises the step of ending the Selective Search if the target detected is at least one of the character string logo or the symbol logo.

7. The method of claim 3, wherein the step of performing target positioning further comprises the steps of:

taking 1.5 times of the character height as a height of a candidate region, wherein the height of the candidate region is a height of a third vehicle logo candidate area, if the target detected is the single character; and

continuing to perform target detection in the third vehicle logo candidate area using the selectivity search.

8. The method of claim 1, wherein the step of performing target positioning further comprises the steps of:

obtaining initial target regions from the vehicle logo candidate areas using an image segmentation algorithm based on a graph theory;

calculating similarity between the adjacent regions in the initial target regions obtained, via a calculation formula;

9. The method according to claim 1, wherein the step of training a vehicle logo location classifier further comprises the steps of:

dividing sample images into positive samples and negative samples, wherein the positive samples comprises single character samples, and small-vehicle logo samples and large-vehicle logo samples in the sample set, and the negative samples are samples having intersection over union (IoU) 20% less than ground truth and having random sizes;

taking the positive samples as training samples, and using Spatial Pyramid Matching based on Sparse Coding (ScSPM) to perform training iteratively until convergence, and finally to have the vehicle logo location classifier, wherein, in the iterative training process, after each training, the samples which are wrongly detected into the negative samples in the vehicle logo location classifier is added into the training samples to form a new training sample set, then the new training sample set is used for retraining; and

determining the vehicle logo from the set of target regions according to the vehicle logo location classified trained, to obtain the position of the vehicle logo.

10. The method according to claim 3, wherein the step of training a multi-class vehicle logo recognition classifier further comprises the steps of:

taking single characters as vehicle logos, feeding the single characters of manually annotated vehicle logos and the character string vehicle logos into the Spatial Pyramid Matching based on Sparse Coding (ScSPM) to perform training iteratively;

taking the wrongly-classified vehicle logos as hard examples and feeding them into the Spatial Pyramid Matching based on Sparse Coding (ScSPM) for training until convergence, to obtain the multi-class vehicle logo recognition classifier; and

performing specific type-recognition for the vehicle logo with the multi-class vehicle logo recognition classifier by taking the result as the vehicle logo type, if the vehicle logo recognition result is a letter logo or symbol;

combining the characters obtained by positioning to form the letter logo; and

recognizing the letter logo.

11. The method according to claim 3, wherein the step of training a multi-class vehicle logo recognition classifier further comprises the steps of:

taking single characters as vehicle logos, feeding the single characters of manually annotated vehicle logos and the character string vehicle logos into the Spatial Pyramid Matching based on Sparse Coding (ScSPM) to perform training iteratively, and taking the wrongly-classified vehicle logos as hard examples and feeding them into the Spatial Pyramid Matching based on Sparse Coding (ScSPM) for training until convergence, to obtain the multi-class vehicle logo recognition classifier; and

performing specific type-recognition for the vehicle logo with the multi-class vehicle logo recognition classifier by detecting the vehicle logo in a third candidate area, if the vehicle logo recognition result is a single character;

combining the characters obtained by positioning to form the letter logo; and

recognizing the letter logo.