Image-based vehicle classification system

• US 10,572,741 B2
• Filed: 04/19/2018
• Issued: 02/25/2020
• Est. Priority Date: 08/17/2017
• Status: Active Grant

• Current Assignee: National Applied Research Laboratories (Government of The Republic of China)
• Original Assignee: National Applied Research Laboratories (Government of The Republic of China)
• Inventors: Huang, Chung-I, Tseng, Chien-Hao, Lin, Fang-Pang
• Primary Examiner(s): Couso, Jose L
• Application Number:
  US15/957,473
• Publication Number:
  US 20190057260A1
• Time in Patent Office: 677 Days
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06V 10/26 Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
• CPC Class Codes: G06V 10/34 Smoothing or thinning of the pattern; Morphological operations; Skeletonisation
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles

Abstract

An image-based vehicle classification system includes a camera and an image server connected to the camera. The camera captures images of a road to result in an image stream. The image server includes a processor for receiving the image stream from the camera. For each of the images of the image stream, the processor performs image segmentation, a thinning process, an erosion process and a dilation process, and classifies, by a neural network classifier, a vehicle image part contained in the image into one of a large-size car class, a passenger car class and a motorcycle class when it is determined that the vehicle image is crossing an imaginary line set in advance in the image for counting vehicles.

Claims

1. An image-based vehicle classification system, comprising:

a camera configured to capture a series of images of a road to result in an image stream and to transmit the image stream; and

an image server electrically connected to said camera, and including

a communication interface configured to receive the image stream from said camera and to transmit the image stream, and

a processor electrically connected to said communication interface for receiving the image stream from said communication interface, and configured to, for each of the images of the image stream,

perform image segmentation on the image so as to result in a background portion, and a foreground portion that includes a plurality of vehicle image parts which respectively correspond to a plurality of vehicles,

perform a thinning process on the foreground portion to result in a thinned foreground portion,

perform an erosion process on the thinned foreground portion to remove at least one connection line between any overlapping two of the vehicle image parts so as to result in an eroded foreground portion where the vehicle image parts are separated from each other,

perform a dilation process on the vehicle image parts of the eroded foreground portion to result in a dilated foreground portion,

determine whether one of the vehicle image parts is crossing an imaginary line set in advance in the image for counting vehicles, and

classify, by a neural network classifier when it is determined that one of the vehicle image parts is crossing the imaginary line, the one of the vehicle image parts into one of a large-size car class, a passenger car class and a motorcycle class.

2. The image-based vehicle classification system as claimed in claim 1, wherein:

said camera is configured to transmit the image stream wirelessly based on mobile communication technology; and

said communication interface of said image server is configured to receive the image stream from said camera based on the mobile communication technology.

3. The image-based vehicle classification system as claimed in claim 1, wherein:

said processor of said image server is configured to perform at least one of a background subtraction technique, a background reconstruction technique or a moving object tracking technique on each of the images of the image stream so as to result in the background portion and the foreground portion for the image.

4. The image-based vehicle classification system as claimed in claim 1, wherein:

said processor of said image server is configured to, for each of the images of the image stream, perform statistical automatic thresholding algorithm so as to determine an image binarization threshold, based on which image binarization is performed on the image, as image segmentation is performed on the image.

5. The image-based vehicle classification system as claimed in claim 1, wherein the neural network classifier is a convolutional neural network (CNN).

6. The image-based vehicle classification system as claimed in claim 1, wherein the neural network classifier is a backpropagation neural network (BPN).

7. The image-based vehicle classification system as claimed in claim 6, wherein:

the BPN includes an input layer including three input parameters, an output layer including three output parameters, and at least one hidden layer between the input layer and the output layer, the input parameters including a width of the vehicle image part, a height of the vehicle image part, and an area of the vehicle image part that is the product of the width and the height of the vehicle image part, the output parameters including the large-size car class, the passenger car class and the motorcycle class.

8. The image-based vehicle classification system as claimed in claim 7, wherein said at least one hidden layer includes ten neurons.

9. The image-based vehicle classification system as claimed in claim 1, wherein said processor is further configured to, after classifying the one of the vehicle image parts into one of the large-size car class, the passenger car class and the motorcycle class, add one to a count of vehicle image parts which have been classified as said one of the large-size car class, the passenger car class and the motorcycle class.

10. The image-based vehicle classification system as claimed in claim 9, further comprising:

a display electrically connected to said processor, and configured to display the image stream, and the counts of vehicle image parts belonging to the large-size car class, the passenger car class and the motorcycle class.

11. An image-based vehicle classification method to be implemented by a system that includes a camera and an image server, the image server including a communication interface and a processor, the image-based vehicle classification method comprising steps of:

by the camera, capturing a series of images of a road to result in an image stream, and transmitting the image stream;

by the communication interface, receiving the image stream from the camera and transmitting the image stream;

receiving, by the processor, the image stream from the communication interface; and

for each of the images of the image stream, by the processor,

performing image segmentation on the image so as to result in a background portion, and a foreground portion that includes a plurality of vehicle image parts which respectively correspond to a plurality of vehicles,

performing a thinning process on the foreground portion to result in a thinned foreground portion,

performing an erosion process on the thinned foreground portion to remove at least one connection line between any overlapping two of the vehicle image parts so as to result in an eroded foreground portion where the vehicle image parts are separated from each other,

performing a dilation process on the vehicle image parts of the eroded foreground portion to result in a dilated foreground portion,

determining whether one of the vehicle image parts is crossing an imaginary line set in advance in the image for counting vehicles, and

classifying, by using a neural network classifier when it is determined that one of the vehicle image parts is crossing the imaginary line, the one of the vehicle image parts into one of a large-size car class, a passenger car class and a motorcycle class.

12. The image-based vehicle classification method as claimed in claim 11, wherein:

the transmitting the image stream includes wirelessly transmitting, by the camera, the image stream based on mobile communication technology; and

the receiving the image stream includes receiving, by the communication interface of the image server, the image stream from the camera based on the mobile communication technology.

13. The image-based vehicle classification method as claimed in claim 11, wherein:

the performing image segmentation includes performing, by the processor of the image server, at least one of a background subtraction technique, a background reconstruction technique or a moving object tracking technique on the image so as to result in the background portion and the foreground portion.

14. The image-based vehicle classification method as claimed in claim 11, wherein:

the performing image segmentation includes performing, by the processor of the image server, statistical automatic thresholding algorithm so as to determine an image binarization threshold, based on which image binarization is performed on the image during performing image segmentation on the image.

15. The image-based vehicle classification method as claimed in claim 11, wherein the neural network classifier is a convolutional neural network (CNN).

16. The image-based vehicle classification method as claimed in claim 11, wherein the neural network classifier is a backpropagation neural network (BPN).

17. The image-based vehicle classification method as claimed in claim 16, wherein:

the BPN includes an input layer including three input parameters, an output layer including three output parameters, and at least one hidden layer between the input layer and the output layer, the input parameters including a width of the vehicle image part, a height of the vehicle image part, and an area of the vehicle image part that is the product of the width and the height of the vehicle image part, the output parameters including the large-size car class, the passenger car class and the motorcycle class.

18. The image-based vehicle classification method as claimed in claim 17, wherein said at least one hidden layer includes ten neurons.

19. The image-based vehicle classification method as claimed in claim 11, further comprising, after classifying the one of the vehicle image parts into one of the large-size car class, the passenger car class and the motorcycle class, a step of:

adding one to a count of vehicle image parts belonging to said one of the large-size car class, the passenger car class and the motorcycle class.

20. The image-based vehicle classification method as claimed in claim 19, further comprising:

displaying, by a display electrically connected to the processor, the image stream, and the counts of the vehicle images belonging to the large-size car class, the passenger car class and the motorcycle class.