Vehicle detection method and system including irrelevant window elimination and/or window score degradation

• US 9,514,366 B2
• Filed: 02/03/2014
• Issued: 12/06/2016
• Est. Priority Date: 02/03/2014
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Schweid, Stuart Alan, Bulan, Orhan, Wang, Yao Rong, Wu, Wencheng, Loce, Robert Paul
• Primary Examiner(s): Elahi, Shan
• Application Number:
  US14/171,022
• Publication Number:
  US 20150222859A1
• Time in Patent Office: 1,037 Days
• Field of Search: 348/148
• US Class Current: 1/1
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles

Abstract

This disclosure provides vehicle detection methods and systems including irrelevant search window elimination and/or window score degradation. According to one exemplary embodiment, provided is a method of detecting one or more parked vehicles in a video frame, wherein candidate search windows are limited to one or more predefined window shapes. According to another exemplary embodiment, the method includes degrading a classification score of a candidate search window based on aspect ratio, window overlap area and/or a global maximal classification.

Claims

1. A computer implemented method of detecting one or more vehicles in a video frame acquired from a fixed parking occupancy video camera including a field of view associated with a vehicle parking region, the computer implemented method comprising the steps of:

a) capturing a video frame from the fixed parking occupancy video camera, the video frame including a ROI (Region of Interest) oriented by an orientation angle relative to an orientation of an image plane associated with the captured video frame, the ROI including one or more parking spaces of the vehicle parking region;

b) performing a sliding window-based search for one or more vehicles within the ROI, the sliding window-based search extracting one or more features associated with each of a plurality of candidate search windows representing a set of windows from which one or more mutually exclusive object bounding boxes are selected, wherein each selected mutually exclusive object bounding box is associated with a vehicle, and wherein the vehicle is not associated with any of the other mutually exclusive object bounding boxes;

c) accessing a classifier to classify each candidate search window as including a vehicle or not including a vehicle, wherein the classifier generates a score for each candidate window based on one or more features associated with each candidate search window, the score of each candidate search window degraded as a function of at least one of a fixed or an adaptive aspect ratio associated with each candidate search window; and

d) suppressing one or more overlapping classified candidate search windows including a common vehicle having a classification score below a predetermined threshold, wherein overlapping candidate search windows are eliminated from detecting the common vehicle,

wherein the classified candidate search windows are limited to candidate search windows of one or more predefined window shapes to exclude search windows limited to a partial detection of a vehicle, and candidate search windows which are not suppressed are considered to be the one or more mutually exclusive object bounding boxes representative of the one or more vehicles detected in the video frame.

2. The computer implemented method of detecting one or more vehicles in a video frame according to claim 1, wherein the candidate search windows are limited to windows associated with one or more of a predefined aspect ratio, length, height or surrogate pixel width.

3. The computer implemented method of detecting one or more vehicles in a video frame according g to claim 1, wherein the classifier is one of a neural network classifier, a K-nearest neighbor classifier, a linear SVM (Support Vector Machine) classifier and a non-linear SVM classifier.

4. The computer implemented method of detecting one or more vehicles in a video frame according to claim 1, the method comprising:

performing steps a) - d) for a plurality of captured video frames.

5. The computer implemented method of detecting one or more vehicles in a video frame according to claim 1, wherein step b) performs the sliding window-based search along a longitudinal axis associated with the ROI, at various window widths.

6. The computer implemented method of detecting one or more vehicles in a video frame according to claim 1, wherein step d) performs NMS (Non-Maximum Suppression) to suppress one or more overlapping classified candidate search windows including a common vehicle.

7. The computer implemented method of detecting one or more vehicles in a video frame according to claim 1, wherein the classified candidate search windows are limited to candidate search windows of one or more predefined window shape(s) after step c) and before step d).

8. The computer implemented method of detecting one or more vehicles in a video frame according to claim 1, wherein the classified candidate search windows are limited to candidate search windows of one or more predefined window shape(s) before step c).

9. The computer implemented method of detecting one or more vehicles in a video frame according to claim 1, wherein the classifier is a SVM classifier, and wherein the candidate search window is eliminated as a function of either a fixed or adaption aspect ratio associated with each candidate search window.

10. The computer implemented method of detecting one or more vehicles in a video frame according to claim 9, wherein the fixed or adaptive aspect ratio is associated with typical dimensions of vehicles detected within the ROI.

11. A computer implemented method of detecting one or more vehicles in a video frame acquired from a fixed parking occupancy video camera including a field of view associated with a vehicle parking region, the method comprising:

a) capturing a video frame from the fixed parking occupancy video, the video frame including a ROI (Region of Interest) oriented by an orientation angle relative to an orientation of an image plane associated with the captured video frame, the ROI including one or more parking spaces of the vehicle parking region;

b) performing a sliding window-based search for one or more vehicles within the ROI, the sliding window-based search extracting one or more features associated with each of a plurality of candidate search windows representing a set of windows from which object bounding boxes are selected, wherein each selected object bounding box is associated with a vehicle, and wherein the vehicle is not associated with any of the other object bounding boxes;

c1) accessing a classifier to score each candidate search window with a classification score calculated by the classifier indicating a probability of candidate search window includes a vehicle relative to a plurality of training images used to train the classifier;

c2) degrading the classification score of one or more classified overlapping search windows including a common vehicle, the classification score degraded by an amount that is a function of an overlap area of the overlapping search windows and a relative size of each of the overlapping windows; and

d) performing a NMS (Non-Maximal Suppression) process to suppress any overlapping classified candidate search window with a classification score below a predetermined threshold,

wherein classified candidate search windows which are not suppressed are considered to be object bounding boxes representative of the one or more vehicles detected in the video frame.

12. The computer implemented method of detecting one or more vehicles in a video frame according to claim 11, wherein step c2) degrades the classification score of one or more classified overlapping search windows to significantly decrease the scores of a classified overlapping search window with significant reciprocal overlap with a respective maximal score classified search window, and step c2) does not significantly degrade the scores of a classified overlapping search window with a minimal reciprocal overlap with a respective maximal score classified search window.

13. The computer implemented method of detecting one or more vehicles in a video frame according to claim 11, wherein step c2) degrades the classification score of one or more classified overlapping windows including a common vehicle, the classification score degraded by an amount that is a function of an overlap area of area of the overlapping search windows, a relative size of each of the overlapping windows, and a global maximal classification score associated with the classified candidate search windows.

14. The computer implemented method of detecting one or more vehicles in a video frame according to claim 11, wherein step c2) degrades the classification score of one or more classified overlapping windows according to the equation

score_window1:

=score_window1−

k×

(score_max−

T+M)×

ƒ

{area(window1∩

w_max)/area (window1)}

where

score_window1 is a score of a window to be modified,

T is a threshold for vehicle detection,

M is a classification margin,

ƒ

{area (window1∩

□

w_max)/area (window1)} is a weighting function related to the area of overlap of window1 and w_max, and

k is a weighting factor related to the strength of the degradation.

15. The computer implemented method of detecting one or more vehicles in a video frame according to claim 14, wherein the weighting function ƒ

{area (window1∩

□

w_max)/area(window1)} is a monotonically increasing function where the degradation increases with increasing overlap and there is no score reduction for windows that are mutually exclusive with the global maximum score window.

16. The computer implemented method of detecting one or more vehicles in a video frame according to claim 14, wherein the weighting function is one of a raised cosine, a cubic spline and a sigmoid function.

17. The computer implemented method of detecting one or more vehicles in a video frame according to claim 11, wherein the classifier is one of a neural network classifier, a K-nearest neighbor classifier, a linear SVM (Support Vector Machine) classifier and a non-linear SVM classifier.

18. The computer implemented method of detecting one or more vehicles in a video frame according to claim 11, the method comprising:

performing steps a) - d) for a plurality of captured video frames.

19. The computer implemented method of detecting one or more vehicles in a video frame according to claim 11, wherein step b) performs the sliding window-based search along a longitudinal axis associated with the ROI, at various widths.

20. The computer implemented method of detecting one or more vehicles in a video frame according to claim 11, wherein steps c2) and d) are performed iteratively to generate one or more mutually exclusive object bounding boxes representative of the one or more vehicles detected in the video frame, where classified candidate search windows which overlap and are not suppressed are merged to generate a mutually exclusive object abounding box.

21. A vehicle detection system associated with a vehicle parking region, the vehicle detection system comprising:

a parking occupancy video camera directed towards the vehicle parking region; and

a controller operatively associated with the parking occupancy video camera, the controller configured to execute computer instructions to perform a process of detecting a vehicle in a video frame including;

a) capturing a video frame from the fixed parking occupancy video, the video frame including a ROI (Region of Interest) oriented by an orientation angle relative to an orientation of an image plane associated with the captured video frame, the ROI including one or more parking spaces of the vehicle parking region;

b) performing a sliding window-based search for one or more vehicles within the ROI, the sliding window-based search extracting one or more features associated with each of a plurality of candidate search windows representing a set of windows from which object bounding boxes are selected, wherein each selected object bounding box is associated with a vehicle, and wherein the vehicle is not associated with any of the other object bounding boxes;

c1) accessing a classifier to score each candidate search window with a classification score calculated by the classifier indicating a probability of candidate search window includes a vehicle relative to a plurality of training images used to train the classifier;

c2) degrading the classification score of one or more classified overlapping search windows including a common vehicle, the classification score degraded by an amount that is a function of an overlap area of the overlapping search windows and a relative size of each of the overlapping windows; and

d) performing a NMS (Non-Maximal Suppression) process to suppress any overlapping classified candidate search window with a classification score below a predetermined threshold,

wherein classified candidate search windows which are not suppressed are considered to be object bounding boxes representative of the one or more vehicles detected in the video frame.

22. The vehicle detection system according to claim 21, wherein step c2) degrades the classification score of one or more classified overlapping search windows to significantly decrease the scores of a classified overlapping search window with significant reciprocal overlap with a respective maximal score classified search window, and step c2) does not significantly degrade the scores of a classified overlapping search window with a minimal reciprocal overlap with a respective maximal score classified search window.

23. The vehicle detection system according to claim 21, wherein step c2) degrades the classification score of one or more classified overlapping windows including a common vehicle, the classification score degraded by an amount that is a function of an overlap area of area of the overlapping search windows, a relative size of each of the overlapping windows, and a global maximal classification score associated with the classified candidate search windows.

24. The vehicle detection system according to claim 21, wherein step c2) degrades the classification score of one or more classified overlapping windows according to the equation

score_window1:

=score_window1−

k×

(score_max−

T+M)×

ƒ

{area(window1∩

□

w_max)/area (window1)}

where

score_window1 is a score of a window to be modified,

T is a threshold for vehicle detection,

M is a classification margin,

ƒ

{area (window1∩

□

w_max)/area(window1)} is a weighting function related to the area of overlap of window1 and w_max, and

k is a weighting factor related to the strength of the degradation.

25. The vehicle detection system according to claim 24, wherein the weighting function ƒ

{area(window1∩

□

w_max)/area(window1)} is a monotonically increasing function where the degradation increases with increasing overlap and there is no score reduction for windows that are mutually exclusive with the global maximum score window.