Method of determining parking lot occupancy from digital camera images

• US 9,129,524 B2
• Filed: 03/29/2012
• Issued: 09/08/2015
• Est. Priority Date: 03/29/2012
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Delibaltov, Diana L., Wu, Wencheng, Loce, Robert P., Bernal, Edgar
• Primary Examiner(s): Mehta, Bhavesh
• Assistant Examiner(s): CONNER, SEAN M
• Application Number:
  US13/433,809
• Publication Number:
  US 20130258107A1
• Time in Patent Office: 1,258 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06V 10/467 Encoded features or binary features, e.g. local binary patterns [LBP]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G06V 20/586 of parking space
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors
• CPC Class Codes: G08G 1/065 by counting the vehicles in a section of the road or in a parking area, i.e. comparing incoming count with outgoing count

Abstract

Described herein is a method of determining parking lot occupancy from digital images, including a set-up procedure that includes receiving a layout of a parking lot and estimating parking space volume for at least one viewing angle and the probability that an observed pixel belongs to the parking space volume. The method further includes acquiring one or more image frames of the parking lot from at least one digital camera; performing pixel classification using a vehicle detector on the acquired image frames to determine a likelihood that a pixel belongs to a vehicle; computing a probability that a parking space is occupied by a vehicle based on a spatially varying membership probability density function and a likelihood of vehicle pixels within a region of interest; and determining parking lot vacancy via a comparison of the computed probability that a parking space is occupied by a vehicle to a pre-determined threshold.

Claims

1. A method of determining parking lot occupancy from digital images, the method comprising:

obtaining a layout of a parking lot having a plurality of parking spaces;

estimating parking space volume for at least one viewing angle of at least one parking space and the probability that an observed pixel belongs to the parking space volume;

acquiring one or more image frames of the parking lot from at least one digital camera;

performing pixel classification using at least one vehicle detector on the acquired one or more image frames to determine a likelihood that a pixel corresponds to a vehicle;

computing a probability that the at least one parking space is occupied by a vehicle based on a weighted sum of the probability of vehicle pixels within the region of interest of each parking space, wherein the region of interest refers to pixels that can potentially belong to a vehicle parked in a given parking space, wherein the weighted sum comprises a weighting function which gives larger weight to pixels located towards a center of the region of interest of each parking space, the weighting function decreasing the weight of a particular pixel as a distance of the particular pixel to the center increases; and

determining parking lot vacancy via a comparison of the computed probability that the at least one parking space is occupied by a vehicle to a pre-determined threshold.

2. The method of claim 1, further comprising:

performing the pixel classification using at least one of a support vector machine (SVM) classifier that uses rotation-invariant local binary patterns (LBPs) as input features and a TextonBoost classifier.

3. The method of claim 2, further comprising using a model for a probability of pixel x being a vehicle, which uses a soft output of the TextonBoost classifier and a hard output of the SVM classifier, the model comprising:

P_v(x)=η

_xP_v^TB(x)+(1−

η

_x)P_v^LBP

where P_v^TB(x)ε

[0,1] is the soft probability of a pixel corresponding to a vehicle, given by the TextonBoost classifier, and P_v^LBP(x)ε

{0,1} is the hard classification of a pixel corresponding to a vehicle, given by the SVM classifier with LBP values as inputs.

4. The method of claim 2, further comprising using a model for a probability of pixel x being a vehicle, which uses a hard output of the TextonBoost classifier and a hard output of the SVM classifier, the model comprising one of:

P_v(x)=max(P_v^TB(x),P_v^LBP) or P_v(x)=min(P_v^TB(x),P_v^LBP)

where P_v^TB(x)ε

{0,1} is the hard classification of a pixel corresponding to a vehicle, given by the TextonBoost classifier, and P_v^LBP(x)ε

{0,1} is the hard classification of a pixel corresponding to a vehicle, given by the SVM classifier with LBPs as inputs.

5. The method of claim 1, further comprising:

performing the pixel classification using one or more of a support vector machine (SVM) classifier that uses rotation-invariant local binary patterns (LBPs) as input features, a TextonBoost classifier, a Histogram of Oriented Gradient (HOG) classifier, a Rotation Invariant Feature Transform (RIFT) classifier, a Generalized RIFT (G-RIFT) classifier, a Speeded Up Robust Feature (SURF) classifier, a Gradient Location and Orientation Histogram (GLOH) classifier, a SIFT (Scale Invariant Feature Transform) classifier, and/or a Harris Corners classifier.

6. The method of claim 1, wherein the pre-determined threshold is 0.5.

7. The method of claim 1, wherein the pre-determined threshold is adjusted based on additional information including at least one of the total number of vehicles entering and exiting the said parking lot and the total number of vacant parking stalls.

8. The method of claim 1, wherein obtaining the parking lot layout further comprises:

automatically detecting one or more markers or lines defining one or more parking spaces from images of the parking lot when the parking lot was at least partially empty or receiving data input by a user via a user interface, wherein the data comprises markers or lines on one or more images of the parking lot and the intersections of the markers or lines define one or more corners of parking spaces.

9. The method of claim 1, wherein estimating the volume associated with the at least one parking space in the image further comprises:

approximating the at least one parking space with a parallelepiped by using marker lines of the at least one parking space and estimating the height as approximately half of the average lengths of the four base sides;

obtaining a mask of the parallelepiped viewed from a given camera angle by geometric operations that yield a projection of a three-dimensional region of interest for the at least one parking space;

for each pixel within the region, assigning a probability of the pixel'"'"'s membership to the at least one parking space, wherein the membership probability density function is a function of the distance from a center of the region of interest and a plurality of possible overlapping regions with neighboring parking spaces.

10. A system for determining parking lot occupancy from digital images, the system comprising:

a database that stores digital images and data related to digital image processing, wherein the data includes one or more parking lot layouts; and

an image processing unit that includes a processor, a system memory, and a system bus that couples the system memory to the processing unit, wherein the image processing unit is operative to;

obtain a layout of a parking lot having a plurality of parking spaces;

estimate parking space volume for at least one viewing angle of at least one parking space and the probability that an observed pixel belongs to the parking space volume;

acquire one or more image frames of the parking lot from at least one digital camera;

perform pixel classification using at least one vehicle detector on the acquired one or more image frames to determine a likelihood that a pixel belongs to a vehicle;

compute a probability that the at least one parking space is occupied by a vehicle based on a weighted sum of the probability of vehicle pixels within the region of interest of each parking space, wherein the region of interest refers to pixels that can potentially belong to a vehicle parked in a given parking space, wherein the weighted sum comprises a weighting function which gives larger weight to pixels located towards a center of the region of interest of each parking space, the weighting function decreasing the weight of a particular pixel as a distance of the particular pixel to the center increases; and

determine parking lot vacancy via a comparison of the computed probability that the at least one parking space is occupied by a vehicle to a pre-determined threshold.

11. The system of claim 10, wherein the image processing unit is further operative to perform the pixel classification using at least one of a support vector machine classifier that uses rotation-invariant local binary patterns (LBPs) as input features and a TextonBoost classifier.

12. The system of claim 11, wherein the image processing unit is further operative to use a model for a probability of pixel x being a vehicle, which uses a soft output of the TextonBoost classifier and a hard output of the SVM classifier, the model comprising:

P_v(x)=η

_xP_v^TB(x)+(1−

η

_x)P_v^LBP

where P_v^TB(x)ε

[0,1] is the soft probability of a pixel corresponding to a vehicle, given by the TextonBoost classifier, and P_v^LBP(x)ε

{0,1} is the hard classification of a pixel corresponding to a vehicle, given by the SVM classifier with LBPs as inputs.

13. The system of claim 11, wherein the image processing unit is further operative to use a model for a probability of pixel x being a vehicle, which uses a hard output of the TextonBoost classifier and a hard output of the SVM classifier, the model comprising one of:

P_v(x)=max(P_v^TB(x),P_v^LBP) or P_v(x)=min(P_v^TB(x),P_v^LBP)

where P_v^TB(x)ε

{0,1} is the hard classification of a pixel corresponding to a vehicle, given by the TextonBoost classifier, and P_v^LBP(x)ε

{0,1} is the hard classification of a pixel corresponding to a vehicle, given by the SVM classifier with LBPs as inputs.

14. The system of claim 10, wherein the pre-determined threshold is 0.5.

15. The system of claim 10, wherein the image processing unit is further operative to adjust the pre-determined threshold based on additional information including at least one of the total number of vehicles entering and exiting the said parking lot and the total number of vacant parking stalls.

16. The system of claim 10, wherein the image processing unit is further operative to obtain the parking lot layout by:

automatically detecting one or more markers or lines defining one or more parking spaces from images of the parking lot when the parking lot was at least partially empty or receiving data input by a user via a user interface, wherein the data comprises markers or lines on one or more images of the parking lot and the intersections of the markers or lines define one or more corners of parking spaces.

17. The system of claim 10, wherein the image processing unit is further operative to estimate the volume associated with the at least one parking space in the image by:

approximating the at least one parking space with a parallelepiped by using marker lines of the parking space and estimating the height as approximately half of the average lengths of the four base sides;

obtaining a mask of the parallelepiped viewed from a given camera angle by geometric operations that yield a projection of a three-dimensional region of interest for the at least one parking space;

for each pixel within the region, assigning a probability of the pixel'"'"'s membership to the at least one parking space, wherein the membership probability density function is a function of the distance from a center of the region of interest and a plurality of possible overlapping regions with neighboring parking spaces.

18. A method of determining parking lot occupancy from digital images, the method comprising:

obtaining a layout of a parking lot having a plurality of parking spaces;

estimating parking space volume for at least one viewing angle of at least one parking space and the probability that an observed pixel belongs to the parking space volume;

acquiring one or more image frames of the parking lot from at least one digital camera;

performing pixel classification using at least one vehicle detector on the acquired one or more image frames to determine a likelihood that a pixel corresponds to a vehicle, wherein the pixel classification is performed using at least one of a support vector machine (SVM) classifier that uses rotation-invariant local binary patterns (LBPs) as input features and a TextonBoost classifier;

computing a probability that the at least one parking space is occupied by a vehicle based on a spatially varying membership probability density function and a likelihood of vehicle pixels within a region of interest;

using one of a first model and a second model to determine a probability of pixel x being a vehicle, wherein the first model uses a soft output of the TextonBoost classifier and a hard output of the SVM classifier and the second model uses a hard output of the TextonBoost classifier and a hard output of the SVM classifier,

wherein the first model comprises;

P_v(x)=η

_xP_v^TB(x)+(1−

η

_x)P_v^LBP

where P_v^TB(x)ε

[0,1] is the soft probability of a pixel corresponding to a vehicle, given by the TextonBoost classifier, and P_v^LBP(x)ε

{0,1} is the hard classification of a pixel corresponding to a vehicle, given by the SVM classifier with LBP values as inputs, and

wherein the second model comprises one of;

P_v(x)=max(P_v^TB(x),P_v^LBP) or P_v(x)=min(P_v^TB(x),P_v^LBP)

where P_v^TB(x)ε

{0,1} is the hard classification of a pixel corresponding to a vehicle, given by the TextonBoost classifier, and P_v^LBP(x)ε

{0,1} is the hard classification of a pixel corresponding to a vehicle, given by the SVM classifier with LBPs as inputs; and

determining parking lot vacancy via a comparison of the computed probability that the at least one parking space is occupied by a vehicle to a pre-determined threshold.

19. The method of claim 18, wherein the pre-determined threshold is 0.5.

20. The method of claim 18, wherein the method further comprises adjusting the pre-determined threshold based on additional information including at least one of the total number of vehicles entering and exiting the said parking lot and the total number of vacant parking stalls.

21. The method of claim 18, wherein obtaining the parking lot layout further comprises:

automatically detecting one or more markers or lines defining one or more parking spaces from images of the parking lot when the parking lot was at least partially empty or receiving data input by a user via a user interface, wherein the data comprises markers or lines on one or more images of the parking lot and the intersections of the markers or lines define one or more corners of parking spaces.

22. The method of claim 18, wherein estimating the volume associated with the at least one parking space in the image further comprises:

approximating the at least one parking space with a parallelepiped by using marker lines of the at least one parking space and estimating the height as approximately half of the average lengths of the four base sides;

obtaining a mask of the parallelepiped viewed from a given camera angle by geometric operations that yield a projection of a three-dimensional region of interest for the at least one parking space;

for each pixel within the region, assigning a probability of the pixel'"'"'s membership to the at least one parking space, wherein the membership probability density function is a function of the distance from a center of the region of interest and a plurality of possible overlapping regions with neighboring parking spaces.