METHOD OF DETERMINING PARKING LOT OCCUPANCY FROM DIGITAL CAMERA IMAGES

• US 20130258107A1
• Filed: 03/29/2012
• Published: 10/03/2013
• Est. Priority Date: 03/29/2012
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Wu, Wencheng, Loce, Robert P., Bernal, Edgar, Delibaltov, Diana L.
• Granted Patent: US 9,129,524 B2
• Time in Patent Office: Days
• US Class Current: 348/148
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06V 10/467 Encoded features or binary features, e.g. local binary patterns [LBP]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G06V 20/586 of parking space
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors
• CPC Class Codes: G08G 1/065 by counting the vehicles in a section of the road or in a parking area, i.e. comparing incoming count with outgoing count

Abstract

Described herein is a method of determining parking lot occupancy from digital images, including a set-up procedure that includes receiving a layout of a parking lot and estimating parking space volume for at least one viewing angle and the probability that an observed pixel belongs to the parking space volume. The method further includes acquiring one or more image frames of the parking lot from at least one digital camera; performing pixel classification using a vehicle detector on the acquired image frames to determine a likelihood that a pixel belongs to a vehicle; computing a probability that a parking space is occupied by a vehicle based on a spatially varying membership probability density function and a likelihood of vehicle pixels within a region of interest; and determining parking lot vacancy via a comparison of the computed probability that a parking space is occupied by a vehicle to a pre-determined threshold.

Claims

1. A method of determining parking lot occupancy from digital images, the method comprising:

obtaining a layout of a parking lot having a plurality of parking spaces;

estimating a volume of a parking space for at least one viewing angle of the parking space and the probability that an observed pixel belongs to the parking space volume;

acquiring one or more image frames of the parking lot from at east one digital camera;

performing pixel classification using at least one vehicle detector on the acquired one or more image frames to determine a likelihood that a pixel corresponds to a vehicle;

computing a probability that a parking space is occupied by a vehicle based on a spatially varying membership probability density function and a likelihood of vehicle pixels within a region of interest; and

determining parking lot vacancy via a comparison of the computed probability that a parking space is occupied by a vehicle to a pre-determined threshold.

2. The method of claim 1, further comprising:

performing the pixel classification step using at least one of a support vector machine (SVM) classifier that uses rotation-invariant local binary patterns (LBPs) as input features and a TextonBoost classifier.

3. The method of claim 2, further comprising using a model for a probability of pixel x being a vehicle, which uses a soft output of the TextonBoost classifier and a hard output of the SVM classifier, the model comprising:

P_v(x)=η

_x P_v^T(x)+(1−

η

_n)P_v^LBP

where P_v^TB(x) ∈

[0,1] is the soft probability of a pixel corresponding to a vehicle, given by the TextonBoost classifier, and P_v^LBP(x) ∈

{0,1} is the hard classification of a pixel corresponding to a vehicle, given by the SVM classifier with LBP values as inputs.

4. The method of claim 2, further comprising using a model for a probability of pixel x being a vehicle, which uses a hard output of the TextonBoost classifier and a hard output of the SVM classifier, the model comprising one of:

P_v(x)=max (P_v^TB(x), P_v^LBP) or P_v(x)=min (P_v^TB(x), P_v^LBP)

where P_v^TB (x) ∈

{0,1} is the hard classification of a pixel corresponding to a vehicle, given by the TextonBoost classifier, and P_v^LBP(x) ∈

{0,1} is the hard classification of a pixel corresponding to a vehicle, given by the SVM classifier with LBPs as inputs

5. The method of claim 1, further comprising:

performing the pixel classification step using one or more of a support vector machine (SVM) classifier that uses rotation-invariant local binary patterns (LBPs) as input features, a TextonBoost classifier, a Histogram of Oriented Gradient (HOG) classifier, a Rotation Invariant Feature Transform (RIFT) classifier, a Generalized RIFT (G-RIFT) classifier, a Speeded Up Robust Feature (SURF) classifier, a Gradient Location and Orientation Histogram (GLOH) classifier, a SIFT (Scale Invariant Feature Transform) classifier, and/or a Harris Corners classifier.

6. The method of claim 1, wherein the pre-determined threshold is 0.5.

7. The method of claim 1, wherein the pre-determined threshold is adjusted based on additional information including at least one of the total number of vehicles entering and exiting the said parking lot and the total number of vacant parking stalls.

8. The method of claim 1, wherein obtaining the parking lot layout further comprises:

automatically detecting one or more markers or lines defining one or more parking spaces from images of the parking lot when the parking lot was at least partially empty or receiving data input by a user via a user interface, wherein the data comprises markers or lines on one or more images of the parking lot and the intersections of the markers or lines define one or more corners of parking spaces.

9. The method of claim 1, wherein estimating the volume associated with a parking space in the image further comprises:

approximating the parking space with a parallelepiped by using the marker lines of the parking space and estimating the height as approximately half of the average lengths of the four base sides;

obtaining a mask of the parallelepiped viewed from a given camera angle by geometric operations that yield a projection of a three-dimensional region of interest for the given parking space;

for each pixel within the region, assigning a probability of the pixel'"'"'s membership to the given parking space, wherein the membership probability density function is a function of the distance from a center of the region of interest and a plurality of possible overlapping regions with neighboring parking spaces.

10. A system for determining parking lot occupancy from digital images, the system comprising:

a database that stores digital images and data related to digital image processing, wherein the data includes one or more parking lot layouts; and

an image processing unit that includes a processor, a system memory, and a system bus that couples the system memory to the processing unit, wherein the image processing unit is operative to;

obtain a layout of a parking lot having a plurality of parking spaces;

estimate a volume of a parking space for at least one viewing angle of the parking space and the probability that an observed pixel belongs to the parking space volume;

acquire one or more image frames of the parking lot from at least one digital camera;

perform pixel classification using at least one vehicle detector on the acquired one or more image frames to determine a likelihood that a pixel belongs to a vehicle;

compute a probability that a parking space is occupied by a vehicle based on a spatially varying membership probability density function and a likelihood of vehicle pixels within a region of interest; and

determine parking lot vacancy via a comparison of the computed probability that a parking space is occupied by a vehicle to a pre-determined threshold.

11. The system of claim 10, wherein the image processing unit is further operative to perform the pixel classification step using at least one of a support vector machine classifier that uses rotation-invariant local binary patterns (LBPs) as input features and a TextonBoost classifier.

12. The system of claim 11, wherein the image processing unit is further operative to use a model for a probability of pixel x being a vehicle, which uses a soft output of the TextonBoost classifier and a hard output of the SVM classifier, the model comprising:

P_v(x)=η

_x P_v^TB(x)+(1−

η

_x)P_v^LBP

where P_v^TB(x) ∈

[0,1] is the soft probability of a pixel corresponding to a vehicle, given by the TextonBoost classifier, and P_v^LBP(x) ∈

{0,1} is the hard classification of a pixel corresponding to a vehicle, given by the SVM classifier with LBPs as inputs.

13. The system of claim 11, wherein the image processing unit is further operative to use a model for a probability of pixel x being a vehicle, which uses a hard output of the TextonBoost classifier and a hard output of the SVM classifier, the model comprising one of:

P_v(x)=max (P_v^TB(x), P_v^LBP) or P_v(x)=min (P_v^TB(x), P_v^LBP)

where P_v^TB(x) ∈

{0,1} is the hard classification of a pixel corresponding to a vehicle, given by the TextonBoost classifier, and P_v^LBP(x) ∈

{0,1} is the hard classification of a pixel corresponding to a vehicle, given by the SVM classifier with LBPs as inputs.

14. The system of claim 10, wherein the pre-determined threshold is 0.5.

15. The system of claim 10, wherein the image processing unit is further operative to adjust the pre-determined threshold based on additional information including at least one of the total number of vehicles entering and exiting the said parking lot and the total number of vacant parking stalls.

16. The system of claim 10, wherein the image processing unit is further operative to obtain the parking lot layout by:

automatically detecting one or more markers or lines defining one or more parking spaces from images of the parking lot when the parking lot was at least partially empty or receiving data input by a user via a user interface, wherein the data comprises markers or lines on one or more images of the parking lot and the intersections of the markers or lines define one or more corners of parking spaces.

17. The system of claim 10, wherein the image processing unit is further operative to estimate the volume associated with a parking space in the image by:

approximating the parking space with a parallelepiped by using the marker lines of the parking space and estimating the height as approximately half of the average lengths of the four base sides;

obtaining a mask of the parallelepiped viewed from a given camera angle by geometric operations that yield a projection of a three-dimensional region of interest for the given parking space;

for each pixel within the region, assigning a probability of the pixel'"'"'s membership to the given parking space, wherein the membership probability density function is a function of the distance from a center of the region of interest and a plurality of possible overlapping regions with neighboring parking spaces.

18. A non-transitory computer-usable data carrier storing instructions that, when executed by a computer, cause the computer to perform a method comprising:

obtaining a layout of a parking lot having a plurality of parking spaces;

estimating a volume of a parking space for at least one viewing angle of the parking space and the probability that an observed pixel belongs to the parking space volume;

acquiring one or more image frames of the parking lot from at least one digital camera;

performing pixel classification using at least one vehicle detector on the acquired one or more image frames to determine a likelihood that a pixel corresponds to a vehicle;

computing a probability that a parking space is occupied by a vehicle based on a spatially varying membership probability density function and a likelihood of vehicle pixels within a region of interest; and

determining parking lot vacancy via a comparison of the computed probability that a parking space is occupied by a vehicle to a pre-determined threshold.

19. The non-transitory computer-usable data carrier of claim 18, wherein the method further comprises:

performing the pixel classification step using at least one of a support vector machine (SVM) classifier that uses rotation-invariant local binary patterns (LBPs) as input features and a TextonBoost classifier.

20. The non-transitory computer-usable data carrier of claim 19, wherein the method further comprises:

using a model for a probability of pixel x being a vehicle, which uses a soft output of the TextonBoost classifier and a hard output of the SVM classifier, the model comprising;

P_v(x)=η

_xP_v^TB(x)+(1−

η

_x)P_v^LBP

where P_v^TB(x) ∈

[0,1] is the soft probability of a pixel corresponding to a vehicle, given by the TextonBoost classifier, and P_v^LBP(x) ∈

{0,1} is the hard classification of a pixel corresponding to a vehicle, given by the SVM classifier with LBP values as inputs.

21. The non-transitory computer-usable data carrier of claim 19, wherein the method further comprises:

using a model for a probability of pixel x being a vehicle, which uses a hard output of the TextonBoost classifier and a hard output of the SVM classifier, the model comprising one of;

P_v(x)=max (P_v^TB(x), P_v^LBP) or P_v (x) =min (P_v^TB(x), P_v^LBP)

where P_v^TB(x) ∈

{0,1} is the hard classification of a pixel corresponding to a vehicle, given by the TextonBoost classifier, and P_v^LBP(x) ∈

{0,1} is the hard classification of a pixel corresponding to a vehicle, given by the SVM classifier with LBPs as inputs.

22. The non-transitory computer-usable data carrier of claim 18, wherein the pre-determined threshold is 0.5.

23. The non-transitory computer-usable data carrier of claim 18, wherein the method further comprises adjusting the pre-determined threshold based on additional information including at least one of the total number of vehicles entering and exiting the said parking lot and the total number of vacant parking stalls.

24. The non-transitory computer-usable data carrier of claim 18, wherein obtaining the parking lot layout further comprises:

automatically detecting one or more markers or lines defining one or more parking spaces from images of the parking lot when the parking lot was at least partially empty or receiving data input by a user via a user interface, wherein the data comprises markers or lines on one or more images of the parking lot and the intersections of the markers or lines define one or more corners of parking spaces.

25. The non-transitory computer-usable data carrier of claim 18, wherein estimating the volume associated with a parking space in the image further comprises:

approximating the parking space with a parallelepiped by using the marker lines of the parking space and estimating the height as approximately half of the average lengths of the four base sides;

obtaining a mask of the parallelepiped viewed from a given camera angle by geometric operations that yield a projection of a three-dimensional region of interest for the given parking space;

for each pixel within the region, assigning a probability of the pixel'"'"'s membership to the given parking space, wherein the membership probability density function is a function of the distance from a center of the region of interest and a plurality of possible overlapping regions with neighboring parking spaces.