SIDE WINDOW DETECTION IN NEAR-INFRARED IMAGES UTILIZING MACHINE LEARNING

• US 20150279036A1
• Filed: 04/01/2014
• Published: 10/01/2015
• Est. Priority Date: 04/01/2014
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Paul, Peter, Artan, Yusuf
• Granted Patent: US 9,652,851 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/025 Extracting rules from data
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/20072 Graph-based image processing
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20152 Watershed segmentation
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 2207/30248 Vehicle exterior or interior
• CPC Class Codes: G06T 2207/30268 Vehicle interior
• CPC Class Codes: G06T 7/149 involving deformable models, e.g. active contour models
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 10/755 Deformable models or variational models, e.g. snakes or active contours
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 20/593 Recognising seat occupancy
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

Methods, systems and processor-readable media for side window detection in near-infrared (NIR) images utilizing machine learning. An image-capturing unit can capture an image/video in a near-infrared (NIR) band via a side window of an incoming vehicle. A deformable part model can be generated utilizing a side window detection and B-frame detection in order to obtain a set of candidate side-windows. Side window detection can be performed based on a mixture of a tree model and a shared pool and can be globally optimized with dynamic programming and still-capture to detect the backseat side window boundary utilizing a B-pillar. A false alarm with respect to the deformable part model can be removed utilizing a super pixel generation and a longest-line detection unit in order to generate a refined deformable part model.

Claims

1. A side window detection method, comprising:

generating a deformable part model with respect to an image of a vehicle captured in a near-infrared band utilizing a side window detection and a B-frame detection module in order to obtain a set of candidate side-windows;

generating a refined deformable part model utilizing a super pixel generation and a longest-line detection in order to remove a false alarm with respect to said deformable part model; and

refining said detection performance of said refined deformable part model based on global regional information utilizing a local self-similarity based metric.

2. The method of claim 1 further comprising identifying a spatial feature associated with presence of an occupant by said side window detection and a B-frame detection module.

3. The method of claim 1 further comprising:

encoding an elastic deformation and 3D structure of an object for a face detection and pose estimation;

modeling a topological change due to a view point utilizing a mixture of trees with a shared pool of part defined at each landmark position and a global mixture wherein said global mixture captures a gross deformation change for a single viewpoint; and

considering said model as a linear classifier, which generates a face detection score for said image region and assigning a highest score as a face region.

4. The method of claim 1 further comprising globally optimizing said side window detection with a dynamic programming and still capture to detect a backseat side window boundary utilizing a B-pillar.

5. The method of claim 1 wherein said longest line detection unit further comprises:

computing an image derivative followed by a non-maximum suppression utilizing a canny edge detector and filling a line on a gradient image;

quantizing a gradient direction into a set of ranges, where all pixels having an orientation within a certain range fall into a corresponding bin and assigned a particular label; and

connecting an edge pixel having similar label via a connected component analysis algorithm with length above a certain threshold.

6. The method of claim 5 further comprising:

summing a binary line image in X direction and performing a low pass filtering of one dimensional line sum image; and

employing a longest line in said image that corresponds to upperilower side of said side window for potential side-window detection.

7. The method of claim 1 wherein said supper pixel generation further comprises:

deriving a superpixel by applying a watershed transform to a grayscale side view image wherein said superpixel with a highest number of elements below said longest line typically corresponds to a body of said vehicle.

8. The method of claim 7 further comprising wherein said body of said vehicle is not considered for said side-window detection unless said face detection score is high in order to reduce a false alarm rate.

9. The method of claim 1 wherein said local self similarity processing furtt comprises:

adding a weighting term to said face detection score in order to remove a homogenous false alarm region;

measuring a local self similarity at every landmark point by correlating a local patch centered at landmark with a larger surrounding image region by said weighting term; and

computing a sum of chi-square distance between a histogram of a local and larger region and incorporating said weighting term into said deformable part model inference and employing as a post-processing step utilizing a detected landmark point.

10. The method of claim 9 wherein said local self similarity processing further eliminates said false alarm due to a strong edge point.

11. A side window detection system, comprising:

a processor;

a data bus coupled to said processor; and

a computer-usable medium embodying computer program code, said computer-usable medium being coupled to said data bus, said computer program code comprising instructions executable by said processor and configured for;

generating deformable part model with respect to an image of a vehicle captured in a near-infrared band utilizing a side window detection and a B-frame detection module in order to obtain a set of candidate side-windows;

generating a refined deformable part model utilizing a superpixel generation and a longest-line detection in order to remove a false alarm with respect to said deformable part model; and

refining said detection performance of said refined deformable part model based on global regional information utilizing a local self-similarity based metric.

12. The system of claim 11 wherein said instructions are further configured for identifying a spatial feature associated with presence of an occupant by said side window detection and a B-frame detection module.

13. The system of claim 11 wherein said instructions are further configured for:

encoding an elastic deformation and 3D structure of an object for a face detection and pose estimation;

modeling a topological change due to a view point utilizing a mixture of trees with a shared pool of part defined at each landmark position and a global mixture wherein said global mixture captures a gross deformation change for a single viewpoint; and

considering said model as a linear classifier, which generates a face detection score for said image region and assigning a highest score as a face region.

14. The system of claim 11 wherein said instructions are further configured for globally optimizing said side window detection with a dynamic programming and still capture to detect a backseat side window boundary utilizing a B-pillar.

15. The system of claim 11 wherein said instructions are further configured for:

computing an image derivative followed by a non-maximum suppression utilizing a canny edge detector and fitting a line on a gradient image;

quantizing a gradient direction into a set of ranges;

where all pixels having an orientation within a certain range fall into a corresponding bin and assigned a particular label; and

connecting an edge pixel having similar label via a connected component analysis algorithm with length above a certain threshold.

16. The system of claim 15 wherein said instructions are further configured for:

summing a binary line image in X direction and performing a low pass filtering of one dimensional line sum image; and

employing a longest line in said image that corresponds to upper/lower side of said side window for potential side-window detection.

17. The system of claim 11 wherein said superpixel generation further comprises:

deriving a superpixel by applying a watershed transform to a grayscale side view image wherein said superpixel with a highest number of elements below said longest line typically corresponds to a body of said vehicle.

18. The system of claim 11 wherein said instructions are further configured for:

adding a weighting term to said face detection score in order to remove a homogenous false alarm region;

measuring a local self similarity at every landmark point by correlating a local patch centered at landmark with a larger surrounding image region by said weighting term; and

computing a sum of chi-square distance between a histogram of a local and larger region and incorporating said weighting term into said deformable part model inference and employing as a post-processing step utilizing a detected landmark point.

19. A processor-readable medium storing computer code representing instructions to cause a process of side window detection, said computer code comprising code to:

generate a deformable part model with respect to an image of a vehicle captured in a near-infrared band utilizing a side window detection and a B-frame detection module in order to obtain a set of candidate side-windows;

generate a refined deformable part model utilizing a superpixel generation and a longest-line detection in order to remove a false alarm with respect to said deformable part model; and

refine said detection performance of said refined deformable part model based on global regional information utilizing a local self-similarity based metric.

20. The processor-readable medium of claim 19 wherein said code further comprises code to identify a spatial feature associated with a presence of an occupant.