Robust windshield detection via landmark localization

• US 9,633,267 B2
• Filed: 04/04/2014
• Issued: 04/25/2017
• Est. Priority Date: 04/04/2014
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Inventors: Xu, Beilei, Artan, Yusuf O., Paul, Peter
• Primary Examiner(s): TSAI, TSUNG YIN
• Application Number:
  US14/245,319
• Publication Number:
  US 20150286883A1
• Time in Patent Office: 1,117 Days
• Field of Search: None
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/20132 Image cropping
• CPC Class Codes: G06T 2207/30268 Vehicle interior
• CPC Class Codes: G06V 10/754 involving a deformation of the sample pattern or of the reference pattern; Elastic matching
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 20/593 Recognising seat occupancy
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G08G 1/0175 by photographing vehicles, e.g. when violating traffic rules
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

A system and method that includes training a classifier using uniquely defined landmark points along the windshield region based on an elastic deformation model. The deformation model uses mixtures of trees with a shared pool of parts and can be globally optimized with dynamic programming and still capture much relevant global elastic structure. Once a candidate area is identified in the scene, a learned threshold is applied to the classification score of the candidate area to determine if the candidate area is a windshield. The identified area is then cropped out for further downstream process.

Claims

1. A system configured to perform landmark-based image analysis for localization of a windshield within an image comprising:

an image capture device operably connected to a data processing device that captures an image of a target vehicle; and

a processor-usable medium embodying computer code, said processor-usable medium being coupled to said data processing device, said computer program code comprising instructions executable by said processor and configured for;

identifying a plurality of landmark points associated with a windshield portion of a vehicle, at least one of the landmark points corresponding to at least one side view mirror adjacent the windshield;

organizing each of the landmark points in a tree structure defined by spatial relationships between the landmark points; and

calculating a classification score;

wherein the classification score is generated using;

S(I,L)=App(I,L)+Shape(L), where I is the image of interest, and L={l_i=(x_i,y_i), i=1, 2, . . . , N_L} is the set of pixel locations for the N_L landmarks defined for the windshield, App(I,L) sums the appearance evidence for placing a template for part i at location l_i for i=1, 2, . . . , N_L, and Shape(L) scores the specific spatial arrangement of parts L.

2. A system as set forth in claim 1, wherein the landmark points define a tree having a start and end points each on a straight line segment of said windshield thereby minimizing the impact of relative position variation between tree start and end points on windshield identification.

3. A system as set forth in claim 2, wherein the start and end points of the tree are at a lower center portion of the windshield.

4. A system as set forth in claim 1, wherein said computer program code comprising instructions executable by said processor is further configured for detecting violations of lane requirements in managed lanes.

5. A system as set forth in claim 1, wherein said computer program code comprising instructions executable by said processor is further configured for comparing the classification score to a threshold value and, if the classification score exceeds the threshold value, cropping a region identified by the landmark points.

6. A computer implemented method of enhancing an image comprising:

using a data processing device to identify a plurality of landmark points associated with a windshield portion of a vehicle;

using a data processing device to organize each of the landmark points in a tree structure defined by spatial relationships between the landmark points and

using a data processing device to calculate a classification score based on the landmark points;

wherein the classification score is generated using;

S(I,L)=App(I,L)+Shape(L), where I is the image of interest, and L={l_i=(x_i,y_i), i=1, 2, . . . , N_L} is the set of pixel locations for the N_L landmarks defined for the windshield, App(I,L) sums the appearance evidence for placing a template for part i at location l_i for i=1, 2, . . . , N_L, and Shape(L) scores the specific spatial arrangement of parts L.

7. A computer implemented method as set forth in claim 6, further comprising comparing the classification score to a threshold value and, if the classification score exceeds the threshold value, cropping the region identified by the landmark points.

8. A computer implemented method as set forth in claim 6, wherein the landmark points define a tree having a start and end points each on a straight line segment of said windshield thereby minimizing the impact of relative position variation between tree start and end points on windshield identification.

9. A computer implemented method as set forth in claim 8, wherein the start and end points of the tree are at a lower center portion of the windshield.

10. A computer implemented method as set forth in claim 6, further comprising detecting violations of lane requirements in managed lanes.

11. A non-transitory computer-usable medium for performing landmark-based image analysis for localization of a windshield within an image, said computer-usable medium embodying a computer program code, said computer program code comprising computer executable instructions configured for:

identifying a plurality of landmark points associated with a windshield portion of a vehicle using an elastic deformation model, at least one of the landmark points corresponding to at least one side view mirror adjacent the windshield;

organizing each of the landmark points in a tree structure defined by spatial relationships between the landmark points; and

calculating a classification score based on the landmark points;

wherein the classification score is generated using;

S(I,L)=App(I,L)+Shape(L), where I is the image of interest, and L={l_i=(x_i,y_i), i=1, 2, . . . , N_L} is the set of pixel locations for the N_L landmarks defined for the windshield, App(I,L) sums the appearance evidence for placing a template for part i at location l_i for i=1, 2, . . . , N_L, and Shape(L) scores the specific spatial arrangement of parts L.

12. A non-transitory computer-usable medium as set forth in claim 11, wherein the instructions further comprise comparing the classification score to a threshold value and, if the classification score exceeds the threshold value, cropping the region identified by the landmark points.

13. A non-transitory computer-usable medium as set forth in claim 11, wherein the landmark points define a tree having a start and end points each on a straight line segment of said windshield thereby minimizing the impact of relative position variation between tree start and end points on windshield identification.

14. A non-transitory computer-usable medium as set forth in claim 11, wherein the start and end points of the tree are at a lower center portion of the windshield.