System for road sign sheeting classification

• US 6,891,960 B2
• Filed: 08/10/2001
• Issued: 05/10/2005
• Est. Priority Date: 08/12/2000
• Status: Active Grant

• Current Assignee: Mandli Communications, Inc., Mandli Inc.
• Original Assignee: Facet Technologies LLC
• Inventors: Retterath, James E., Laumeyer, Robert A.
• Primary Examiner(s): Johns, Andrew W.
• Assistant Examiner(s): NAKHJAVAN, SHERVIN K
• Application Number:
  US09/928,218
• Publication Number:
  US 20020106109A1
• Time in Patent Office: 1,369 Days
• Field of Search: 382/104, 382/141, 382/170, 382/171, 382/173, 209/524, 209/536, 209/576, 209/580, 209/581, 209/587, 356/152.3, 356/421, 356/422, 348/91, 348/92, 359/531, 359/532, 359/533, 340/5.1, 340/901, 340/905
• US Class Current: 382/104
• CPC Class Codes: G01N 21/251 Colorimeters; Construction thereof
• CPC Class Codes: G01N 21/55 Specular reflectivity
• CPC Class Codes: G01N 21/84 Systems specially adapted for particular applications
• CPC Class Codes: G01N 2201/06113 Coherent sources; lasers
• CPC Class Codes: G01N 2201/102 Video camera
• CPC Class Codes: G01N 2201/12 Circuits of general importance; Signal processing
• CPC Class Codes: G01S 17/06 Systems determining position data of a target
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/582 of traffic signs
• CPC Class Codes: G08G 1/096758 where no selection takes place on the transmitted or the received information
• CPC Class Codes: G08G 1/096783 where the origin of the information is a roadside individual element

Abstract

A system for classifying different types of sheeting materials of road signs depicted in a videostream compares estimated retroreflectivity values against known minimum retroreflectivity values for each of a plurality of colors. Once a road sign has been identified in the videostream, the frames associated with that road sign are analyzed to determine each of a plurality of colors present on the road sign. An estimated retroreflectivity for each of the plurality of colors present on the road sign is then determined. By comparing the estimated retroreflectivity for each of the plurality of colors against known minimum retroreflectivity values for the corresponding color for different types of sheeting materials, an accurate determination of the classification of the sheeting material of the road sign is established. Preferably, certain conditions of gross failure of the sheeting material are filtered out before classification of the sheeting material is determined.

Claims

1. A method of automated classification of different types of sheeting materials on road signs comprising:

providing a database of known retrorefiectivity values associated with a plurality of colors for each of a plurality of different sheeting types;

identifying a road sign in at least a portion of at least one image frame;

determining at least one color associated with the road sign;

determining a retrorefiectivity value for the at least one color associated with the road sign; and

making an automated comparison of the retroreflectivity values for the at least one color with the known retroreflectivity values in the database to classify a sheeting material of the road sign as one of the plurality of different sheeting types.

2. The method of claim 1 wherein all of the steps are performed by a computer processing system.

3. The method of claim 1 wherein the step of identifying the road sign further comprises:

filtering out any road sign exhibiting a gross failure of a sheeting material of the road sign.

4. The method of claim 1 wherein the database includes at least one retroreflectivity value associated with a boundary of a range of retroreflectivity values for each of the plurality of colors for each of the plurality of different sheeting types.

5. The method of claim 1 wherein the at least one color associated with the road sign includes a foreground color and a background color.

6. The method of claim 5 wherein a retroreflectivity value is determined and the automated comparison is made first for the background color and then for the foreground color if a comparison of the background color does not isolate a unique value in the database for each sheeting type.

7. The method of claim 6 wherein the step of making the automated comparison further comprises:

comparing the sheeting type indicated for the foreground color with the sheeting type indicated for the background color;

if the sheeting types are the same, using the sheeting type as the sheeting type for the road sign; and

if the sheeting types are different, analyzing the image of road sign to account for potential variations in the retroreflectivity values for each of the foreground color and background color in order to resolve a single sheeting type for the road sign.

8. The method of claim 7 wherein the sheeting type of the lighter of the foreground and background color is weighted more heavily in determining the sheeting type for the road sign.

9. The method of claim 1 wherein the steps of identifying a road sign, determining at least one color associated with the road sign and determining a retroreflectivity value for the at least one color associated with the road sign are performed as an automated process based on a plurality of image and measurements recorded for road signs disposed along a roadway.

10. The method of claim 9 wherein the automated process comprises:

using a light source to illuminate an area that includes at least one road sign as the light source is traversed along the roadway;

collecting a plurality of light intensity measurements with at least one intensity sensor;

collecting a plurality of color images with at least one color camera directed to cover a field of view which includes at least a portion of the area illuminated by the light source; and

using a computer processing system to determine at least one color associated with the road sign and a retroreflectivity value for the at least one color associated with the road sign based on the plurality of color images and the plurality of light intensity measurements.

11. The method of claim 10 wherein the light source is strobed as the light source is traversed along the roadway and the step of collecting the plurality of light intensity measurements is synchronized to the strobing of the light source.

12. The method of claim 10 further comprising:

creating a characterization profile for the light source, the characterization profile including an array of known luminance values of reflections of the light source; and

utilizing the characterization profile for the light source as part of determining the at least one retroreflectivity value for that road sign.

13. The method of claim 12 wherein the array of known luminance values of reflection comprises reflected intensity values for the light source over a range of colors and reflected intensity values over a range of relative angles between the light source and the reflective surface.

14. The method of claim 12 further comprising:

collecting locational information for each of the plurality of light intensity measurements;

using the computer processing system and the locational information to determine a coordinate location for each road sign; and

,

utilizing the characterization profile for the light source and the coordinate location as part of determining the at least one retroreflectivity value for that road sign.

15. The method of claim 10 further comprising:

creating a characterization profile for the intensity sensor, the characterization profile including an array of intensity values of reflections measured for a known light source; and

utilizing the characterization profile for the intensity sensor as part of determining the at least one retroreflectivity value for that road sign.

16. The method of claim 10 wherein each light intensity measurement comprises a frame of pixel intensity values and a plurality of road signs are present in at least one of the light intensity measurements.

17. The method of claim 10 wherein the step of collecting a plurality of light intensity measurements is accomplished without targeting a particular road sign.

18. The method of claim 10 wherein the light source is mounted on a vehicle that traverses the roadway at speeds approximately equal to posted speed limits for the roadway.

19. The method of claim 10 further comprising:

providing a characterization profile for the light source, the characterization profile including an array of known luminance values associated with reflections of the light source;

providing a characterization profile for the intensity sensor, the characterization profile including an array of intensity values measured for reflections of a known light source; and

collecting locational information for at least each of the plurality of light intensity measurements,

wherein the step of using the computer processing system to determine the at least one color associated with the road sign and the retroreflectivity value for the at least one color associated with the road sign includes;

identifying a portion of at least one light intensity measurement associated with the road sign;

utilizing the locational information to determine a location of the light source for that light intensity measurement and a location of that road sign;

utilizing at least one of the plurality of color images to identify at least one color associated with that road sign;

utilizing the characterization profile of the light source and the characterization profile for the intensity sensor to determine a luminance value associated with the portion of that light intensity measurement associated with that road sign based on the location of the light source for that light intensity measurement, the location of that road sign and one color associated with that road sign; and

converting the luminance value to a retroreflectivity value for the one color.

20. The method of claim 19 wherein each light intensity measurement comprises a frame of pixel intensity values and wherein the step of identifying the portion of at least one light intensity measurement is accomplished by analyzing at least one of the color images to determine the presence of a road sign in a portion of the color image and correlating the portion of the color image to a corresponding light intensity measurement to identify the portion of the frame of pixel intensity values associated with that road sign.

21. A system for acquiring information to classify sheeting types of road signs disposed along a roadway comprising:

a vehicle having;

at least one high output light source;

at least one intensity sensor;

at least one color camera;

a positioning system; and

a control system operably connected to the light source, intensity sensor, color camera and positioning system such that the intensity sensor, color camera and positioning system record information associated with an area that includes at least one road sign as the vehicle traverses along the roadway in response to repeated illumination of the area by the light source; and

a computer processing system that utilizes the recorded information to determine at least one color associated with the road sign and a retroreflectivity value for the at least one color associated with the road sign and compare the retroreflectivity values with at least one known retroreflectivity value in a database of known retroreflectivity values associated with a plurality of colors for each of a plurality of different sheeting types to classify a sheeting material of the road sign as one of the plurality of different sheeting types.

22. The system of claim 21 further wherein the vehicle further comprises:

a laser scanning system that records distance information including at least a distance between the vehicle and each road sign,

wherein the computer processing system utilizes the distance information to determine a normal vector for a face of the road sign.

23. The system of claim 21 wherein the at least one high output light source comprises at least two strobe lights arranged to alternatively illuminate the area at an effective strobe rate of at least one flash per second.

24. The system of claim 21 wherein the at least one intensity sensor comprises a black and white camera.

25. The system of claim 21 wherein the at least one color camera comprises a pair of digital color cameras mounted on the vehicle to generate stereoscopic images of the area.

26. The system of claim 21 wherein the positioning system comprises a global positioning system.

27. The system of claim 21 wherein at least a portion of the control system is implemented using the computer processing system.

28. A system for automated determination of sheeting types for reflective surfaces disposed along a roadway comprising:

means for storing a set of known retroreflectivity values associated with a plurality of colors for each of a plurality of different sheeting types;

means for identifying a road sign in at least a portion of at least one image frame;

means for determining at least one color associated with the road sign;

means for determining a retroreflectivity value for the at least one color associated with the road sign; and

means for comparing the retroreflectivity values for the at least color with the known retroreflectivity values in the database to classify a sheeting material of the road sign as one of the plurality of different sheeting types.

29. A method for the determination of road sign sheeting type comprising:

creating a reference database containing at least one manufacturing standard retroreflectivity value for each of the plurality color for each of a plurality of commercially available road sign sheeting materials;

acquiring retroflectivity values for at least one color for a road sign;

comparing the acquired retroreflectivity values with the reference database using a computer processor; and

assigning a road sign sheeting type to the road sign in response to the comparison.

30. A method for classifying target objects having one of a plurality of types of reflective sheeting material according to sheeting type comprising:

identifying target objects in a videostream by isolating a portion of at least one videoframe containing a target object;

assigning a set of color values to each of a plurality of colors on the target object;

calculating the retroreflectivity for each color value of the target object;

comparing the retroreflectivity values with a database of standard retroreflectivity values; and

classifying a sheeting type of the target object based on the comparison.