Methods, apparatus and systems for surface type detection in connection with locate and marking operations

• US 9,046,413 B2
• Filed: 08/15/2011
• Issued: 06/02/2015
• Est. Priority Date: 08/13/2010
• Status: Active Grant

• Current Assignee: Certusview Technologies LLC (Dycom Industries Incorporated)
• Original Assignee: Certusview Technologies LLC (Dycom Industries Incorporated)
• Inventors: Nielsen, Steven, Chambers, Curtis, Farr, Jeffrey, Vice, Jack M.
• Primary Examiner(s): LE, TOAN M
• Application Number:
  US13/210,291
• Publication Number:
  US 20120065924A1
• Time in Patent Office: 1,387 Days
• Field of Search: 702/135, 702/81, 702/84, 702/127, 702/130, 702/134, 702/136, 702188-189, 427136-138, 427/160, 348/82, 348/164, 404/17, 404 71- 72, 404/83, 404/101, 404/108, 404/110, 700/283, 700/285
• US Class Current: 1/1
• CPC Class Codes: G01C 15/02 Means for marking measuring points
• CPC Class Codes: G01J 3/0264 Electrical interface; User interface
• CPC Class Codes: G01J 3/463 Colour matching
• CPC Class Codes: G06V 20/13 Satellite images

Abstract

Systems, methods, and apparatus for performing surface type detection in connection with locate and marking operations. In some embodiments, one or more sensors may be employed to collect information regarding a ground surface on which marking material is to be dispensed which is then analyzed to provide an estimate of a type of the surface. For example, a still-image or video camera may be used as a sensor that detects visible light reflecting from a surface. One or more images captured by the camera may be analyzed using analysis software to identify one or more characteristics indicative of a surface type. As another example, one or more radiation sensors may be employed to measure an amount of electromagnetic radiation reflected by the sensed surface one or more selected wavelengths or ranges of wavelengths to identify a spectral signature that may also be indicative of a surface type.

Claims

1. An apparatus for determining a surface type of a surface on which marking material is to be dispensed by a marking device to mark a presence or an absence of at least one underground facility within a dig area, wherein at least a portion of the dig area is planned to be excavated or disturbed during excavation activities, the apparatus comprising:

at least one communication interface;

at least one memory to store processor-executable instructions; and

at least one processor communicatively coupled to the at least one memory and the at least one communication interface, wherein, upon execution of the processor-executable instructions, the at least one processor;

A) obtains sensor data relating to the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area, the sensor data being collected by one or more sensors attached to the marking device for dispensing the marking material, wherein the one or more sensors comprise at least one camera and the sensor data of A) comprises image data representing at least one image of the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area captured by the camera;

B) retrieves reference data associated with a plurality of surface types of the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area; and

C) generates surface type information based at least in part on the sensor data of A) and the reference data of B),

wherein in C), the at least one processor;

C1) identifies, based at least in part on the sensor data of A) and the reference data of B), at least one of the plurality of surface types as a candidate surface type for the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area;

C2) determines a surface signature associated with the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area, based at least in part on at least one of;

i) a frequency at which edges are detected in the at least one image;

ii) a luminance distribution associated with the image data;

iii) color information associated with the image data;

iv) an entropy value derived based on the image data;

v) a presence or an absence of one or more line patterns in the at least one image; and

vi) a compression operation on at least a portion of the image data to determine a surface compression ratio;

C3) compares the surface signature with a plurality of reference signatures in the reference data of B) to identify the candidate surface type, wherein each reference signature is indicative of at least one of;

i) a characteristic frequency of edges for at least one corresponding surface type in the plurality of surface types;

ii) a characteristic luminance distribution for at least one corresponding surface type in the plurality of surface types;

iii) a color characteristic for at least one corresponding surface type in the plurality of surface types;

iv) a characteristic entropy value for at least one corresponding surface type in the plurality of surface types;

v) one or more characteristic line patterns for at least one corresponding surface type in the plurality of surface types; and

vi) a reference compression ratio associated with at least one corresponding surface type in the plurality of surface types.

2. The apparatus of claim 1, wherein the reference data of B) comprises the plurality of reference signatures, each reference signature being associated with the at least one corresponding surface type in the plurality of surface types of the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area, and wherein in C), the at least one processor:

identifies at least one reference signature as being a closest match with the surface signature, wherein the at least one surface type corresponding to the at least one reference signature is identified as the candidate surface type in C1).

3. The apparatus of claim 1, wherein the reference data of B) comprises the plurality of reference signatures, each reference signature being associated with at least one corresponding surface type in the plurality of surface types, and wherein in C), the at least one processor:

associates at least one confidence score with each one of the plurality of surface types, based at least in part on an extent to which the at least one surface signature matches at least one reference signature associated with the one of the plurality of surface types on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area.

4. The apparatus of claim 3, wherein in C), the at least one processor:

identifies the at least one of the plurality of surface types as being unlikely to match the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area, based at least in part on the at least one confidence score associated with the at least one of the plurality of surface types.

5. The apparatus of claim 3, wherein the at least one surface signature comprises first and second surface signatures, and where in C), the at least one processor:

identifies, based at least in part on the first surface signature and the reference data of B), a first surface type of the plurality of surface types as possibly matching the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area, the first surface type being associated with a first confidence score;

identifies, based at least in part on the second surface signature and the reference data of B), a second surface type of the plurality of surface types as possibly matching the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area, the second surface type being associated with a second confidence score; and

identifies one of the first and the second surface types as the candidate surface type for the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area, based at least in part on the first and the second confidence scores.

6. The apparatus of claim 5, wherein the sensor data of A) comprises first sensor data collected by at least a first sensor and second sensor data collected by at least a second sensor, and wherein the first surface signature is determined based at least in part on the first sensor data and the second surface signature is determined based at least in part on the second sensor data.

7. The apparatus of claim 1, wherein in C1), the at least one processor:

determines the surface signature associated with the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area, based at least in part on the luminance distribution associated with the image data; and

compares the surface signature with the plurality of reference signatures in the reference data of B) to identify the candidate surface type, wherein each reference signature is indicative of the characteristic luminance distribution for the at least one corresponding surface type in the plurality of surface types.

8. The apparatus of claim 1, wherein in C1), the at least one processor:

determines the surface signature associated with the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area, based at least in part on the color information associated with the image data; and

compares the surface signature with the plurality of reference signatures in the reference data of B) to identify the candidate surface type, wherein each reference signature is indicative of the color characteristic for the at least one corresponding surface type in the plurality of surface types.

9. The apparatus of claim 8, wherein the color information associated with the image data comprises a spatial distribution of color in the at least one image.

10. The apparatus of claim 8, wherein the color information associated with the image data comprises a most prevalent color in the at least one image.

11. The apparatus of claim 8, wherein the color information associated with the image data comprises an average color in the at least one image.

12. The apparatus of claim 1, wherein in C1), the at least one processor:

determines the surface signature associated with the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area, based at least in part on the entropy value derived based on the image data; and

compares the surface signature with the plurality of reference signatures in the reference data of B) to identify the candidate surface type, wherein each reference signature is indicative of the characteristic entropy value for the at least one corresponding surface type in the plurality of surface types.

13. The apparatus of claim 12, wherein the entropy value is derived based at least in part on intensity information in the image data.

14. The apparatus of claim 12, wherein the entropy value is derived based at least in part on color information in the image data.

15. The apparatus of claim 1, wherein in C1), the at least one processor:

determines the surface signature associated with the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area, based at least in part on the frequency at which the edges are detected in the at least one image; and

compares the surface signature with the plurality of reference signatures in the reference data of B) to identify the candidate surface type, wherein each reference signature is indicative of the characteristic frequency of the edges for the at least one corresponding surface type in the plurality of surface types.

16. The apparatus of claim 1, wherein in C1), the at least one processor:

determines the surface signature associated with the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area, based at least in part on the presence or the absence of the one or more line patterns in the at least one image; and

compares the surface signature with the plurality of reference signatures in the reference data of B) to identify the candidate surface type, wherein each reference signature is indicative of the one or more characteristic line patterns for the at least one corresponding surface type in the plurality of surface types.

17. The apparatus of claim 1, wherein the surface signature is a first surface signature, the plurality of reference signatures are a first plurality of reference signatures, and in C), the at least one processor:

determines a second surface signature associated with the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area, based at least in part on a second portion of the at least one image;

compares the second surface signature with a second plurality of reference signatures in the reference data of B) to identify a second surface type as possibly matching the second portion of the at least one image; and

determines whether the first surface type is different from the second surface type.

18. The apparatus of claim 1, wherein in C), the at least one processor:

determines the surface compression ratio associated with the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area, at least in part by performing the compression operation on at least the portion of the image data; and

compares the surface compression ratio with the plurality of reference compression ratios in the reference data of B) to identify the candidate surface type, wherein each reference compression ratio is associated with the at least one corresponding surface type in the plurality of surface types.

19. An apparatus for determining a surface type of a surface on which marking material is to be dispensed by a marking device to mark a presence or an absence of at least one underground facility within a dig area, wherein at least a portion of the dig area is planned to be excavated or disturbed during excavation activities, the apparatus comprising:

at least one communication interface;

at least one memory to store processor-executable instructions; and

at least one processor communicatively coupled to the at least one memory and the at least one communication interface, wherein, upon execution of the processor-executable instructions, the at least one processor;

A) obtains sensor data relating to the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area, the sensor data being collected by one or more sensors attached to the marking device for dispensing the marking material, wherein the one or more sensors comprise an IR sensor and the sensor data of A) comprises at least one temperature measurement of the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area taken by the IR sensor;

B) retrieves reference data associated with a plurality of surface types of the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area; and

C) generates surface type information based at least in part on the sensor data of A) and the reference data of B),

wherein in C), the at least one processor;

C1) identifies, based at least in part on the sensor data of A) and the reference data of B), at least one of the plurality of surface types as a candidate surface type for the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area; and

C2) compares the at least one temperature measurement with a plurality of reference temperatures in the reference data of B) to identify the candidate surface type, wherein each reference temperature is associated with at least one corresponding surface type in the plurality of surface types.

20. The apparatus of claim 19, wherein the one or more sensors further comprise an ambient temperature sensor and an ambient light sensor, and the sensor data of A) further comprises an ambient temperature level and an ambient light level measured respectively by the temperature sensor and the ambient light sensor, and wherein the plurality of reference temperatures are selected based at least in part on the ambient temperature level and the ambient light level.

21. An apparatus for determining a surface type of a surface on which marking material is to be dispensed by a marking device to mark a presence or an absence of at least one underground facility within a dig area, wherein at least a portion of the dig area is planned to be excavated or disturbed during excavation activities, the apparatus comprising:

at least one communication interface;

at least one memory to store processor-executable instructions; and

at least one processor communicatively coupled to the at least one memory and the at least one communication interface, wherein, upon execution of the processor-executable instructions, the at least one processor;

A) obtains sensor data relating to the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area, the sensor data being collected by one or more sensors attached to the marking device for dispensing the marking material, wherein the one or more sensors comprises at least one radiation sensor capable of detecting electromagnetic radiation in at least one frequency band;

B) retrieves reference data associated with a plurality of surface types of the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area; and

C) generates surface type information based at least in part on the sensor data of A) and the reference data of B),

wherein in C), the at least one processor;

C1) identifies, based at least in part on the sensor data of A) and the reference data of B), at least one of the plurality of surface types as a candidate surface type for the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area;

C2) determines, based at least in part on the sensor data of A), a surface spectral signature associated with the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area; and

C3) compares the surface spectral signature with a plurality of reference spectral signatures in the reference data of B) to identify the candidate surface type, wherein each reference spectral signature is associated with at least one corresponding surface type in the plurality of surface types.

22. In a system comprising at least one communication interface, at least one memory to store processor-executable instructions, and at least one processor communicatively coupled to the at least one memory and the at least one communication interface, a method for determining a surface type of a surface on which marking material is to be dispensed by a marking device to mark a presence or an absence of at least one underground facility within a dig area, wherein at least a portion of the dig area is planned to be excavated or disturbed during excavation activities, the method comprising acts of:

A) obtaining sensor data relating to the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area, the sensor data being collected by one or more sensors attached to the marking device for dispensing the marking material, wherein the one or more sensors comprise at least one camera and the sensor data of A) comprises image data representing at least one image of the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area captured by the camera;

B) retrieving, from the at least one memory, reference data associated with a plurality of surface types of the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area; and

C) using the at least one processor to generate surface type information based at least in part on the sensor data of A) and the reference data of B),

wherein in C), the at least one processor;

C1) identifies, based at least in part on the sensor data of A) and the reference data of B), at least one of the plurality of surface types as a candidate surface type for the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area;

C2) determines a surface signature associated with the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area, based at least in part on at least one of;

i) a frequency at which edges are detected in the at least one image;

ii) a luminance distribution associated with the image data;

iii) color information associated with the image data;

iv) an entropy value derived based on the image data;

v) a presence or an absence of one or more line patterns in the at least one image; and

vi) a compression operation on at least a portion of the image data to determine a surface compression ratio; and

C3) compares the surface signature with a plurality of reference signatures in the reference data of B) to identify the candidate surface type, wherein each reference signature is indicative of at least one of;

i) a characteristic frequency of edges for at least one corresponding surface type in the plurality of surface types;

ii) a characteristic luminance distribution for at least one corresponding surface type in the plurality of surface types;

iii) a color characteristic for at least one corresponding surface type in the plurality of surface types;

iv) a characteristic entropy value for at least one corresponding surface type in the plurality of surface types;

v) one or more characteristic line patterns for at least one corresponding surface type in the plurality of surface types; and

vi) a reference compression ratio associated with at least one corresponding surface type in the plurality of surface types.

23. At least one non-transitory computer-readable storage medium encoded with at least one program including processor-executable instructions that, when executed by at least one processor, perform a method for determining a surface type of a surface on which marking material is to be dispensed by a marking device to mark a presence or an absence of at least one underground facility within a dig area, wherein at least a portion of the dig area is planned to be excavated or disturbed during excavation activities, the method comprising acts of:

A) obtaining sensor data relating to the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area, the sensor data being collected by one or more sensors attached to the marking device for dispensing the marking material, wherein the one or more sensors comprise at least one camera and the sensor data of A) comprises image data representing at least one image of the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area captured by the camera;

B) retrieving, from at least one memory, reference data associated with a plurality of surface types of the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area; and

C) generating surface type information based at least in part on the sensor data of A) and the reference data of B), and;

C1) identifying, based at least in part on the sensor data of A) and the reference data of B), at least one of the plurality of surface types as a candidate surface type for the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area;

C2) determining a surface signature associated with the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area, based at least in part on at least one of;

i) a frequency at which edges are detected in the at least one image;

ii) a luminance distribution associated with the image data;

iii) color information associated with the image data;

iv) an entropy value derived based on the image data;

v) a presence or an absence of one or more line patterns in the at least one image; and

vi) a compression operation on at least a portion of the image data to determine a surface compression ratio; and

C3) comparing the surface signature with a plurality of reference signatures in the reference data of B) to identify the candidate surface type, wherein each reference signature is indicative of at least one of;

i) a characteristic frequency of edges for at least one corresponding surface type in the plurality of surface types;

ii) a characteristic luminance distribution for at least one corresponding surface type in the plurality of surface types;

iii) a color characteristic for at least one corresponding surface type in the plurality of surface types;

iv) a characteristic entropy value for at least one corresponding surface type in the plurality of surface types;

v) one or more characteristic line patterns for at least one corresponding surface type in the plurality of surface types; and

vi) a reference compression ratio associated with at least one corresponding surface type in the plurality of surface types.

24. A marking apparatus for performing a marking operation to mark on a surface a presence or an absence of at least one underground facility, the marking apparatus comprising:

at least one actuator to dispense a marking material so as to form at least one locate mark on the surface to mark the presence or the absence of the at least one underground facility;

at least one sensor for sensing the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area;

at least one user interface including at least one display device;

at least one communication interface;

at least one memory to store processor-executable instructions; and

at least one processor communicatively coupled to the at least one memory, the at least one communication interface, the at least one user interface, and the at least one actuator, wherein upon execution of the processor-executable instructions, the at least one processor;

A) obtains sensor data relating to the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area, the sensor data being collected by the at least one sensor;

B) retrieves, from the at least one memory, reference data associated with a plurality of surface types of the surface on which the marking material is to be dispensed by the marking device to mark the presence or the absence of the at least one underground facility within the dig area; and

C) generates surface type information based at least in part on the sensor data of A) and the reference data of B).