RAIL TRACK ASSET SURVEY SYSTEM

• US 20170066459A1
• Filed: 12/29/2015
• Published: 03/09/2017
• Est. Priority Date: 09/03/2015
• Status: Active Grant

• Current Assignee: Sameer Singh
• Original Assignee: Sameer Singh
• Inventors: Singh, Sameer
• Granted Patent: US 10,081,376 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B61K 9/08 Measuring installations for surveying permanent way applications of measuring apparatus or devices for track building purposes E01B35/00; measuring techniques G01
• CPC Class Codes: B61L 15/0072 On-board train data handling
• CPC Class Codes: B61L 15/0081 On-board diagnosis or maintenance
• CPC Class Codes: B61L 2205/04 Satellite based navigation systems, e.g. global positioning system [GPS]
• CPC Class Codes: B61L 23/04 for monitoring the mechanical state of the route
• CPC Class Codes: B61L 23/042 Track changes detection
• CPC Class Codes: B61L 23/044 Broken rails
• CPC Class Codes: B61L 23/045 Rail wear
• CPC Class Codes: B61L 23/047 Track or rail movements
• CPC Class Codes: B61L 23/048 Road bed changes, e.g. road bed erosion
• CPC Class Codes: B61L 25/021 Measuring and recording of train speed
• CPC Class Codes: B61L 25/023 Determination of driving direction of vehicle or train
• CPC Class Codes: B61L 25/025 Absolute localisation, e.g. providing geodetic coordinates
• CPC Class Codes: G01B 11/22 for measuring depth
• CPC Class Codes: G01N 21/8851 Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges G01N21/8806 and G01N21/93 - G01N21/95692 take precedence; optical measurement of dimensions G01B11/00; optical scanning G02B26/10; image transformation G06T3/00; computerised image enhancement G06T5/00; image processing per se for flaw detection G06T7/0002
• CPC Class Codes: G01N 2201/10 Scanning
• CPC Class Codes: G01N 2201/12 Circuits of general importance; Signal processing
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 18/2431 Multiple classes
• CPC Class Codes: G06V 20/52 : Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 2201/06 : Recognition of objects for industrial automation
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

The present application involves a railroad track asset surveying system comprising an image capture sensor, a location determining system, and an image processor. The image capture sensor is mounted to a railroad vehicle. The location determining system holds images captured by the image capture sensor. The image processor includes an asset classifier and an asset status analyser. The asset classifier detects an asset in one or more captured images and classifies the detected asset by assigning an asset type to the detected asset from a predetermined list of asset types according to one or more features in the captured image. The asset status analyser identifies an asset status characteristic and compares the identified status characteristic to a predetermined asset characteristic so as to evaluate a deviation therefrom.

Claims

1-40. -40. (canceled)

41. A railroad track asset surveying system comprising:

an image capture sensor mounted on a railroad vehicle;

a location determining system for images captured by the sensor; and

an image processor comprising;

an asset classifier for detecting an asset in one or more captured image and classifying the detected asset by assigning an asset type to the detected asset from a predetermined list of asset types according to one or more feature in the captured image; and

an asset status analyser for identifying an asset status characteristic and comparing the identified status characteristic to a predetermined asset characteristic so as to evaluate a deviation therefrom.

42. The system according to claim 41, wherein the predetermined asset status characteristic comprises a nominal asset status characteristic and/or a previously detected asset status characteristic.

43. The system according to claim 41, wherein the image capture sensor comprises a shape and/or surface property sensor.

44. The system according to claim 43, wherein the system comprises a plurality of image capture sensor types, comprising any combination, or all, of:

an areascan imaging sensor, a linescan imaging sensor, a three-dimensional surface profile sensor and an asset distance sensor.

45. The system according to claim 44, wherein at least one of the three-dimensional surface profile sensor and the asset distance sensor comprises a laser sensor device.

46. The system according to claim 43, wherein the image capture sensor comprises a light sensor for sensing brightness and/or colour within a visible wavelength band.

47. The system according to claim 46, wherein the system comprises a plurality of image capture sensor types, comprising any combination, or all, of an areascan imaging sensor, a linescan imaging sensor, a three-dimensional surface profile sensor and an asset distance sensor.

48. The system according to claim 47, wherein at least one of the three-dimensional surface profile sensor and the asset distance sensor comprises a laser sensor device.

49. The system according to claim 43, wherein the system comprises a thermal imaging sensor.

50. The system according to claim 41, wherein a plurality of image capture sensors are used and at least one of the asset classifier and asset status analyser correlates an asset feature in an image captured by one sensor with a corresponding image captured by one or more further sensor.

51. The system according to claim 41, comprising:

a plurality of image capture sensors, each mounted in a different angular alignment so as to have a different field of view relative to the direction of travel of the rail road vehicle;

wherein each image capture sensor faces in a different one of a forward, downward, upward or lateral direction relative to the direction of travel and the image processor automatically processes said different fields of view for a common location.

52. The system according to claim 41, wherein the asset feature comprises an edge profile and/or dimension of the asset, and geometric features or template matching is used to determine a degree of similarity between the feature of the detected asset and a predetermined geometric feature or template.

53. The system according to claim 52, wherein a confidence score is determined and assigned to an asset classification or status analysis by the system according to a degree of the match between the detected asset feature and the predetermined feature or template.

54. The system according to claim 41, wherein the asset feature comprises a surface property profile of the asset and surface property feature or template matching is used to determine a degree of similarity between the feature of the detected asset and a predetermined surface property feature or template.

55. The system according to claim 54, wherein a confidence score is determined and assigned to an asset classification or status analysis by the system according to a degree of the match between the detected asset feature and the predetermined feature or template.

56. The system according to claim 41, wherein the system outputs an indication of anomaly detection in the event that the degree of similarity between the detected asset and the predetermined template is below a threshold level for positive asset classification.

57. The system according to claim 41, comprising a plurality of asset classifiers comprising two or more of a rule-based classifier, a template-based classifier and a statistical feature matching tool.

58. The system according to claim 41, wherein the image processor identifies one or more pixel cluster within an image according to one or more pixel brightness or colour property, each pixel clusters being used by the asset classifier and/or asset status analyser used to determine an edge, colour, texture or shape feature of an asset.

59. The system according to claim 41, wherein the asset status analyser determines the spacing between an asset and the railroad track and compares the determined spacing with a predetermined threshold minimum clearance and outputs the asset status characteristic in dependence thereon.

60. The system according to claim 41, wherein the predetermined asset status comprises a previously determined asset status characteristic, and the asset status analyser determines a change in asset orientation, shape, edge and/or colour relative to the previously determined asset status characteristic.

61. The system according to claim 41, wherein the asset status analyser analyses a plurality of asset status characteristics and outputs an asset status indicator comprising an aggregate of said asset status characteristic analyses as a single value on an asset status severity scale.

62. The system according to claim 41, wherein the image processor comprises a plurality of automated data processors, wherein a first processor is arranged to collate and log captured images and associated location data in real time in a data store, and at least one further processor may be arranged to process the captured images in parallel with the first processor to classify and/or analyse asset status, wherein the data store may comprise a buffer portion for amassing images to be processed by the at least one further processor.

63. The system according to claim 41, wherein the image processor comprises a central processor and either or both of a field programmable gate array and a graphics card for performing a function of the asset classifier and/or asset status analyser.

64. The system according to claim 41, wherein the location determining system comprises a vehicle travel distance sensor and/or a track position sensor, the location determining system outputting a location indication as a one-dimensional measure of distance along the track from a fixed track reference point.

65. The system according to claim 41, wherein the rate of image capture is controlled automatically according to the railroad vehicle travel speed.

66. The system according to claim 41, wherein the operation of the image processor including at least one of the asset classifier and the asset status analyser is automated upon receipt of captured image data and/or location determination data.

67. The system according to claim 66, wherein the asset classifier and asset status analyser perform automated image analytics by use of software modules for processing image data to generate one or more output comprising the identity, properties and condition of each identified asset.

68. The system according to claim 67, wherein asset and condition information, including defect data with location information, is transmitted wirelessly from the railroad vehicle to a central control or data centre from where end-users can retrieve it for further analysis and maintenance planning.

69. The system according to claim 41, wherein the image processor cross references images captured by a plurality of different types of image capture sensor according to a corresponding time stamp and/or location determination between said images.

70. The system according to claim 69, comprising at least one two dimensional image capture sensor and a laser sensor for capturing a three dimensional image in the form of an asset surface profile, wherein the asset classifier and/or asset status analyser automatically integrates said different sensor inputs for a common location determination.

71. The system according to claim 41, further comprising:

a transmitter arranged to transmit a visual output signal to an operator console, the visual output signal having at least one of a combination of 2D or 3D maps indicating identified assets, asset risk reports, and/or asset data graphs suitable for maintenance purposes.

72. The system according to claim 41, further comprising:

a local data store, wherein captured images from the image capture sensor and location data from the location determining system are indexed by the image processor within one or more database in the local data store.

73. The system according to claim 41, wherein the asset classifier comprises a vegetation classifier and the asset status analyser comprises a vegetation status analyser.

74. The system according to claim 41, wherein the asset status analyser outputs risk assessment data for railroad track assets comprising signalling, level crossings and overhead lines.

75. The system according to claim 41, wherein the image capture sensor, the image processor and location determining system is configured for mounting to a passenger and freight railway vehicle or locomotive in use.

76. The system according to claim 41, wherein the image capture sensor, the image processor and location determining system is provided as a singular assembly, said assembly further comprising at least one of the following:

a battery for operation of the system in isolation of external power and;

a removable processing unit configured to allow the image processing unit and an associated data store to be removed from the remainder of the system in a modular fashion.

77. The system according to claim 41, being arranged to operate selectively in both an attended mode in which a human operator provides control inputs for image data acquisition or control of the asset status analyser, wherein operator software tools are available for review and reporting of asset status information, and an unattended mode, wherein the system operates fully autonomously according to a preprogramed set of machine readable instructions embedded within it.

78. The system according to claim 77, wherein the unattended modes do not require human intervention for starting, stopping, data acquisition or analysis and is fully automated from data collection to transmission of data analysis to an operational control centre.

79. The system according to claim 41, wherein the assets comprise the railroad track, railroad track-mounted assets and objects in the vicinity of the railroad track.

80. A railroad track asset surveying method comprising:

operating an image capture sensor mounted on a railroad vehicle during motion of the railroad vehicle along a track,

determining and logging a location of the image capture sensor for each image captured by the sensor, and

automatically processing the captured images upon receipt using one or more computer processor by detecting an asset in one or more captured image and classifying the detected asset by assigning an asset type to the detected asset from a predetermined list of asset types according to one or more feature identified in the captured image, and identifying an asset status characteristic and comparing the identified status characteristic to a predetermined asset characteristic so as to evaluate a deviation therefrom.

81. A data carrier comprising machine readable code of the operation of railroad track asset survey image processor configured to:

receive images captured by an image capture sensor mounted on a railroad vehicle during motion of the railroad vehicle along a track;

log a geographic location for each image captured by the sensor;

process the captured images by detecting an asset in one or more captured image and classifying the detected asset by assigning an asset type to the detected asset from a predetermined list of asset types according to one or more feature identified in the captured image; and

identify an asset status characteristic and comparing the identified status characteristic to a predetermined asset characteristic so as to evaluate a deviation therefrom, and output an asset status indication based upon said deviation.

82. The data carrier according to claim 81, comprising individual modules responsible for:

real-time data acquisition of said images and geographic locations;

high speed data analysis comprising said classifying the detected asset identifying asset status characteristic; and

, transfer of asset status indications to one or more external interfaces, wherein the high speed data analysis module is selectively useable and/or reprogrammable.

83. The data carrier according to claim 81, wherein the asset survey image processor is arranged under the control of the machine readable instructions to receive raw captured image data as input either directly from the image capture sensor, or after a plurality of individual images into one or more combined image, and to output automatically a determination on the presence or absence of a recognisable asset, an asset classification and the asset status indication comprising any or any combination of asset condition grade, an asset size, shape and/or colour property, and asset change relative to a previously recorded asset status indication.

84. The data carrier according to claim 81, wherein the image capture sensor comprises one or more of linescan, areascan and thermal imaging sensors mounted with a view of the rail track and the machine readable instructions for classifying the asset and/or outputting the asset status indication comprise a plurality of image analysis operators for any combination or all of edge detection, edge analysis, texture analysis, shape analysis, statistical analysis of pixel distribution data, colour analysis, image enhancement, image segmentation, template matching using statistical and neural network methods, feature based object recognition, rail track environment semantic knowledge analysis.

85. The data carrier according to claim 81, comprising instructions to store in a database an image location for the asset including an asset boundary, minimum and maximum image coordinates in horizontal and vertical directions, a centroid for the asset and one or more visual feature of the asset such as a colour or topology features.