Machine vision based track-occupancy and movement validation

• US 10,311,551 B2
• Filed: 12/13/2016
• Issued: 06/04/2019
• Est. Priority Date: 12/13/2016
• Status: Active Grant

• Current Assignee: Westinghouse Air Brake Technologies Corporation
• Original Assignee: Westinghouse Air Brake Technologies Corporation
• Inventors: Menon Gopalakrishna, Prahlad G.
• Primary Examiner(s): Shaawat, Mussa A
• Assistant Examiner(s): Jha, Abdhesh K
• Application Number:
  US15/377,451
• Publication Number:
  US 20180165532A1
• Time in Patent Office: 903 Days
• Field of Search: None
• CPC Class Codes: B61L 15/0072 On-board train data handling
• CPC Class Codes: B61L 23/00 Control, warning or like safety means along the route or between vehicles or trains
• CPC Class Codes: B61L 23/041 Obstacle detection
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle

Abstract

A train monitoring system includes an image sensor positioned on a rail vehicle to sense an area in front. The image sensor generates sensor signals. At least one non-transitory computer-readable medium has program instructions that, when executed by at least one processor in communication with the image sensor, causes the at least one processor to: (a) generate static image data based on the sensor signals, the static image data including a line representation of rails located in the area in front of the rail vehicle; (b) determine track occupancy of the rail vehicle by determining the set of rails occupied by the rail vehicle based on the static image data; and (c) determine an upcoming direction of movement of the rail vehicle by determining a direction of the set of rails occupied by the rail vehicle based on the static image data.

Claims

1. A train monitoring system comprising:

an image sensor disposed on a rail vehicle and positioned to sense an area in front of the rail vehicle, the image sensor generating sensor signals; and

at least one non-transitory computer-readable medium comprising program instructions that, when executed by at least one processor in communication with the image sensor, cause the at least one processor to;

(a) generate static image data based, at least in part, on the sensor signals, the static image data comprising a line representation of rails located in the area in front of the rail vehicle;

(b) determine a track occupancy of the rail vehicle by determining a set of rails occupied by the rail vehicle based, at least in part, on the static image data;

(c) determine an upcoming direction of movement of the rail vehicle by determining a direction of the set of rails occupied by the rail vehicle based, at least in part, on the static image data; and

(d) validate the determined track occupancy based, at least in part, on the upcoming direction of movement of the rail vehicle,

wherein generating the static image data comprises filtering the sensor signals with a low pass filter to smoothen an instantaneous image of the area in front of the rail vehicle, and

wherein generating the static image data comprises generating an enhanced smoothened image to exaggerate pixels of the smoothened image that belong to substantially tubular structures in the area in front of the rail vehicle.

2. The train monitoring system of claim 1, wherein the program instructions, when executed by the at least one processor, further cause the at least one processor to validate the determined track occupancy of the rail vehicle and the determined upcoming direction of movement of the rail vehicle based, at least in part, on time series data, wherein the time series data comprises data from the sensor signals obtained at two or more different times.

3. The train monitoring system of claim 1, wherein determining the track occupancy of the rail vehicle further comprises determining a number of sets of rails located in the area in front of the rail vehicle.

4. The train monitoring system of claim 1, wherein the image sensor is a monochrome camera, an RGB camera without depth perception, or a depth-sensing camera.

5. The train monitoring system of claim 1, further comprising an on-board computer system in communication with the image sensor, the on-board computer system comprising the at least one processor configured to perform steps (a)-(d).

6. The train monitoring system of claim 1, wherein the at least one processor comprises a microprocessor dedicated solely to processing sensor data from the image sensor.

7. The train monitoring system of claim 1, wherein generating the static image data comprises determining two largest components in the enhanced smoothened image in each quantified angular direction category.

8. The train monitoring system of claim 7, wherein generating the static image data comprises quantifying lines corresponding to detected connected-component regions representing the rails in front of the rail vehicle in each quantified angular direction category.

9. The train monitoring system of claim 8, wherein generating the static image data comprises separating lines representing rails into categories based, at least in part, on relative slopes of the rails.

10. The train monitoring system of claim 1, wherein determining the upcoming direction of movement of the rail vehicle comprises determining an upcoming angular direction of the set of rails occupied by the rail vehicle.

11. The train monitoring system of claim 1, further comprising an event data recorder in communication with the at least one processor, the event data recorder configured to store information from the sensor signals.

12. The train monitoring system of claim 1, wherein the program instructions cause the processor to detect obstacles along the rails based, at least in part, on the sensor signals from the image sensor.

13. The train monitoring system of claim 1, wherein the program instructions cause the processor to evaluate a condition of the rails based, at least in part, on the sensor signals from the image sensor.

14. A method of monitoring a train comprising:

sensing an area in front of a rail vehicle using an image sensor, the image sensor generating sensor signals;

generating static image data based, at least in part, on the sensor signals with at least one processor in communication with the image sensor, wherein the static image data comprises a line representation of rails located in the area in front of the rail vehicle;

determining, with the at least one processor, track occupancy of the rail vehicle by determining a set of rails occupied by the rail vehicle based, at least in part, on the static image data;

determining, with the at least one processor, an upcoming direction of movement of the rail vehicle by determining a direction of the set of rails occupied by the rail vehicle based, at least in part, on the static image data; and

validating the determined track occupancy based, at least in part, on the upcoming direction of movement of the rail vehicle,

wherein generating the static image data comprises filtering the sensor signals with a low pass image filter to smoothen an instantaneous image of the area in front of the rail vehicle, and

wherein generating the static image data comprises generating an enhanced smoothened image to exaggerate pixels of the smoothened image that belong to substantially tubular structures in the area in front of the rail vehicle.

15. The method of claim 14, further comprising:

validating, with the at least one processor, the determined track occupancy of the rail vehicle and the determined upcoming direction of movement of the rail vehicle based, at least in part, on time series data, wherein the time series data comprises data from the sensor signals obtained at two or more different times.

16. The method of claim 14, further comprising:

determining the track occupancy of the rail vehicle further comprises determining a number of sets of rails located in the area in front of the rail vehicle.

17. The method of claim 14, wherein generating the static image data comprises determining two largest the two largest components in the enhanced smoothened image in each quantified angular direction category.

18. The method of claim 17, wherein generating the static image data comprises quantifying lines corresponding to detected connected-component regions representing the rails in front of the rail vehicle in each quantified angular direction category.

19. The method of claim 18, wherein generating the static image data comprises separating lines representing rails into categories based, at least in part, on relative slopes of the rails.

20. The method of claim 14, wherein determining the upcoming direction of movement of the rail vehicle comprises determining an upcoming angular direction of the set of rails occupied by the rail vehicle.

21. The method of claim 14, wherein an event data recorder is in communication with the at least one processor, the event data recorder configured to store information from the sensor signals.

22. The method of claim 14, wherein the at least one processor is configured to detect obstacles along the rails based, at least in part, on the sensor signals from the image sensor.

23. The method of claim 14, wherein the at least one processor is configured to evaluate a condition of the rails based, at least in part, on the sensor signals from the image sensor.

24. A computer program product comprising at least one non-transitory computer-readable medium including program instructions that, when executed by at least one computer including at least one processor, causes the computer to:

generate static image data based, at least in part, on sensor signals generated from an image sensor in communication with the at least one processor, wherein the image sensor is disposed on a rail vehicle and positioned to sense an area in front of the rail vehicle, wherein the static image data comprises a line representation of rails located in the area in front of the rail vehicle;

determine track occupancy of the rail vehicle by determining a set of rails occupied by the rail vehicle based, at least in part, on the static image data;

determine an upcoming direction of movement of the rail vehicle by determining a direction of the set of rails occupied by the rail vehicle based, at least in part, on the static image data; and

validate the determined track occupancy based, at least in part, on the upcoming direction of movement of the rail vehicle,

wherein generating the static image data comprises filtering the sensor signals with a low pass image filter to smoothen an instantaneous image of the area in front of the rail vehicle, and

wherein generating the static image data comprises generating an enhanced smoothened image to exaggerate pixels of the smoothened image that belong to substantially tubular structures in the area in front of the rail vehicle.

25. The computer program product of claim 24, wherein the program instructions cause the computer to validate the determined track occupancy of the rail vehicle and the determined upcoming direction of movement of the rail vehicle based, at least in part, on time series data, wherein the time series data comprises data from the sensor signals obtained at two or more different times.

26. The computer program product of claim 24, wherein determining the track occupancy of the rail vehicle further comprises determining a number of sets of rails located in the area in front of the rail vehicle.

27. The computer program product of claim 24, wherein determining the upcoming direction of movement of the rail vehicle comprises determining an upcoming angular direction of the set of rails occupied by the rail vehicle.