Real time machine vision system for train control and protection

• US 10,086,857 B2
• Filed: 11/26/2014
• Issued: 10/02/2018
• Est. Priority Date: 11/27/2013
• Status: Active Grant

• Current Assignee: Condor Acquisition SUB II Incorporated (Luminar Technologies, Inc.)
• Original Assignee: Solfice Research Inc. (Luminar Technologies, Inc.)
• Inventors: Puttagunta, Shanmukha Sravan, Chraim, Fabien
• Primary Examiner(s): Tissot, Adam D
• Assistant Examiner(s): Dunn, Alex C
• Application Number:
  US14/555,501
• Publication Number:
  US 20160121912A1
• Time in Patent Office: 1,406 Days
• Field of Search: 701 19
• CPC Class Codes: B61L 2205/04 Satellite based navigation systems, e.g. global positioning system [GPS]
• CPC Class Codes: B61L 23/041 Obstacle detection
• CPC Class Codes: B61L 23/34 for indicating the distance between vehicles or trains by the transmission of signals therebetween
• CPC Class Codes: B61L 25/025 Absolute localisation, e.g. providing geodetic coordinates
• CPC Class Codes: B61L 27/04 Automatic systems, e.g. controlled by train; Change-over to manual control

Abstract

A system, method, and apparatus are disclosed for a machine vision system that incorporates hardware and/or software, remote databases, and algorithms to map assets, evaluate railroad track conditions, and accurately determine the position of a moving vehicle on a railroad track. One benefit of the invention is the possibility of real-time processing of sensor data for guiding operation of the moving vehicle.

Claims

1. A vehicle localization apparatus comprising:

a GPS receiver mounted to a vehicle, the GPS receiver providing a first geographical position of the vehicle;

a local map cache residing within the vehicle, the local map cache storing a local map of assets comprising, for each asset;

a location of the asset, properties associated with the asset, and one or more relationships relative to other assets;

one or more local environment sensors mounted on the vehicle to enable collection of data comprising, for observed assets present in a local environment in the vicinity of the vehicle;

position data of the observed assets relative to the vehicle, and properties associated with the observed assets;

one or more vehicle computers, the vehicle computers receiving the first geographical position from the GPS receiver to retrieve, from the local map cache, records associated with assets previously mapped in the vicinity of the first geographical position;

a feature extraction component implemented by the vehicle computers, the feature extraction component receiving the local environment sensor data to identify and locate observed assets presently within the vicinity of the vehicle; and

a position refinement component implemented by the vehicle computers, the position refinement component comparing the identity and location of observed assets from the feature extraction component with asset information retrieved from the local map cache to determine a second position of the vehicle that is refined relative to the first geographical position of the vehicle;

a wireless vehicular communication device via which the local map cache can download local map data from a remote database during vehicle operation; and

a map audit component identifying differences between the local map of assets and the observed assets and outputting said differences to the vehicular communication device for transmission to the remote database.

2. The vehicle localization apparatus of claim 1, in which the map audit component comprises a missing asset detector identifying assets that are present within the observed assets and not present within the local map of assets, or that are not present within the observed assets and present within the local map of assets.

3. The vehicle localization apparatus of claim 1, in which the map audit component comprises an asset alteration detector identifying assets within the observed assets having characteristics indicative of damage or tampering that differ from characteristics associated with the asset within the local map of assets.

4. A vehicle localization apparatus comprising:

a GPS receiver mounted to a vehicle adapted for travel on railway tracks, the GPS receiver providing a first geographical position of the vehicle;

a local map cache residing within the vehicle, the local map cache storing a local map of assets comprising, for each asset;

a location of the asset, properties associated with the asset, and one or more relationships relative to other assets;

one or more local environment sensors mounted on the vehicle to enable collection of data comprising, for observed assets present in a local environment in the vicinity of the vehicle;

position data of the observed assets relative to the vehicle, and properties associated with the observed assets;

one or more vehicle computers, the vehicle computers receiving the first geographical position from the GPS receiver to retrieve, from the local map cache, records associated with assets previously mapped in the vicinity of the first geographical position;

a feature extraction component implemented by the vehicle computers, the feature extraction component receiving the local environment sensor data to identify and locate observed assets presently within the vicinity of the vehicle; and

a position refinement component implemented by the vehicle computers, the position refinement component comparing the identity and location of observed assets from the feature extraction component with asset information retrieved from the local map cache to determine a second position of the vehicle that is refined relative to the first geographical position of the vehicle;

a wireless vehicular communication device via which the local map cache can download local map data from a remote database during vehicle operation; and

a track clearance evaluation component receiving information from the feature extraction component indicating a location of a first asset, the track clearance evaluation component identifying the first asset as an obstruction and reporting the obstruction location to a backend server via the vehicular communication device.

5. A method of updating asset information within a centralized map database implemented by one or more network-connected servers, the method comprising the steps of:

receiving by the centralized map database a request for map data from a remote vehicle, where the vehicle is a train, the vehicle having local environment sensors and a local map cache;

transmitting a first set of map data from the centralized map database to the remote vehicle in response to the request, the first set of map data comprising asset information, the asset information comprising identification, features and location of one or more assets; and

receiving at the centralized map database, from the remote vehicle, a report indicative of one or more differences between the first set of map data and information detected by the vehicle local environment sensors, the report indicative of obstruction clearance relative to the path of the train; and

updating the centralized map database based on information within the report.