Aerial camera system and method for identifying route-related hazards

• US 9,873,442 B2
• Filed: 02/17/2015
• Issued: 01/23/2018
• Est. Priority Date: 06/04/2002
• Status: Expired due to Term

• Current Assignee: Westinghouse Air Brake Technologies Corporation
• Original Assignee: General Electric Company
• Inventors: Costa, Brad Thomas, Snyder, Seneca, Duncan, Jerry, Kraeling, Mark Bradshaw, Miner, Michael Scott, Clouse, Shannon Joseph, Azam, Anwarul, Blair, Matthew Lawrence, Naithani, Nidhi, Rao, Dattaraj Jagdish, Bind, Anju, Chaki, Sreyashi Dey, Nelson, Scott Daniel, Naphade, Nikhil Uday, Chung, Wing Yeung, Ballesty, Daniel Malachi, Shaffer, Glenn Robert, Kisak, Jeffrey James, DiDomenico, Dale Martin
• Primary Examiner(s): Huang, Frank
• Application Number:
  US14/624,069
• Publication Number:
  US 20150158513A1
• Time in Patent Office: 1,071 Days
• CPC Class Codes: B61L 23/04 for monitoring the mechanical state of the route
• CPC Class Codes: B61L 23/041 Obstacle detection
• CPC Class Codes: B61L 23/042 Track changes detection
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64U 20/87 Mounting of imaging devices, e.g. mounting of gimbals
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 80/86 Land vehicles
• CPC Class Codes: G06V 20/182 Network patterns, e.g. roads or rivers
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control

Abstract

An aerial camera system includes an aerial device disposed onboard a non-aerial vehicle as the non-aerial vehicle moves along a route. The aerial device also can be configured to fly above the route during movement of the vehicle along the route. The camera unit is configured to be disposed onboard the aerial device and to generate image data during flight of the aerial device. The one or more image analysis processors are configured to examine the image data and to identify a hazard disposed ahead of the non-aerial vehicle along a direction of travel of the non-aerial vehicle based on the image data. A method for identifying route-related hazards using image data obtained from a camera unit on an aerial device.

Claims

1. A system, comprising:

a first aerial device configured to be disposed onboard a non-aerial vehicle as the non-aerial vehicle moves along a route, the first aerial device also configured to fly above the route during movement of the non-aerial vehicle along the route;

a first camera unit configured to be disposed onboard the first aerial device and to generate first image data during flight of the first aerial device;

one or more image analysis processors configured to examine the first image data and to identify a hazard disposed ahead of the non-aerial vehicle along a direction of travel of the non-aerial vehicle based on the first image data; and

a control unit configured to control the flight of the first aerial device, wherein the control unit is configured to autonomously prevent the first aerial device from flying outside of a three dimensional movement envelope disposed ahead of the non-aerial vehicle during the movement of the non-aerial vehicle.

2. The system of claim 1, wherein the first camera unit is configured to generate the first image data representative of a segment of the route disposed ahead of the non-aerial vehicle and the one or more image analysis processors are configured to examine the segment of the route based on the first image data.

3. The system of claim 1, wherein the one or more image analysis processors are configured to be disposed onboard the non-aerial vehicle and the first aerial device is configured to wirelessly communicate the first image data to the one or more image analysis processors during flight of the first aerial device off of the non-aerial vehicle.

4. The system of claim 1, wherein the first camera unit is configured to be a forward-facing camera unit of the first aerial device that generates the first image data representative of an area ahead of the first aerial device along a direction of travel of the first aerial device, and further comprising one or more of a downward-facing camera unit configured to be disposed onboard the first aerial device and oriented in a downward direction to generate second image data representative of a portion of the route that is beneath the first aerial device or a rearward-facing camera unit configured to be disposed onboard the first aerial device and oriented in a rearward direction to generate third image data representative of a portion of the route that is between the first aerial device and the non-aerial vehicle.

5. The system of claim 4, wherein the first aerial device is configured to communicate one or more of the second image data or the third image data to the one or more image analysis processors during flight of the first aerial device, the one or more image analysis processors configured to examine the one or more of the second image data or the third image data to identify the hazard.

6. The system of claim 1, wherein the one or more image analysis processors are configured to receive the first image data generated by the first camera unit during flight of the first aerial device and to identify at least one of a broken component of the route or an obstacle on the route based on the first image data.

7. The system of claim 1, further comprising a second aerial device and a third aerial device, wherein each of the first aerial device, the second aerial device, and the third aerial device is configured to dock on the non-aerial vehicle to be charged by and travel with the non-aerial vehicle during movement of the non-aerial vehicle, wherein, during a time period that the first aerial device is flying off of the non-aerial vehicle, the second aerial device is charged and landed on the non-aerial device and the third aerial device is landed on and being charged by the non-aerial device.

8. The system of claim 1, wherein:

the non-aerial vehicle is a rail vehicle, and the route is a track on which the rail vehicle is configured to travel;

the one or more image analysis processors are configured to be disposed onboard the rail vehicle and the first aerial device is configured to wirelessly communicate the first image data to the one or more image analysis processors during flight of the first aerial device off of the rail vehicle; and

the one or more image analysis processors are configured to receive the first image data generated by the first camera unit during flight of the first aerial device and to identify at least one of a broken component of the track or an obstacle on the track based on comparing the first image data to a benchmark visual profile of the track.

9. The system of claim 1, wherein the one or more image analysis processors are configured to be disposed off-board the first aerial device, and further comprising a communication device configured to be disposed onboard the first aerial device to wirelessly communicate the first image data to the one or more image analysis processors, the communication device comprising one or more communication processors configured to communicate the first image data using one or more of L-band communication or S-band communication.

10. A method comprising:

generating first image data from a first camera unit disposed onboard a first aerial device configured to land on a non-aerial vehicle during movement of the non-aerial vehicle along a route;

the first image data generated during flight of the first aerial device above the route being traveled by the non-aerial vehicle;

examining, using one or more image analysis processors, the first image data to identify a hazard disposed ahead of the non-aerial vehicle along a direction of travel of the non-aerial vehicle based on the first image data; and

controlling the flight of the first aerial device by autonomously preventing the first aerial device from flying outside of the three dimensional movement envelope disposed ahead of the non-aerial vehicle during the movement of the non-aerial vehicle.

11. The method of claim 10, wherein the first image data represents a segment of the route disposed ahead of the non-aerial vehicle and the hazard is identified by the one or more image analysis processors by examining the segment of the route based on the first image data.

12. The method of claim 10, further comprising wirelessly communicating the first image data from the first aerial device to the one or more image analysis processors during flight of the first aerial device off of the non-aerial vehicle.

13. The method of claim 10, wherein the first image data represents an area ahead of the first aerial device along a direction of travel of the first aerial device, and further comprising one or more of:

generating second image data from a downward-facing camera unit disposed onboard the first aerial device and representative of a portion of the route that is beneath the first aerial device;

or

generating third image data from a rearward-facing camera unit disposed onboard the first aerial device and representative of a portion of the route that is between the first aerial device and the non-aerial vehicle.

14. The method of claim 13, further comprising:

communicating one or more of the second image data or the third image data to the one or more image analysis processors during flight of the first aerial device; and

examining, using the one or more image analysis processors, the one or more of the second image data or the third image data to identify the hazard.

15. The method of claim 10, wherein the hazard is identified by the one or more image analysis processors as at least one of a broken component of the route or an obstacle on the route based on the first image data.

16. The method of claim 10, wherein the non-aerial vehicle is a rail vehicle, the route is a track on which the rail vehicle is configured to travel, and the one or more image analysis processors are disposed onboard the rail vehicle, and further comprising:

wirelessly communicating the first image data to the one or more image analysis processors during flight of the first aerial device off of the rail vehicle;

receiving the first image data generated by the first camera unit during flight of the first aerial device; and

identifying at least one of a broken component of the track or an obstacle on the track based on comparing the first image data to a benchmark visual profile of the track.

17. A system comprising:

a first aerial device configured to land and dock to a non-aerial vehicle as the non-aerial vehicle moves along a route, the first aerial device also configured to fly off of the non-aerial vehicle and fly ahead of the non-aerial vehicle along a direction of travel of the non-aerial vehicle along the route;

a first camera unit configured to be disposed onboard the first aerial device and to generate first image data representative of a portion of the route ahead of the non-aerial vehicle during movement of the non-aerial vehicle along the route;

a second aerial device configured to land and dock to the non-aerial vehicle as the non-aerial vehicle moves along a route, the second aerial device also configured to fly off of the non-aerial vehicle and fly ahead of the non-aerial vehicle along the direction of travel of the non-aerial vehicle along the route;

a second camera unit configured to be disposed onboard the second aerial device and to generate second image data representative of the portion of the route ahead of the non-aerial vehicle during movement of the non-aerial vehicle along the route;

one or more image analysis processors configured to be disposed onboard the non-aerial vehicle and to receive the first image data and the second image data, the one or more image analysis processors configured to identify a hazard along the route ahead of the non-aerial vehicle; and

a control unit configured to control flight of the first aerial device and the second aerial device, the control unit also configured to prevent the first aerial device and the second aerial device from flying outside of a three dimensional movement envelope defined ahead of the non-aerial vehicle.

18. The system of claim 17, wherein the one or more image analysis processors are configured to one or more of:

generate a warning signal to notify an operator of the non-aerial vehicle of the hazard or automatically control movement of the non-aerial vehicle responsive to the hazard being identified.

19. The system of claim 17, wherein only one of the first aerial device or the second aerial device flies off of the non-aerial vehicle at a time.

20. The system of claim 17, wherein the control unit is configured to change a size of the movement envelope based on one or more of a speed of the non-aerial vehicle, a coefficient of friction between the route and the non-aerial vehicle, a grade of the route, or a radius of curvature of the route.

21. The system of claim 17, wherein:

the non-aerial vehicle is a rail vehicle, and the route is a track on which the rail vehicle is configured to travel;

the one or more image analysis processors are configured to be disposed onboard the rail vehicle and one or more of the first aerial device or the second aerial device is configured to wirelessly communicate one or more of the first image data or the second image data to the one or more image analysis processors during flight of the one or more of the first aerial device or the second aerial device off of the rail vehicle; and

the one or more image analysis processors are configured to receive the one or more of the first image data or the second image data generated by one or more of the first camera unit or the second camera unit during flight of the one or more of the first aerial device or the second aerial device and to identify at least one of a broken component of the track or an obstacle on the track based on comparing the one or more of the first image data or the second image data to a benchmark visual profile of the track.