Obstacle detection system

• US 6,163,755 A
• Filed: 06/11/1999
• Issued: 12/19/2000
• Est. Priority Date: 02/27/1996
• Status: Expired due to Fees

• Current Assignee: Israel Aerospace Industries Limited (Government of Israel), Thinkware Limited
• Original Assignee: Israel Aerospace Industries Limited (Government of Israel), Thinkware Limited
• Inventors: Auerbach, Jacob, Peer, Arik, Sverdlov, Erez, Baum, Abraham
• Primary Examiner(s): NGUYEN, TAN QUANG
• Application Number:
  US09/125,626
• Time in Patent Office: 557 Days
• Field of Search: 701/301, 701/19, 701/28, 701/117, 340/901, 340/903, 340/937, 340/943, 340/435, 340/436, 246/170, 246/191, 246/182 R, 342/357, 342/455
• US Class Current: 701/301
• CPC Class Codes: B61L 2205/04 Satellite based navigation systems, e.g. global positioning system [GPS]
• CPC Class Codes: B61L 23/041 Obstacle detection
• CPC Class Codes: B61L 23/044 Broken rails

Abstract

A system for alerting a driver of a vehicle of the presence of an obstacle in a track of the vehicle, comprising a sensor mounted on the vehicle for producing at least one sensor signal representative of a predetermined field of view of the track in front of the vehicle, and an obstacle detection device coupled to the sensor for processing the at least one sensor signal produced thereby so as to detect an obstacle in the track and produce an obstacle detect signal consequent thereto. An obstacle avoidance device is mounted in the vehicle and coupled to the obstacle detection device and is responsive to the obstacle detect signal for producing an obstacle avoidance signal. According to a preferred embodiment, the track is a rail track, the vehicle is a railway engine and the sensor includes a video camera for imaging the track. The resulting image is processed so as to detect a potential obstacle on the tracks allowing the brakes to be applied either manually or automatically.

Claims

1. A system for alerting a controller of a track-led vehicle of the presence of an obstacle in a track of said vehicle, the system comprising:

at least one sensor mounted on the vehicle for sensing a field of view of the track in front of the vehicle so as to produce successive sensor signals each representative of a respective successive section of track ahead of the vehicle,

an obstacle detection device coupled to the sensor means for processing said successive sensor signals so as to detect therefrom a discontinuity in the track and to produce an obstacle detect signal consequent thereto,

an obstacle avoidance device mounted in the vehicle and coupled to the obstacle detection device and being responsive to the obstacle detect signal for producing an obstacle avoidance signal, and

a memory containing pre-stored obstacle data indicative of recognizable obstacle characteristics;

the obstacle detection device being coupled to the memory for comparing the at least one sensor signal with the pre-stored obstacle data so as to produce the obstacle detect signal consequent to a match.

2. The system according to claim 1, wherein the at least one sensor includes a video camera having means for automatically directing the video camera towards the track for producing a video image thereof, and

the obstacle detection device is coupled to the video camera for processing the video image produced thereby so as to detect said discontinuity in the video image of the track indicative of an obstacle on the track;

there being further included a video monitor coupled to the video camera for displaying said video image.

3. The system according to claim 2, wherein the video camera is mounted on gimbals.

4. The system according to claim 2, wherein the video camera is a day/night video camera.

5. The system according to claim 2, wherein there are coupled to the video monitor a control means for controlling at least one feature of the displayed video image.

6. The system according to claim 2, further including a video recorder coupled to the video monitor for recording the video image.

7. The system according to claim 2, further including:

a receiver coupled to the obstacle detection means for receiving at least one auxiliary video image of a section of the vehicle'"'"'s track outside of the field of view of said video camera, and

at least one post or tower having mounted thereon a respective auxiliary video camera for imaging a region of said track within its field of view and producing a corresponding auxiliary video image, and

a transmitter coupled to the auxiliary video camera for transmitting the auxiliary video image to the receiver.

8. The system according to claim 7, wherein the auxiliary video camera is a day/night video camera.

9. The system according to claim 1, wherein:

the controller is a driver of the vehicle, and

the obstacle avoidance device includes an alarm for warning the driver of a possible impending collision.

10. The system according to claim 1, wherein:

the controller is a driver of the vehicle, and

the obstacle avoidance device includes an automatic brake for automatically operating brakes in the vehicle.

11. The system according to claim 10, wherein:

the at least one sensor signal is transmitted to, and processed by a monitoring and control center in real time in order to decide whether or not to apply the brakes, and

the monitoring and control center includes means for relaying a brake control signal to the vehicle for automatically operating said brakes.

12. The system according to claim 1, wherein:

the vehicle is automatically controlled by said controller, and

the obstacle avoidance device includes an automatic brake for automatically operating brakes in the vehicle.

13. The system according to claim 12, wherein:

the at least one sensor signal is transmitted to, and processed by a monitoring and control center in real time in order to decide whether or not to apply the brakes, and

the monitoring and control center includes means for relaying a brake control signal to the vehicle for automatically operating said brakes.

14. The system according to claim 1, wherein the at least one sensor includes a radar in addition to an electro-optical imaging system for improving the detection of obstacles in adverse weather conditions.

15. The system according to claim 14, further including reflectors placed between or alongside the rails for detection by the radar so that an obstacle hides the reflectors from the radar thus preventing their detection.

16. The system according to claim 1, wherein the vehicle is a railway engine and the track is a rail track.

17. The system according to claim 16, wherein the at least one sensor includes an imaging device mounted on the engine and automatically directed towards the track for producing an image thereof, and

the obstacle detection device is coupled to the imaging device for processing the image produced thereby so as to detect a discontinuity in the image of the track and produce the obstacle detect signal consequent thereto;

there being further included a display monitor coupled to the imaging device for displaying said video image.

18. The system according to claim 17, further including:

a database for storing therein coordinates of background objects in a region of the track,

a Global Positioning System (GPS) mounted in the engine for determining a location in 3-dimensional space thereof, and

directing means coupled to the imaging means and to the Global Positioning System for directing the imaging means towards the track so as to image an area thereof having a known location in 3-dimensional space;

the obstacle detection means being responsively coupled to the database for extracting from the database the coordinates of background objects in a region of the imaged area so as to eliminate said background objects as potential obstacles thereby reducing false alarms.

19. The system according to claim 17, wherein the obstacle detection device includes:

a database construction unit for preparing a set of pictures, including potential obstacles, imaged from a specified distance and from various angles so as to construct dynamically a database of potential obstacles,

a locating unit for locating a rail in said image, and

a comparator for comparing a segment of said image within an area of the rail with at least some of the pictures in said database so as to determine whether said area of the image corresponds to an obstacle on the rail.

20. The system according to claim 19, wherein the comparator is a neural network for providing at an output thereof a decision as to whether or not an obstacle were detected on the rails within said area.

21. The system according to claim 17, wherein:

the obstacle detection device is adapted to identify personnel on the track for producing the obstacle detection signal,

and there is further provided;

a transmitter coupled to the obstacle detection device and responsive to the obstacle detection signal for transmitting a warning signal to a receiver/alarm unit carried by the personnel so as to warn the personnel of an approaching train.

22. The system according to claim 1, for automatically guiding a vehicle along a track defined by a visible or otherwise detectable line on a road surface.

23. A system for alerting a controller of a track-led vehicle of the presence of an obstacle in a track of said vehicle, the system comprising:

at least one sensor including a video camera mounted on the vehicle for sensing a field of view of the track in front of the vehicle so as to produce successive video images thereof each representative of a respective successive section of track ahead of the vehicle,

an obstacle detection device coupled to the video camera for processing successive video images produced thereby so as to detect therefrom a discontinuity in the video image of the track indicative of an obstacle on the track and to produce an obstacle detect signal consequent thereto,

an obstacle avoidance device mounted in the vehicle and coupled to the obstacle detection device and being responsive to the obstacle detect signal for producing an obstacle avoidance signal,

a video monitor coupled to the video camera for displaying said video image, and

a directing unit coupled to the video camera for automatically directing the video camera towards the track, said directing unit comprising;

an apparent movement device for determining apparent movement of the track between successive frames of video image data each corresponding to a respective section of the track, and

an adjusting device coupled to the apparent movement device and to the video camera for automatically adjusting the orientation of the video camera in order to compensate for said apparent movement.

24. The system according to claim 23, wherein the apparent movement device comprises:

a comparator for comparing said successive frames of video data so as to determine those areas which are common to a preceding and subsequent frame,

a derivation device coupled to the comparator for deriving that part of the subsequent frame corresponding to the continuation of the track from the preceding frame so as to identify the point in the preceding frame where the subsequent frame commences, and

a computer coupled to the derivation device for computing the direction of a far end of the track in the subsequent frame relative to a start thereof so as thereby to derive the continuation of the subsequent frame;

the adjusting device being responsive to the computer for cyclically directing the video camera to the start of the subsequent frame, corresponding to the end of the preceding frame.

25. The system according to claim 23, further including:

a receiver coupled to the obstacle detection means for receiving at least one auxiliary video image of a section of the vehicle'"'"'s track outside of the field of view of said video camera, and

at least one post or tower having mounted thereon a respective auxiliary video camera for imaging a region of said track within its field of view and producing a corresponding auxiliary video image, and

a transmitter coupled to the auxiliary video camera for transmitting the auxiliary video image to the receiver.

26. The system according to claim 25, further including a steering unit coupled to the auxiliary video camera for operating under control of the controller so as vary the field of view of the auxiliary video camera.

27. The system according to claim 25, wherein the auxiliary video camera is a day/night video camera.

28. The system according to claim 23, wherein the video camera is a day/night video camera.

29. The system according to claim 23, wherein:

the controller is a driver of the vehicle, and

the obstacle avoidance device includes an alarm for warning the driver of a possible impending collision.

30. The system according to claim 23, wherein:

the controller is a driver of the vehicle, and

the obstacle avoidance device includes an automatic brake for automatically operating brakes in the vehicle.

31. The system according to claim 30, wherein:

the at least one sensor signal is transmitted to, and processed by a monitoring and control center in real time in order to decide whether or not to apply the brakes, and

the monitoring and control center includes means for relaying a brake control signal to the vehicle for automatically operating said brakes.

32. The system according to claim 23, wherein:

the vehicle is automatically controlled by said controller, and

the obstacle avoidance device includes an automatic brake for automatically operating brakes in the vehicle.

33. The system according to claim 32, wherein:

the at least one sensor signal is transmitted to, and processed by a monitoring and control center in real time in order to decide whether or not to apply the brakes, and

the monitoring and control center includes means for relaying a brake control signal to the vehicle for automatically operating said brakes.

34. The system according to claim 23, wherein the at least one sensor includes a radar in addition to an electro-optical imaging system for improving the detection of obstacles in adverse weather conditions.

35. The system according to claim 34, further including reflectors placed between or alongside the rails for detection by the radar so that an obstacle hides the reflectors from the radar thus preventing their detection.

36. The system according to claim 23, wherein the vehicle is a railway engine and the track is a rail track.

37. The system according to claim 36, wherein the at least one sensor includes an imaging device mounted on the engine and automatically directed towards the track for producing an image thereof, and

the obstacle detection device is coupled to the imaging device for processing the image produced thereby so as to detect a discontinuity in the image of the track and produce the obstacle detect signal consequent thereto;

there being further included a display monitor coupled to the imaging device for displaying said video image.

38. The system according to claim 37, further including:

a database for storing therein coordinates of background objects in a region of the track,

a Global Positioning System (GPS) mounted in the engine for determining a location in 3-dimensional space thereof, and

directing means coupled to the imaging means and to the Global Positioning System for directing the imaging means towards the track so as to image an area thereof having a known location in 3-dimensional space;

the obstacle detection means being responsively coupled to the database for extracting from the database the coordinates of background objects in a region of the imaged area so as to eliminate said background objects as potential obstacles thereby reducing false alarms.

39. The system according to claim 37, wherein the obstacle detection device includes:

a database construction unit for preparing a set of pictures, including potential obstacles, imaged from a specified distance and from various angles so as to construct dynamically a database of potential obstacles,

a locating unit for locating a rail in said image, and

a comparator for comparing a segment of said image within an area of the rail with at least some of the pictures in said database so as to determine whether said area of the image corresponds to an obstacle on the rail.

40. The system according to claim 39, wherein the comparator is a neural network for providing at an output thereof a decision as to whether or not an obstacle were detected on the rails within said area.

41. The system according to claim 37, wherein:

the obstacle detection device is adapted to identify personnel on the track for producing the obstacle detection signal,

and there is further provided;

a transmitter coupled to the obstacle detection device and responsive to the obstacle detection signal for transmitting a warning signal to a receiver/alarm unit carried by the personnel so as to warn the personnel of an approaching train.

42. The system according to claim 23, for automatically guiding a vehicle along a track defined by a visible or otherwise detectable line on a road surface.

43. A system for alerting a controller of a track-led vehicle of the presence of an obstacle in a track of said vehicle, the system comprising:

at least one sensor including a video camera mounted on the vehicle for sensing a field of view of the track in front of the vehicle so as to produce successive video images thereof each representative of a respective successive section of track ahead of the vehicle,

an obstacle detection device coupled to the video camera for processing successive video images produced thereby so as to detect therefrom a discontinuity in the video image of the track indicative of an obstacle on the track and to produce an obstacle detect signal consequent thereto,

an obstacle avoidance device mounted in the vehicle and coupled to the obstacle detection device and being responsive to the obstacle detect signal for producing an obstacle avoidance signal,

a video monitor coupled to the video camera for displaying said video image, and

a directing unit coupled to the video camera for automatically directing the day/night video camera towards the track,

a receiver coupled to the obstacle detection unit for receiving at least one auxiliary video image of a section of the vehicle'"'"'s track outside of the field of view of said day/night video camera,

at least one post or tower having mounted thereon a respective auxiliary video camera for imaging a region of said track within its field of view and producing a corresponding auxiliary video image,

a transmitter coupled to the auxiliary video camera for transmitting the auxiliary video image to the receiver, and

a steering unit coupled to the auxiliary video camera for operating under control of the controller so as vary the field of view of the auxiliary video camera.

44. The system according to claim 43, wherein at least one of the video camera and the auxiliary video camera is a day/night video camera.

45. The system according to claim 43, further including:

a memory containing pre-stored obstacle data indicative of recognizable obstacle characteristics;

the obstacle detection device being coupled to the memory for comparing the at least one sensor signal with the pre-stored obstacle data so as to produce the obstacle detect signal consequent to a match.

46. The system according to claim 43, wherein the directing unit comprises:

an apparent movement device for determining apparent movement of the track between successive frames of video image data each corresponding to a respective section of the track, and

an adjusting device coupled to the apparent movement device and to the video camera for automatically adjusting the orientation of the video camera in order to compensate for said apparent movement.

47. A system for alerting a controller of a railway engine of the presence of an obstacle on a railway track thereof, the system comprising:

at least one sensor including a video camera mounted on the railway engine for sensing a field of view of the railway track in front of the railway engine so as to produce successive video images thereof each representative of a respective successive section of railway track ahead of the railway engine,

an obstacle detection device coupled to the video camera for processing successive video images produced thereby so as to detect therefrom a discontinuity in the video image of the railway track indicative of an obstacle on the railway track and to produce an obstacle detect signal consequent thereto,

an obstacle avoidance device mounted in the railway engine and coupled to the obstacle detection device and being responsive to the obstacle detect signal for producing an obstacle avoidance signal,

a video monitor coupled to the video camera for displaying said video image,

a directing unit coupled to the video camera for automatically directing the video camera towards the track,

a database construction unit for preparing a set of pictures, including potential obstacles, imaged from a specified distance and from various angles so as to construct dynamically a database of potential obstacles,

a locating unit for locating a rail in said image, and

a comparator for comparing a segment of said image within an area of the rail with at least some of the pictures in said database so as to determine whether said area of the image corresponds to an obstacle on the rail.

48. The system according to claim 47, wherein the at least one sensor includes an imaging device mounted on the engine and automatically directed towards the track for producing an image thereof, and

the obstacle detection device is coupled to the imaging device for processing the image produced thereby so as to detect a discontinuity in the image of the track and produce the obstacle detect signal consequent thereto;

there being further included a display monitor coupled to the imaging device for displaying said video image.

49. The system according to claim 48, further including:

a database for storing therein coordinates of background objects in a region of the track,

a Global Positioning System (GPS) mounted in the engine for determining a location in 3-dimensional space thereof, and

directing means coupled to the imaging means and to the Global Positioning System for directing the imaging means towards the track so as to image an area thereof having a known location in 3-dimensional space;

the obstacle detection means being responsively coupled to the database for extracting from the database the coordinates of background objects in a region of the imaged area so as to eliminate said background objects as potential obstacles thereby reducing false alarms.

50. The system according to claim 48, wherein:

the obstacle detection device is adapted to identify personnel on the track for producing the obstacle detection signal,

and there is further provided;

a transmitter coupled to the obstacle detection device and responsive to the obstacle detection signal for transmitting a warning signal to a receiver/alarm unit carried by the personnel so as to warn the personnel of an approaching train.

51. The system according to claim 47, further including:

a memory containing pre-stored obstacle data indicative of recognizable obstacle characteristics;

the obstacle detection device being coupled to the memory for comparing the at least one sensor signal with the pre-stored obstacle data so as to produce the obstacle detect signal consequent to a match.

52. The system according to claim 47, wherein the comparator is a neural network for providing at an output thereof a decision as to whether or not an obstacle were detected on the rails within said area.

53. A method for alerting a controller of a track-led vehicle of the presence of an obstacle in a track of said vehicle comprising at least one rail, the method comprising the steps of:

(1) automatically directing a video camera towards the track for producing successive frames of video image data each representative of a successive section of track ahead of the vehicle, by;

a) determining apparent movement of the at least one rail of the track between successive frames of video image data each corresponding to a respective section of the track, and

b) automatically adjusting the orientation of the video camera in order to compensate for said apparent movement,

(2) processing said successive video images so as to detect therefrom a discontinuity in the at least one rail of said track, and

(3) producing an obstacle detect signal consequent thereto.

54. The method according to claim 53, wherein the step of determining apparent movement of the track comprises:

(1) comparing said successive frames of video data so as to determine those areas which are common to a preceding and subsequent frame,

(2) deriving that part of the subsequent frame corresponding to the continuation of the track from the preceding frame so as to identify the point in the preceding frame where the subsequent frame commences, and

(3) computing the direction of a far end of the track in the subsequent frame relative to a start thereof so as thereby to derive the continuation of the subsequent frame.

55. The method according to claim 53, further including the steps of:

(4) determining the position of each rail in the section of track,

(5) defining around the track'"'"'s position a window containing a segment of each rail of the section of track as seen from a pre-determined range, and

(6) passing each image produced by the sensor and contained within the window though a neural network so as to provide at an output thereof a decision as to whether or not an obstacle were detected on the section of track within the window.

56. The method according to claim 55, wherein the step of determining the position of each rail in the section of track includes:

a) obtaining successive frames each containing respective segments of track at successive instants of time, and

b) comparing each frame with a subsequent frame in order to determine those areas which are common to both frames thereby deriving that part of the subsequent frame corresponding to a continuation of the rail from the preceding frame.