METHODS, APPARATUS AND SYSTEMS FOR ENHANCED SYNTHETIC VISION AND MULTI-SENSOR DATA FUSION TO IMPROVE OPERATIONAL CAPABILITIES OF UNMANNED AERIAL VEHICLES

• US 20080215204A1
• Filed: 12/06/2007
• Published: 09/04/2008
• Est. Priority Date: 12/06/2006
• Status: Active Grant

• Current Assignee: Honeywell International Inc.
• Original Assignee: Mercury Computer Systems Incorporated
• Inventors: Roy, Philippe, Yu, Jun, Linden, David S.
• Granted Patent: US 7,747,364 B2
• Time in Patent Office: Days
• US Class Current: 701/28
• CPC Class Codes: G05D 1/0044 by providing the operator with a computer generated representation of the environment of the vehicle, e.g. virtual reality, maps maps used for automatic navigation G05D1/0274; flight directors G01C23/005
• CPC Class Codes: H04B 7/18504 Aircraft used as relay or high altitude atmospheric platform

Abstract

The invention provides, in some aspects, improved methods, apparatus and systems for unmanned aerial vehicle (UAV) operation that utilize multiple data links between a UAV and a control station in order to transmit control and actionable intelligence data. Such methods, apparatus and systems can be used, for example, to monitor a selected environment (e.g., an oil field or other terrain/environment of interest). In a related aspect, such data links comprise satellite communication channels.

Claims

1. A system for unmanned vehicle operation, comprising

A. a control station,

B. a craft disposed remotely from the control station,

C. an image acquisition device coupled to the craft and arranged for generating one or more images of one or more scenes in a vicinity of the craft,

D. a first digital data processor coupled to the image acquisition device, the first digital processor identifying anomalies in one or more scenes in one or more images generated by the image acquisition device,

E. a second digital data processor disposed at the control station in communications coupling with the first digital data processor, the second digital data processor responding to signaling from the first digital data processor indicative of an anomaly by generating an image representative of a scene in which the anomaly was detected, and

F. wherein the first digital data processor is coupled with the second digital data processor by way of any of a low-bandwidth and/or small-antenna satellite link.

7. The system of claim 1, wherein the image acquisition device is a video camera.

10. The system of claim 10, wherein the second digital data processor is disposed at a distance that is beyond line of sight from the craft.

11. A system for unmanned vehicle operation, comprising

A. a control station,

B. a craft disposed remotely from the control station,

C. an image acquisition device coupled to the craft and arranged for generating one or more images of one or more scenes in a vicinity of the craft,

D. a first digital data processor coupled to the image acquisition device, the first digital processor identifying anomalies in one or more scenes in one or more images generated by the image acquisition device,

E. a second digital data processor disposed at the control station in communications coupling with the first digital data processor, the second digital data processor responding to signaling from the first digital data processor indicative of an anomaly by generating an image representative of a scene in which the anomaly was detected, and

F. wherein the image acquisition device is a video camera.

12. The system of claim 11, wherein the first digital data processor and the second digital data processor are coupled via any of a low-bandwidth and/or small-antenna satellite link.

13. The system of claim 12, wherein the craft includes a dish antenna that supports communications comprising the satellite link.

14. The system of claim 12, wherein the disk antenna has a diameter of about 3 inches or less.

15. The system of claim 11, wherein the satellite link supports communications of about 9600 bps (bytes per second) or less.

16. A digital data processor for use in remote surveillance, the digital data processor comprising

A. a first subsystem in communications coupling with a remote craft including an image acquisition device, the first subsystem including logic that communicates command &

control information with the craft,

B. a second subsystem in communications coupling with the first subsystem and with the craft via a satellite communications link, the second subsystem including logic that responds to signaling indicative of an anomaly in a vicinity of the craft by generating an image of scene including a location of that anomaly,

C. wherein the satellite communications link is any of a low-bandwidth and/or small-antenna satellite communications link.

17. The system of claim 16, wherein the disk antenna has a diameter of about 3 inches or less.

18. The system of claim 16, wherein the satellite link supports communications of about 9600 bps (bytes per second) or less.

19. The digital data processor of claim 16 further comprising a third subsystem in communications coupling with the second subsystem, the third subsystem storing images relayed from the craft and/or generated by the second subsystem.

20. The digital data processor of claim 16, wherein the second subsystem comprises a three-dimensional synthetic vision system that generates synthetic images of scenes within a vicinity of the craft that include the anomaly.

21. The digital data processor of claim 20, wherein the synthetic vision system generates synthetic images that highlight a location of the anomaly.

22. The digital data processor of claim 16, comprising any of a terrain, obstacle and/or navigational database.

23. A digital data processor for use on a surveillance aircraft, comprising

A. a video processor that is coupled to an image acquisition device disposed on the craft, the image acquisition device generating one or more images of one or more scenes in a vicinity of the craft,

B. a first digital data processor coupled to the image acquisition device, the first digital processor identifying anomalies in one or more scenes in one or more images generated by the image acquisition device,

C. where the image acquisition device is a video camera.

2. The system of claim 1, wherein the craft includes a dish antenna that supports communications comprising the satellite link.

3. The system of claim 2, wherein the disk antenna has a diameter of about 3 inches or less.

4. The system of claim 2, wherein the disk antenna has a diameter of about 5 inches or less.

5. The system of claim 2, wherein the disk antenna has a diameter of about 10 inches or less.

6. The system of claim 1, wherein the satellite link supports communications of about 9600 bps (bytes per second) or less.

24. The digital data processor of claim 23, wherein the first digital processor utilizes an SSRX (subspace RX) methodology.

25. The digital data processor of claim 24, wherein the first digital processor applies the SSRX methodology to images generated by the video camera.

26. The digital data processor of claim 25, wherein the first digital processor applies the SSRX methodology to RGB color images generated by the video camera.

27. The digital data processor of claim 25, wherein the first digital processor applies the SSRX methodology to identify anomalies whose Mahalanobis distances correlate with actual or suspected alarm instances.

28. The digital data processor of claim 25, wherein the first digital processor performs spatial filtering on images resulting from application of the SSRX methodology.

23-1. A digital data processor for use on a surveillance aircraft, comprising a chassis having

A. plug-and-play logic for sensor payload management,

B. plug-and-play logic for avionics,

C. plug-and-play logic for datalink management and communications,

D. plug-and-play logic for flight management.

24-2. The digital data processor of claim 23, comprising an embedded real-time operating system.

25-3. The digital data processor of claim 24, wherein the real-time operating system supports a master scheduler and one or more of a data flow manager, an emergency handler, a resource manager and a redundancy manager.

26-4. The digital data processor of claim 23, comprising a data channel manager that includes one or more of a connection manager and an encryption/decryption module, each of which is coupled to a data-link interface.

27-5. The digital data processor of claim 23, comprising a flight manager that includes one or more of a metadata generator, a traffic manager, a telemetry generator and a backup autopilot, each of which is coupled to an avionics interface.

28-6. The digital data processor of claim 23, comprising a data processor that executes one or more of a data fusion module, a data analysis module, a data compression module and a data conversion module, each of which is coupled to a sensor interface.

29. The digital data processor of claim 28, wherein the first digital processor utilizes image pyramids to perform said spatial filtering.

8. The system of claim 2, wherein the image acquisition device acquires video images of terrain, airspace, or other scenes in a vicinity of the craft.

9. The system of claim 1, wherein the first digital data processor is disposed on the craft.

30. The digital data processor of claim 28, wherein the first digital processor performs said spatial filtering in order to eliminate any of artifacts and objects of small width.

31. The digital data processor of claim 28, wherein the first digital data processor highlights an image generated from the video camera in accord with results of the spatial filtering.

32. The digital data processor of claim 31, wherein the first digital processor applies the SSRX methodology to RGB color images generated by the video camera.

33. The digital data processor of claim 31, wherein the first digital processor applies the SSRX methodology to identify anomalies whose Mahalanobis distances correlate with actual or suspected alarm instances.

34. The digital data processor of claim 31, wherein the first digital processor performs spatial filtering on images resulting from application of the SSRX methodology.

35. The digital data processor of claim 34, wherein the first digital processor utilizes image pyramids to perform said spatial filtering.

36. The digital data processor of claim 34, wherein the first digital processor performs said spatial filtering in order to eliminate any of artifacts and objects of small width.

37. The digital data processor of claim 34, wherein the first digital data processor highlights an image generated from the video camera in accord with results of the spatial filtering.

38. The system of claim 29, wherein the second digital data processor comprises a first subsystem in communications coupling with a remote craft including an image acquisition device, the first subsystem including logic that communicates command &

control information with the craft.

39. The system of claim 30, wherein the second digital data processor comprises a second subsystem in communications coupling with the first subsystem and with the craft via a satellite communications link, the second subsystem including logic that responds to signaling indicative of an anomaly in a vicinity of the craft by generating an image of scene including a location of that anomaly.

29-7. A system for unmanned vehicle operation, comprising

A. a control station,

B. a craft disposed remotely from the control station,

C. an image acquisition device coupled to the craft and arranged for generating one or more images of one or more scenes in a vicinity of the craft,

D. a first digital data processor coupled to the image acquisition device, the first digital processor identifying anomalies in one or more scenes in one or more images generated by the image acquisition device,

E. a second digital data processor disposed at the control station in communications coupling with the first digital data processor, the second digital data processor responding to signaling from the first digital data processor indicative of an anomaly by generating an image representative of a scene in which the anomaly was detected,

F. wherein the first digital data processor is coupled with the second digital data processor by way of any of a low-bandwidth and/or small-antenna satellite link, and

G. wherein the image acquisition device is a video camera.

30-8. The system of claim 29, wherein the first digital data processor utilizes an SSRX (subspace RX) methodology.

31-9. The digital data processor of claim 30, wherein the first digital processor applies the SSRX methodology to images generated by the video camera.

40. A method for unmanned vehicle operation, comprising

A. generating, with an image acquisition device that is coupled to a craft, one or more images of one or more scenes in a vicinity of the craft,

B. identifying, with a first digital data processor, anomalies in one or more scenes in one or more images generated by the image acquisition device,

C. communicating between the first digital data processor and a second digital data processor by way of any of a low-bandwidth and/or small-antenna satellite link, and

C. responding, with a second digital data processor that is disposed remotely from the first digital data processor, to signaling from the first digital data processor indicative of an anomaly by generating an image representative of a scene in which the anomaly was detected.

41. The method of claim 40, wherein the image acquisition device is a video camera.

42. A method of operating a digital data processor for use in remote surveillance, the method comprising

A. communicating between the digital data processor and a remote craft that has an on-board image acquisition device, wherein such communication is conducted over a low-bandwidth and/or small-antenna satellite communications link, and

B. responding to signaling indicative received from the craft of an anomaly in a vicinity thereof by generating an image of scene including a location of that anomaly,

43. The method of claim 42, wherein the image acquisition device is a video camera.

44. A method of operating a digital data processor that is disposed on a surveillance aircraft, comprising

A. acquiring, from image acquisition device that is disposed on the aircraft, one or more images of one or more scenes in a vicinity of the craft,

B. identifying, utilizing the digital data processor, anomalies in one or more scenes in one or more images generated by the image acquisition device, and

C. where the image acquisition device is a video camera.

45. The method of claim 44, wherein step (B) includes executing an SSRX (subspace RX) algorithm on a video image received from the image acquisition device.