Augmented reality traffic control center

• US 7,129,887 B2
• Filed: 04/15/2004
• Issued: 10/31/2006
• Est. Priority Date: 04/15/2004
• Status: Expired due to Fees

• Current Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Original Assignee: LOCKHEED MARTIN MS2
• Inventors: Mitchell, Steven W.
• Primary Examiner(s): Sotomayor, John B.
• Application Number:
  US10/824,410
• Publication Number:
  US 20050231419A1
• Time in Patent Office: 929 Days
• Field of Search: 342 29- 37, 342/41, 342 52- 58, 342/61, 342/90, 342176-186, 342/195, 342/197, 345/8, 345/30, 345/156, 345/158, 345/419, 345/473
• US Class Current: 342/36
• CPC Class Codes: G08G 5/0026 located on the ground
• CPC Class Codes: G08G 5/0082 for monitoring traffic from a ground station radar or analogous systems specially adapted for traffic control G01S13/91

Abstract

In an exemplary embodiment, an augmented reality system for traffic control combines data from a plurality of sensors to display, in real time, information about traffic control objects, such as airplanes. The sensors collect data, such as infrared, ultraviolet, and acoustic data. The collected data is weather-independent due to the combination of different sensors. The traffic control objects and their associated data are then displayed visually to the controller regardless of external viewing conditions. The system also responds to the controller'"'"'s physical gestures or voice commands to select a particular traffic control object for close-up observation or to open a communication channel with the particular traffic control object.

Claims

1. A augmented reality system, comprising:

a display for use by an air traffic controller in an operations center on a water-based craft;

a sensor for collecting data associated with air traffic control objects in a traffic control space;

a computer receiving said data from said sensor, and operative to display said data on said display to the air traffic controller in the operations center in real time; and

means for detecting a physical gesture of the air traffic controller in the operations center selecting an traffic control object displayed on said display of the air traffic controller.

2. The system of claim 1, further comprising means for displaying flight data about said air traffic control objects on said display.

3. The system of claim 2, wherein said flight data comprises at least one of a trajectory, heading, altitude, speed, call sign, and/or flight number.

4. The system of claim 1, further comprising means for opening a communication channel to said selected air traffic control object.

5. The system of claim 1, wherein said display comprises a plurality of displays arranged to simulate a plurality of windows in an air craft carrier primary flight (PriFly) control tower.

6. The system of claim 1, further comprising:

means for opening a computer data file containing data about said selected air traffic control object; and

means for displaying said data as a textual annotation on said display.

7. The system of claim 6, wherein said data about said selected air traffic control object comprises at least one of:

a passenger list or a physical characteristic of said selected air traffic control object.

8. The system of claim 1, wherein said physical gesture to be detected comprises at least one of a hand gesture, a pointing gesture, a voice command, a sustained visual look, and/or a change of visual focus.

9. The system of claim 1, wherein said sensor comprises at least one of an infrared image sensor, a radio frequency image sensor, RADAR, LIDAR, a millimeter wave imaging sensor, an acoustic sensor, a digital infrared camera, a digital ultraviolet camera, an electro-optical camera, digital RADAR, and/or high-resolution radar.

10. The system of claim 1, wherein said display comprises a virtual reality helmet.

11. The system of claim 1, wherein said traffic control space is an aircraft carrier air traffic control space.

12. The system of claim 1, wherein said means for detecting comprise a laser pointer, a gyro-mouse, a video observation system, a data glove, a touch-sensitive screen, and/or a voice observation system.

13. The system of claim 1, wherein said data collected by said sensor comprises non-visual data.

14. The system of claim 1, wherein the water-based craft is an aircraft carrier.

15. A method, comprising:

(a) collecting data associated with air traffic control objects in an air traffic control space;

(b) displaying said data to an air traffic controller in an operation center on a water-based craft in real time; and

(c) detecting a physical gesture of the traffic controller selecting one of said air traffic control objects displayed.

16. The method of claim 15, further comprising:

(d) opening a communication channel with said selected air traffic control object.

17. The method of claim 15, further comprising:

(d) displaying flight data about said air traffic control objects.

18. The method of claim 17, wherein (d) comprises displaying at least one of a trajectory, heading, altitude, speed, call sign, and/or flight number.

19. The method of claim 15, further comprising:

opening a computer data file containing data about said selected air traffic control object; and

displaying said data as a textual annotation on said display.

20. The method of claim 15, wherein (a) comprises collecting said data from at least one of an infrared image sensor, a radio frequency image sensor, RADAR, LIDAR, a millimeter wave imaging sensor, an acoustic sensor, a digital infrared camera, a digital ultraviolet camera, digital RADAR, and electro-optical camera, and/or high-resolution radar.

21. The method of claim 15, wherein (c) comprises detecting at least one of a hand gesture, a pointing gesture, a voice command, a sustained visual look, and/or a change of visual focus.

22. The method of claim 15, wherein (b) comprises displaying said data on at least one of a plurality of displays arranged to simulate a plurality of windows in a flight control tower, and/or a virtual reality helmet.

23. The method of claim 15, wherein (a) comprises collecting non-visual data associated with traffic control objects in the traffic control space.