Multi-modal cockpit interface for improved airport surface operations

• US 7,737,867 B2
• Filed: 04/04/2007
• Issued: 06/15/2010
• Est. Priority Date: 04/13/2006
• Status: Expired due to Fees

• Current Assignee: United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration
• Original Assignee: United States Administrator of National Aeronautics and Space Administration
• Inventors: Bailey, Randall E., Williams, Steven P., Arthur, Jarvis J., Prinzel, Lawrence J. III, Kramer, Lynda J.
• Primary Examiner(s): Previl; Daniel
• Application Number:
  US11/696,333
• Publication Number:
  US 20070241936A1
• Time in Patent Office: 1,168 Days
• Field of Search: 340/980, 340/945, 340/947, 340/971, 340904-905, 340909-910, 340/915, 340/946, 340/948, 340/951, 340/959, 340/963, 340972-973, 340/982, 701/120, 701/3, 701/16, 701/301, 701/4, 701/10
• US Class Current: 340/980
• CPC Class Codes: G01C 23/00 Combined instruments indicating more than one navigational value, e.g. for aircraft; Combined measuring devices for measuring two or more variables of movement, e.g. distance, speed or acceleration
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G08G 5/0021 located in the aircraft
• CPC Class Codes: G08G 5/065 Navigation or guidance aids, e.g. for taxiing or rolling
• CPC Class Codes: G09G 2320/0626 for control of overall brightness

Abstract

A system for multi-modal cockpit interface during surface operation of an aircraft comprises a head tracking device, a processing element, and a full-color head worn display. The processing element is configured to receive head position information from the head tracking device, to receive current location information of the aircraft, and to render a virtual airport scene corresponding to the head position information and the current aircraft location. The full-color head worn display is configured to receive the virtual airport scene from the processing element and to display the virtual airport scene. The current location information may be received from one of a global positioning system or an inertial navigation system.

Claims

1. A system for multi-modal cockpit interface during surface operation of an aircraft, the system comprising:

a head tracking device;

a processing element configured to, during surface operation of the aircraft, receive head position information from the head tracking device, to receive current location information of the aircraft, and to render a virtual airport scene corresponding to the head position information and the current aircraft location; and

a full-color head worn display configured to receive the virtual airport scene from the processing element and to display the virtual airport scene.

2. The system of claim 1, wherein the processing element receives the current location information from a global positioning system, an inertial navigation system, or a multi-lateration position system.

3. The system of claim 1, wherein the processing element is further configured to receive current location information of other vehicles near the aircraft and to render the other vehicles within the virtual airport scene.

4. The system of claim 1, wherein the processing element is further configured to receive taxi clearance information and wherein the head worn display is further configured to receive the taxi clearance information from the processing element and to display the taxi clearance information.

5. The system of claim 4, wherein the processing element receives the taxi information via one of a voice link or a data link, and wherein, if the taxi information is received via a voice link, the processing element is further configured to convert the taxi information into text.

6. The system of claim 4, wherein the taxi clearance information comprises a plurality of sequential instructions, and wherein the processing element is configured to determine, based on the current aircraft location, which of the sequential instructions has been performed and which is next to be performed such that the next sequential instruction to be performed may be indicated on the display of the taxi clearance information.

7. The system of claim 4, wherein the processing element is further configured to render a top-down airport map illustrating a taxi route based on the taxi clearance information, and wherein the head worn display is further configured to receive the top-down airport map from the processing element and to display the top-down airport map.

8. The system of claim 7, wherein the processing element is further configured to render an indicator of the current aircraft location on the top-down airport map.

12. The system of claim 1, wherein the processing element is further configured to receive a pilot display preference, and wherein the head worn display is further configured to receive and display one or more of the virtual airport scene, the taxi clearance information, the top-down airport map, and the representation of the taxiway/runway upon which the aircraft is currently located, based on the received pilot display preference.

9. A system for multi-modal cockpit interface during surface operation of an aircraft, the system comprising:

a head tracking device;

a processing element configured to receive head position information from the head tracking device, to receive current location information of the aircraft, and to render a virtual airport scene corresponding to the head position information and the current aircraft location; and

a full-color head worn display configured to receive the virtual airport scene from the processing element and to display the virtual airport scene,

and wherein the processing element is further configured to receive taxi clearance information and wherein the head worn display is further configured to receive the taxi clearance information from the processing element and to display the taxi clearance information,

wherein the processing element is further configured to render a top-down airport map illustrating a taxi route based on the taxi clearance information, and wherein the head worn display is further configured to receive the top-down airport map from the processing element and to display the top-down airport map,

wherein the processing element is further configured to render a top-down representation of a portion of a taxiway/runway upon which the aircraft is currently located, the portion of the taxiway/runway comprising a predetermined length of taxiway/runway forward of the aircraft, the representation comprising edges of the rendered portion of the taxiway/runway, the representation further comprising an aircraft indicator showing a position of the aircraft relative to the edges, the aircraft indicator comprising wheel assembly indicators showing positions of the wheel assemblies relative to the edges,

and wherein the head worn display is further configured to receive the representation from the processing element and to display the representation.

10. The system of claim 9, wherein the representation further comprises an identifier of the taxiway/runway upon which the aircraft is currently located.

11. The system of claim 9, wherein the representation further comprises forward markings showing a planned route of the aircraft on the rendered portion of the taxiway/runway.

13. A device for multi-modal cockpit interface during surface operation of an aircraft, the device comprising:

a processing element configured to, during surface operation of the aircraft;

receive head position information from a head tracking device;

receive current location information of the aircraft;

render a virtual airport scene corresponding to the head position information and the current aircraft location; and

provide the virtual airport scene to a full-color head worn display capable of displaying the virtual airport scene.

14. The device of claim 13, wherein the processing element receives the current location information from a global positioning system, an inertial navigation system, or a multi-lateration position system.

15. The device of claim 13, wherein the processing element is further configured to receive current location information of other vehicles near the aircraft and to render the other vehicles within the virtual airport scene.

16. The device of claim 13, wherein the processing element is further configured to receive taxi clearance information and to provide the taxi information to the head worn display.

17. The device of claim 16, wherein the processing element receives the taxi information via one of a voice link or a data link, and wherein, if the taxi information is received via a voice link, the processing element is further configured to convert the taxi information into text.

18. The device of claim 16, wherein the taxi clearance information comprises a plurality of sequential instructions, and wherein the processing element is configured to determine, based on the current aircraft location, which of the sequential instructions has been performed and which is next to be performed such that the next sequential instruction to be performed may be indicated on the display of the taxi clearance information.

19. The device of claim 16, wherein the processing element is further configured to render a top-down airport map illustrating a taxi route based on the taxi clearance information, and to provide the top-down airport map to the head worn display.

20. The device of claim 19, wherein the processing element is further configured to render an indicator of the current aircraft location on the top-down airport map.

24. The device of claim 13, wherein the processing element is further configured to receive a pilot display preference, and wherein the processing element is further configured to provide to the head worn display one or more of the virtual airport scene, the taxi clearance information, the top-down airport map, and the representation of the taxiway/runway upon which the aircraft is currently located, based on the received pilot display preference.

21. A device for multi-modal cockpit interface during surface operation of an aircraft, the device comprising:

a processing element configured to;

receive head position information from a head tracking device;

receive current location information of the aircraft;

render a virtual airport scene corresponding to the head position information and the current aircraft location; and

provide the virtual airport scene to a full-color head worn display capable of displaying the virtual airport scene,

and wherein the processing element is further configured to receive taxi clearance information and to provide the taxi information to the head worn display,

wherein the processing element is further configured to render a top-down airport map illustrating a taxi route based on the taxi clearance information, and to provide the top-down airport map to the head worn display,

wherein the processing element is further configured to render a top-down representation of a portion of a taxiway/runway upon which the aircraft is currently located, the portion of the taxiway/runway comprising a predetermined length of taxiway/runway forward of the aircraft, the representation comprising edges of the rendered portion of the taxiway/runway, the representation further comprising an aircraft indicator showing a position of the aircraft relative to the edges, the aircraft indicator comprising wheel assembly indicators showing positions of the wheel assemblies relative to the edges,

and wherein the processing element is configured to provide the representation to the head worn display.

22. The device of claim 21, wherein the representation further comprises an identifier of the taxiway/runway upon which the aircraft is currently located.

23. The device of claim 21, wherein the representation further comprises forward markings showing a planned route of the aircraft on the rendered portion of the taxiway/runway.

25. A method for multi-modal cockpit interface during surface operation of an aircraft, the method comprising the steps of:

receiving, in a processing element during surface operation of the aircraft, head position information from a head tracking device;

receiving, in the processing element during surface operation of the aircraft, current location information of the aircraft;

rendering, in the processing element during surface operation of the aircraft, a virtual airport scene corresponding to the head position information and the current aircraft location;

receiving, in a full-color head worn display during surface operation of the aircraft, the virtual airport scene from the processing element; and

displaying, during surface operation of the aircraft, the virtual airport scene on the head worn display.

26. The method of claim 25, wherein the processing element receives the current location information from one of a global positioning system, an inertial navigation system, or a multi-lateration position system.

27. The method of claim 25, further comprising the steps of:

receiving, in the processing element, current location information of other vehicles near the aircraft; and

rendering, in the processing element, the other vehicles within the virtual airport scene.

28. The method of claim 25, further comprising the steps of:

receiving, in the processing element, taxi clearance information;

receiving, in the head worn display, the taxi clearance information from the processing element; and

displaying the taxi clearance information on the head worn display.

29. The method of claim 28, wherein the processing element receives the taxi information via one of a voice link or a data link, and wherein, if the processing element receives the taxi information via a voice link, the method further comprises the step of converting the taxi information into text in the processing element.

30. The method of claim 28, wherein the taxi clearance information comprises a plurality of sequential instructions, and wherein the method further comprises the step of determining, in the processing element and based on the current aircraft location, which of the sequential instructions has been performed and which is next to be performed such that the next sequential instruction to be performed may be indicated on the display of the taxi clearance information.

31. The method of claim 28, further comprising the steps of:

rendering, in the processing element, a top-down airport map illustrating a taxi route based on the taxi clearance information;

receiving, in the head worn display, the top-down airport map from the processing element; and

displaying the top-down airport map on the head worn display.

32. The method of claim 31, further comprising the step of rendering, in the processing element, an indicator of the current aircraft location on the top-down airport map.

36. The method of claim 25, further comprising the steps of:

receiving, in the processing element, a pilot display preference; and

receiving and displaying, in the head worn display, one or more of the virtual airport scene, the taxi clearance information, the top-down airport map, and the representation of the taxiway/runway upon which the aircraft is currently located, based on the received pilot display preference.

33. A method for multi-modal cockpit interface during surface operation of an aircraft, the method comprising the steps of:

receiving, in a processing element, head position information from a head tracking device;

receiving, in the processing element, current location information of the aircraft;

rendering, in the processing element, a virtual airport scene corresponding to the head position information and the current aircraft location;

receiving, in a full-color head worn display, the virtual airport scene from the processing element;

displaying the virtual airport scene on the head worn display;

receiving, in the processing element, taxi clearance information;

receiving, in the head worn display, the taxi clearance information from the processing element;

displaying taxi clearance information on the head worn display;

rendering, in the processing element, a top-down airport map illustrating a taxi route based on the taxi clearance information;

receiving, in the head worn display, the top-down airport map from the processing element;

displaying the top-down airport map on the head worn display;

rendering, in the processing element, a top-down representation of a portion of a taxiway/runway upon which the aircraft is currently located, the portion of the taxiway/runway comprising a predetermined length of taxiway/runway forward of the aircraft, the representation comprising edges of the rendered portion of the taxiway/runway, the representation further comprising an aircraft indicator showing a position of the aircraft relative to the edges, the aircraft indicator comprising wheel assembly indicators showing positions of the wheel assemblies relative to the edges;

receiving, in the head worn display, the representation from the processing element; and

displaying the representation on the head worn display.

34. The method of claim 33, wherein the representation further comprises an identifier of the taxiway/runway upon which the aircraft is currently located.

35. The method of claim 33, wherein the representation further comprises forward markings showing a planned route of the aircraft on the rendered portion of the taxiway/runway.