Systems and methods for incorporating virtual network computing into a cockpit display system and controlling a remote aircraft system with the VNC-incorporated CDS

• US 8,725,318 B1
• Filed: 09/19/2011
• Issued: 05/13/2014
• Est. Priority Date: 05/20/2011
• Status: Active Grant

• Current Assignee: Rockwell Collins, Inc. (Raytheon Technologies Corporation d/b/a RTX)
• Original Assignee: Rockwell Collins, Inc. (Raytheon Technologies Corporation d/b/a RTX)
• Inventors: McCusker, Patrick D.
• Primary Examiner(s): Badii, Behrang
• Assistant Examiner(s): Smith-Stewart, Demetra
• Application Number:
  US13/235,614
• Time in Patent Office: 967 Days
• Field of Search: None
• US Class Current: 701/3
• CPC Class Codes: B64D 43/00 Arrangements or adaptations of instruments arrangements of cameras B64D47/08; aeronautical measuring instruments per se G01C
• CPC Class Codes: G06F 3/1423 controlling a plurality of local displays, e.g. CRT and flat panel display
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: G09G 2380/12 Avionics applications
• CPC Class Codes: H04L 67/08 specially adapted for terminal emulation, e.g. Telnet
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks

Abstract

Novel and non-trivial systems and methods for incorporating virtual network computing (“VNC”) into a cockpit display system (“CDS”) and controlling an aircraft system with a VNC-incorporated CDS are disclosed. A system for incorporating VNC into the CDS is comprised of a VNC server of an aircraft system, at least one first user application (“UA”), a second UA, a pilot input device, and a CDS comprised of, in part, a processing module (“PM”) configured with a corresponding method that employs both an aviation-industry standard protocol (e.g., ARINC 661) and a VNC protocol. A system for controlling an aircraft system with a VNC-incorporated CDS is comprised with a VNC server of an aircraft system, a UA, a pilot input device, a display surface format, and a CDS comprised of, in part, a PM configured with a corresponding method that employs both an aviation-industry standard protocol and a VNC protocol.

Claims

1. A system for incorporating Virtual Network Computing (“

VNC”

) into a cockpit display system, such system comprising;

a VNC server of an aircraft-installed system;

at least one first user application;

a second user application;

a pilot input device; and

a cockpit display system comprised of

a processing module configured to

establish a pixel data set representative of a first image, where

such first image is comprised of at least one non-VNC widget, and

each non-VNC widget is based upon corresponding non-VNC widget data from at least one first user application,

receive first input data from the pilot input device, where

such first input data corresponds to a request to display a second image,

generate first data in response to the first input data, where

such first data is representative of the request to display a second image,

send the first data to the second user application, such that

a VNC client of the second user application is configured to

 

generate second data, where

 

such second data is comprised of a request for remote frame buffer data of the second image,

 

send the second data to the VNC server,

 

receive third data from the VNC server, where

 

such third data is responsive to the request for remote frame buffer data and is comprised of remote frame buffer data of the second image, and

 

send fourth data responsive to the first data, where

 

such fourth data is comprised of first VNC image widget data generated from the third data,

receive the fourth data, and

incorporate the fourth data into the pixel data set, whereby

an image comprised of at least the VNC image widget is represented in the pixel data set.

2. The system of claim 1, wherein

the first data and the fourth data comport to a protocol established by an aviation-industry standard, and

the second data and the third data comport to a protocol established for a VNC system.

3. The system of claim 1, wherein

the processing module is further configured to

activate a layer to which the VNC image widget is assigned, such that

the request to display a second image represented in the first data is comprised of a notice of layer activation, and

the first VNC image widget data is representative of VNC image widget parameters and a request to activate the visibility of the layer.

4. The system of claim 1, wherein

the processing module is further configured to

provide the pixel data set to a display unit, and

the cockpit display system is further comprised of

at least one display unit configured to

present the image represented in the pixel data set.

5. The system of claim 1, wherein the cockpit display system incorporates at least one of the following:

the VNC server at least one first user application, the second user application, and the pilot input device.

6. The system of claim 1, wherein

the processing module is further configured to

receive second input data from the pilot input device, where

such second input data corresponds to a VNC image widget and includes location information of the event,

generate fifth data in response to the second input data, where

such fifth data is representative of both the location information and a state of the VNC image widget event,

send the fifth data to the second user application, such that

the VNC client is further configured to

generate sixth data, where

 

such sixth data is comprised of

 

a notification of the location information and the state, and

 

a request for updated remote frame buffer data of the second image,

send the sixth data to the VNC server,

receive seventh data from the VNC server, where

 

such seventh data is responsive to the request for updated remote frame buffer data and is comprised of updated remote frame buffer data of the second image, and

send eighth data responsive to the fifth data, where

 

such eighth data is updated VNC image widget data generated from the seventh data,

receive the eighth data, and

incorporate the eighth data into the pixel data set, whereby

an image comprised of at least an updated VNC image widget is represented in the pixel data set.

7. The system of claim 6, wherein

the fifth data and the eighth data comport to a protocol established by an aviation-industry standard, and

the sixth data and the seventh data comport to a protocol established for a VNC system.

8. The system of claim 6, wherein

the location information of the second input data corresponds to the location coordinates of the display unit surface, and

the location information of the sixth data corresponds to the location coordinates of the VNC image.

9. A system for controlling an aircraft system with a cockpit display system incorporating Virtual Network Computing (“

VNC”

), such system comprising;

a VNC server of an aircraft-installed system;

a user application;

a pilot input device;

a display surface format comprised of

at least one layer to which at least one non-VNC widget is assigned, and

at least one layer to which at least one VNC image widget is assigned; and

a cockpit display system comprised of

the processing module configured to

establish a pixel data set representative of a first image, where

such first image is comprised of at least one VNC image widget,

receive first input data from the pilot input device, where

such first input data corresponds to a VNC image widget event and includes location information of the event,

generate first data in response to the first input data, where

such first data is representative of both the location information and a state of the VNC image widget event,

send the first data to the user application, such that

a VNC client of the user application is configured to

 

generate second data, where

 

such second data is comprised of

 

a notification of the location information and the state, and

 

a request for updated remote frame buffer data of the VNC image corresponding to the VNC image widget,

 

send the second data to the VNC server,

 

receive third data from the VNC server, where

 

such third data is responsive to the request for updated remote frame buffer data and is comprised of updated remote frame buffer data of the VNC image corresponding to the VNC image widget, and

 

send fourth data responsive to the first data, where

 

such fourth data is comprised of updated VNC image widget data generated from the third data,

receive the fourth data, and

incorporate the fourth data into the pixel data set, whereby

an image comprised of at least an updated VNC image widget is represented in the pixel data set.

10. The system of claim 9, wherein

the non-VNC widget and the VNC image widget are assigned to the same layer,

the first data and the fourth data comport to a protocol established by an aviation-industry standard, and

the second data and the third data comport to a protocol established for a VNC system.

11. The system of claim 9, wherein

the location information of the first input data corresponds to the location coordinates of the display unit surface, and

the location information of the second data corresponds to the location coordinates of the VNC image.

12. The system of claim 9, wherein

the processing module is further configured to

provide the pixel data set to a display unit, and

the cockpit display system is further comprised of

at least one display unit configured to

present the image represented in the pixel data set.

13. The system of claim 9, wherein the cockpit display system incorporates at least one of the following:

the VNC server, the user application, the pilot input device, and the display surface format.

14. A method for incorporating Virtual Network Computing (“

VNC”

) into a cockpit display system, such method comprising;

establishing a pixel data set representative of a first image, where

such first image is comprised of at least one non-VNC widget, and

each non-VNC widget is based upon corresponding non-VNC widget data from at least one first user application;

receiving first input data from a pilot input device, where

such first input data corresponds to a request to display a second image;

generating first data in response to the first input data, where

such first data is representative of the request to display a second image;

sending the first data to a second user application, such that

a VNC client of a second user application is configured to

generate second data, where

such second data is comprised of a request for remote frame buffer data of the second image,

send the second data to a VNC server of an aircraft-installed system,

receive third data from the VNC server, where

such third data is responsive to the request for remote frame buffer data and is comprised of remote frame buffer data of the second image, and

send fourth data responsive to the first data, where

such fourth data is comprised of first VNC image widget data generated from the third data;

receiving the fourth data; and

incorporating the fourth data into the pixel data set, whereby

an image comprised of at least the VNC image widget is represented in the pixel data set.

15. The method of claim 14, wherein

the first data and the fourth data comport to a protocol established by an aviation-industry standard, and

the second data and the third data comport to a protocol established for a VNC system.

16. The method of claim 14, further comprising:

activating a layer to which the VNC image widget is assigned, such that

the request to display a second image represented in the first data is comprised of a notice of layer activation, and

the first VNC image widget data is representative of VNC image widget parameters and a request to activate the visibility of the layer.

17. The method of claim 14, further comprising:

providing the pixel data set to a display unit, whereby

the image represented in the pixel data set is presented.

18. The method of claim 14, further comprising:

receiving second input data from the pilot input device, where

such second input data corresponds to a VNC image widget event and includes location information of the event;

generating fifth data in response to the second input data, where

such fifth data is representative of both the location information and a state of the VNC image widget event;

sending the fifth data to the second user application, such that

the VNC client is further configured to

generate sixth data, where

such sixth data is comprised of

 

a notification of the location information and the state, and

 

a request updated remote frame buffer data of the second image,

send the sixth data to the VNC server,

receive seventh data from the VNC server, where

such seventh data is responsive to the request for updated remote frame buffer data and is comprised of updated remote frame buffer data of the second image, and

send eighth data responsive to the fifth data, where

such eighth data is updated VNC image widget data generated from the seventh data;

receiving the eighth data; and

incorporating the eighth data into the pixel data set, whereby

an image comprised of at least an updated VNC image widget is represented in the pixel data set.

19. The method of claim 18, wherein

the fifth data and the eighth data comport to a protocol established by an aviation-industry standard, and

the sixth data and the seventh data comport to a protocol established for a VNC system.

20. The method of claim 18, wherein

the location information of the second input data corresponds to the location coordinates of the display unit surface, and

the location information of the sixth data corresponds to the location coordinates of the VNC image.

21. A method for controlling an aircraft system with a cockpit display system incorporating Virtual Network Computing (“

VNC”

), such method comprising;

establishing a pixel data set representative of a first image, where

such first image is comprised of at least one VNC image widget and based upon a display surface format comprised of

at least one layer to which at least one non-VNC widget is assigned, and

at least one layer to which at least one VNC image widget is assigned;

receiving first input data from a pilot input device, where

such first input data corresponds to a VNC image widget event and includes location information of the event;

generating first data in response to the first input data, where

such first data is representative of both the location information and a state of the VNC image widget event;

sending the first data to a user application, such that

a VNC client of the user application is configured to

generate second data, where

such second data is comprised of

 

a notification of the location information and the state, and

 

a request for updated remote frame buffer data of the VNC image corresponding to the VNC image widget,

send the second data to a VNC server of an aircraft-installed system,

receive third data from the VNC server, where

such third data is responsive to the request for updated remote frame buffer data and is comprised of updated remote frame buffer data of the VNC image corresponding to the VNC image widget, and

send fourth data responsive to the first data, where

such fourth data is comprised of updated VNC image widget data generated from the third data;

receiving the fourth data; and

incorporating the fourth data into the pixel data set, whereby

an image comprised of at least an updated VNC image widget is represented in the pixel data set.

22. The method of claim 21, wherein

the non-VNC widget and the VNC image widget are assigned to the same layer,

the first data and the fourth data comport to a protocol established by an aviation-industry standard, and

the second data and the third data comport to a protocol established for a VNC system.

23. The method of claim 21, wherein

the location information of the first input data corresponds to the location coordinates of the display unit surface, and

the location information of the second data corresponds to the location coordinates of the VNC image.

24. The method of claim 21, further comprising:

providing the pixel data set to a display unit, whereby

the image represented in the pixel data set is presented.