Methods for remote display of an enhanced image

• US 8,874,284 B2
• Filed: 02/21/2011
• Issued: 10/28/2014
• Est. Priority Date: 06/02/2005
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Sanders-Reed, John N.
• Primary Examiner(s): Nguyen, Cuong H
• Application Number:
  US13/031,330
• Publication Number:
  US 20110187563A1
• Time in Patent Office: 1,345 Days
• Field of Search: 701/3, 340/971, 382/284, 345/419, 345629-630, 345/634
• US Class Current: 701/3
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64U 20/87 Mounting of imaging devices, e.g. mounting of gimbals
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G09G 2340/02 Handling of images in compressed format, e.g. JPEG, MPEG
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2350/00 Solving problems of bandwidth in display systems
• CPC Class Codes: G09G 2370/16 Use of wireless transmission of display information
• CPC Class Codes: G09G 2380/12 Avionics applications

Abstract

A method is provided for providing remote display of an enhanced image. The method receives data from the image source, creates an enhanced image with the received data, and transmits the enhanced image via a wireless link to a display device capable of displaying the enhanced image. The method may create the enhanced image by stitching image data from at least two image sources having different fields of view, fusing image data from at least two image sources having different image modalities, or combining image data from at least one image source with synthetic image data. The method may also modify at least one transmission characteristic based on a bandwidth of the wireless link. The transmission characteristic may be selected from the group comprising field of view, frame rate, resolution, and compression.

Claims

1. A method of providing remote display of an enhanced image, the method comprising:

receiving line of sight data from a display device that corresponds to a position of the display device, wherein the display device comprises one of a helmet mounted display or virtual reality goggles;

receiving data from at least one image source that corresponds to the line of sight data;

creating, with an image processor, an enhanced image with the received data by performing at least one of stitching image data from at least two image sources having different fields of view, fusing image data from at least two image sources having different image modalities, or combining image data from at least one image source with synthetic image data, wherein creating the enhanced image comprises creating the enhanced image corresponding to the line of sight data;

modifying at least one transmission characteristic based on a bandwidth of a wireless link, wherein the transmission characteristic is selected from the group comprising field of view, frame rate, resolution and compression; and

transmitting the enhanced image via the wireless link to the display device configured to display the enhanced image,

wherein modifying the at least one transmission characteristic comprises modifying the transmission characteristics in a different order or to a different extent based upon a portion of the enhanced image that is to be modified, a location of a vehicle body that carries the at least one image source or an operating mode of a vehicle system that includes the at least one image source.

7. The method of claim 1, wherein the vehicle body is an aircraft, and wherein transmitting the enhanced image comprises transmitting the enhanced image via the wireless link to the display device offboard the aircraft.

8. The method of claim 1, wherein the enhanced image is a first enhanced image, wherein the display device is a first display device, wherein the method further comprises creating a second enhanced image and transmitting the second enhanced image via the wireless link to a second display device.

2. A method of providing remote display of an enhanced image, the method comprising:

receiving line of sight data from first and second display devices that correspond to a position of the first and second display devices, wherein at least one of the first and second display devices comprises one of a helmet mounted display or virtual reality goggles;

receiving data from at least one image source of a vehicle system that corresponds to the line of sight data;

creating, with an image processor, first and second enhanced images with the received data by performing at least one of stitching image data from at least two image sources having different fields of view, fusing image data from at least two image sources having different image modalities, or combining image data from at least one image source with synthetic image data, wherein creating the first and second enhanced images comprises creating the first and second enhanced images corresponding to the line of sight data from the first and second display devices, respectively;

modifying at least one transmission characteristic of the first and second enhanced images based on at least one of a bandwidth of a wireless link with the first and second display devices, a location of the image source or an operating mode of the vehicle system, wherein modifying at least one transmission characteristic comprises modifying at least one first transmission characteristic corresponding to the first enhanced image and at least one second transmission characteristic corresponding to the second enhanced image based on a bandwidth of the wireless link, wherein the first and second transmission characteristics are selected from the group comprising field of view, frame rate, resolution and compression; and

transmitting the first and second enhanced images via the wireless link to the first and second display devices, respectively, configured to display the first and second enhanced images.

3. The method of claim 2, wherein the image source is mounted on an aircraft, and wherein transmitting the first and second enhanced images comprises transmitting the first and second enhanced images via the wireless link to the first and second display devices, respectively, offboard the aircraft.

4. The method of claim 2, further comprising:

independently modifying the first and second transmission characteristics based on respective requirements of users of the first and second display devices.

5. The method of claim 2, further comprising:

allocating different portions of the bandwidth of the wireless link to transmission of the first and second enhanced images based upon at least one of a location or an operating mode of the vehicle system.

6. A method of providing remote display of an enhanced image, the method comprising:

receiving line of sight data from first and second display devices that corresponds to a position of the first and second display devices, respectively, wherein at least one of the first and second display devices comprises one of a helmet mounted display or virtual reality goggles;

receiving data from at least one image source that corresponds to the line of sight data;

creating, with an image processor, first and second enhanced images with the received data by performing, for at least one of the enhanced images, at least one of stitching image data from at least two image sources having different fields of view, fusing image data from at least two image sources having different image modalities, or combining image data from at least one image source with synthetic image data, wherein creating the first and second enhanced images comprises creating the first and second enhanced images corresponding to the line of sight data from first and second display devices, respectively;

modifying at least one first transmission characteristic corresponding to the first enhanced image and at least one second transmission characteristic corresponding to the second enhanced image based on a bandwidth of a wireless link, wherein the first and second transmission characteristics are selected from the group comprising field of view, frame rate, resolution, and compression; and

transmitting the first and second enhanced images via the wireless link to the first and second display devices, respectively, configured to display the first and second enhanced images.

9. The method of claim 6, wherein the image source is a component of a vehicle system and is mounted on a vehicle body, and wherein the method further comprises:

modifying at least one of the first and second transmission characteristics based on at least one of a location of the vehicle body or an operating mode of the vehicle system.

10. The method of claim 6, further comprising:

independently modifying the first and second transmission characteristics based on respective requirements of users of the first and second display devices.

11. The method of claim 6, wherein the image source is a component of a vehicle system and is mounted on a vehicle body, and wherein the method further comprises:

allocating different portions of the bandwidth of the wireless link to transmission of the first and second enhanced images based upon at least one of a location of the vehicle body or an operating mode of the vehicle system.

12. The method of claim 6, wherein at least one of the first and second transmission characteristics is modified for only a portion of the respective transmitted enhanced image.