Low bandwidth closed loop imagery control and communication system for remotely piloted vehicle

• US 4,405,943 A
• Filed: 08/19/1981
• Issued: 09/20/1983
• Est. Priority Date: 08/19/1981
• Status: Expired due to Term

• Current Assignee: Harris Corporation Melbourne
• Original Assignee: Harris Corporation (L3Harris Technologies, Inc.)
• Inventors: Kanaly, David B.
• Primary Examiner(s): Griffin, Robert L.
• Assistant Examiner(s): Coles, Edward L.
• Application Number:
  US06/294,190
• Time in Patent Office: 762 Days
• Field of Search: 358/133, 358/109, 358/103
• US Class Current: 375/240.08
• CPC Class Codes: G01S 5/0009 Transmission of position information to remote stations involving assistance data G01S5/0236
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: H04N 7/005 using at least one opto-electrical conversion device
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control

Abstract

In a video imaging and transmission system, the entire field of view of camera equipment aboard a remotely piloted vehicle is imaged as high resolution data. This high resolution data is digitized and stored in memory aboard the remote vehicle. In order to reduce the bandwidth of the information which is to be transmitted to a ground-based operator, only a selected portion of the stored high resolution data, corresponding to the output of oculometer and head position equipment that monitors the operator'"'"'s look angle at a display, is read out from memory and transmitted as high resolution data. The remaining portion of the image provided on the operator'"'"'s display is comprised of low resolution data derived from the high resolution data stored in memory. Because the high resolution data is obtained from memory and not from the camera equipment directly, the camera is effectively decoupled from the data link. As such, the system permits the camera to be a single movable sensor or an array of multiple fixed sensors. It also substantially increases the speed of operation of the system.

Claims

1. A system for displaying, at a local station a high resolution image of a portion of scene optically imaged at a remote station comprising:

at said remote station,

a memory;

first means for generating high resolution data representative of said optically imaged scene to be displayed and storing said data in said memory;

second means for selectively accessing said memory to read out therefrom a first portion of said data as high resolution image information and a second portion of said data as low resolution image information; and

third means for transmitting signals corresponding to said first portion of said data containing said high resolution image information and said second portion of said data containing said low resolution image information to said local station; and

at said local station,

fourth means for generating signals representative of a prescribed portion of said scene for which a high resolution image is to be displayed; and

fifth means for transmitting signals representative of the signals generated by said fourth means to said remote station to cause said second means to selectively access said memory to read out therefrom as high resolution image information that portion of said data corresponding to said prescribed portion of said scene.

2. A system according to claim 1, wherein said second portion of said data read out from said memory by said second means as low resolution image information corresponds to a portion or said scene adjacent to that to which said high resolution image information corresponds.

3. A system according to claim 1, wherein said system further includes, at said local station,

a display device;

and

sixth means for receiving the signals transmitted to said local station by said third means and causing said high resolution image information to be coupled to said display device whereby a high resolution image of said portion of said scene is displayed thereby.

4. A system according to claim 3, wherein said second portion of said data read out from said memory by said second means as low resolution image information corresponds to a portion of said scene adjacent to that to which said high resolution image information corresponds.

5. A system according to claim 4, wherein said sixth means includes means for causing said low resolution image information to be coupled to said display device, whereby a low resolution image of a portion of said scene adjacent to the high resolution portion thereof is displayed by said display device.

6. A system according to claim 3, wherein said fourth means comprises means, responsive to the observation, by an operator, of a location of interest on said display device, for causing the signals generated thereby to be representative of a delimited portion of said scene associated with said location of interest on said display device at which a high resolution image is to be displayed.

7. A system according to claim 6, wherein said fourth means comprises oculometer means and head movement sensing means, respectively responsive to the eye and head movement of said operator, for generating signals representative of the location of observation by said operator on said display device at which said high resolution image is to be displayed.

8. A system according to claim 1, wherein said third and fifth means respectively include means for modifying the signals transmitted thereby so as to cause said signals to be effectively immune to jamming.

9. A system according to claim 1, wherein said system further includes an anti-jam communication link over which signals are transmitted between said remote and local stations.

10. A system according to claim 9, wherein said anti-jam communication link comprises a spread spectrum communication link.

11. A system according to claim 6, wherein said remote station comprises a remotely piloted vehicle the operation of which is controlled by said operator observing said display device.

12. A system according to claim 11, wherein said system further includes an anti-jam communication link over which signals are transmitted between said remote and local stations.

13. A system according to claim 1, where

said second means further includes means for selectively accessing said memory to read out therefrom all the data stored thereby associated with said scene as high resolution data, and

said third means includes means for transmitting signals corresponding to the entirety of said scene as high resolution data to said local station.

14. A system according to claim 13, wherein

said fourth means includes means for selectively generating signals representative of the entirety of said scene for which a high resolution image is to be displayed, and

said fifth means includes means for transmitting signals representative of the signals generated by said fourth means to said remote station to cause said second means to access said memory so as to read out therefrom as high resolution image information data corresponding to the entirety of the scene.

15. A system according to claim 14, wherein said system further comprises, at said local station,

a display device, and

sixth means for receiving the signals transmitted to said local station by said third means and causing said high resolution image information for the entirety of said scene to be displayed on said display device.

16. A system according to claim 15, wherein said system further includes, at said local station,

a recording device, and wherein

said sixth means includes means for causing said high resolution image information for the entirety of said scene to be recorded on said recording device.

17. A system according to claim 1, wherein said first means comprises means for continuously causing there to be generated from the entirety of said optically imaged scene high resolution image data and updating the contents of said memory with said high resolution image data.

18. A system according to claim 17, wherein said system comprises, at said remote station, optical sensor means for generating output signals representative of the entirety of the scene optically imaged thereby, and said first means includes means, coupled to said optical sensor means, for generating digital data representative said output signals and updating the contents of said memory with said digital data.

19. A system according to claim 18, wherein said second portion of said data read out from said memory by said second means as low resolution image information corresponds to a portion of said scene adjacent to that to which said high resolution image information corresponds.

20. A system according to claim 18, wherein said system further includes, at said local station,

a display device;

and

sixth means for receiving the signals transmitted to said local station by said third means and causing said high resolution image information to be coupled to said display device whereby a high resolution image of said portion of said scene is displayed thereby.

21. A system according to claim 20, wherein said fourth means comprises means, responsive to the observation, by an operator, of a location of interest on said display device, for causing the signals generated thereby to be representative of a delimited portion of said scene associated with said location of interest on said display device at which a high resolution image is to be displayed.

22. A system according to claim 21, wherein said system further includes an anti-jam communication link over which signals are transmitted between said remote and local stations.

23. A system according to claim 22, wherein said remote station comprises a remotely piloted vehicle the operation of which is controlled by said operator observing said display device.

24. A system according to claim 23, wherein said second portion of said data read out from said memory by said second means as low resolution image information corresponds to a portion of said scene adjacent to that to which said high resolution image information corresponds.

25. A system according to claim 24, wherein said sixth means includes means for causing said low resolution image information to be coupled to said display device, whereby a low resolution image of a portion of said scene adjacent to the high resolution portion thereof is displayed by said display device.

26. A system according to claim 25, wherein said fourth means comprises oculometer means and head movement sensing means, respectively responsive to the eye and head movement of said operator, for generating signals representative of the location of observation by said operator on said display at which said high resolution image is to be displayed.

27. A video storage and retrieval system comprising:

a memory;

first means for generating high resolution data representative of an optically imaged scene and causing said data to be stored in said memory; and

second means for selectively accessing said memory to read out therefrom a first portion of said data as high resolution image information and a second portion of said data as low resolution image information.

28. A video storage and retrieval system according to claim 27, further including third means for transmitting signals corresponding to said first portion of said data containing said high resolution image information and said second portion of said data containing said low resolution image information.

29. A video storage and retrieval system according to claim 27, wherein said second portion of said data read out from said memory by said second means as low resolution image information corresponds to a portion of said scene adjacent to that to which said high resolution image information corresponds.

30. A video storage and retrieval system according to claim 28, wherein

said second means further includes means for selectively accessing said memory to read out therefrom all the data stored thereby associated with said scene as high resolution data, and

said third means includes means for transmitting signals corresponding to the entirety of said scene as high resolution data.

31. A video storage and retrieval system according to claim 27, wherein said first means comprises means for continuously causing there to be generated from the entirety of said optically imaged scene high resolution image data and updating the contents of said memory with said high resolution image data.

32. A video storage and retrieval system according to claim 27, wherein said system comprises optical sensor means for generating output signals representative of the entirety of the scene optically imaged thereby, and said first means includes means, coupled to said optical sensor means, for generating digital data representative said output signals and updating the contents of said memory with said digital data.

33. A video storage and retrieval system according to claim 28, wherein said third means includes means for modifying the signals transmitted thereby so as to cause said signals to be effectively immune to jamming.

34. A video storage and retrieval system according to claim 28, wherein said third means includes means for transmitting signals over a spread spectrum communication link.

35. A video storage and retrieval system according to claim 27, wherein said system is equipped for use aboard a remotely piloted vehicle and said second means includes means, responsive to control signals from a control station, for selectively accessing said memory in accordance therewith.

36. For use with a remote device having a memory in which are stored high resolution data representative of an optically imaged scene to be displayed, and being adapted to be coupled to a display device upon which said scene may be displayed to an operator, a system for controlling the presentation of scene data to said operator comprising:

first means for sensing the location of the visual observation by said operator on said display device and, in response thereto, generating first signals identifying a prescribed portion of the scene containing said location of visual observation; and

second means, responsive to said first signals generated by said first means, for causing said remote device to transmit second signals as high resolution information representative of the high resolution data stored in said memory that corresponds to said prescribed portion of the scene containing said location of visual observation, and third signals as low resolution information representative of scene data stored in said memory other than that corresponding to said prescribed portion.

37. A system according to claim 36, wherein said system further comprises

third means, coupled to said display device and responsive to said second and third signals, for causing the scene image displayed by said display device to be displayed as a high resolution image within said prescribed portion thereof and as a low resolution image at a portion other than said prescribed portion thereof.

38. For use with a remote device having a memory in which are stored high resolution data representative of a scene to be displayed and with a display device upon which said scene may be displayed to an operator, a method of controlling the presentation of scene data to said operator comprising the steps of:

(a) sensing the location of visual observation by said operator on said display device and thereby identifying a prescribed portion of the scene containing said location of visual observation; and

(b) in response to step (a), causing said remote device to transmit high resolution information signals representative of high resolution scene data stored in said memory that corresponds to said prescribed portion of the scene containing said location of visual observation, and low resolution information signals representative of scene data stored in said memory other than that corresponding to said prescribed portion.

39. A method according to claim 38, further comprising the steps of:

(c) in response to step (b), causing the scene image displayed by said display device to be displayed as a high resolution image within said prescribed portion thereof and as a low resolution image at a portion other than said prescribed portion thereof.

40. A system for displaying, at a local station, a high resolution image of a portion of scene optically imaged at a remote station comprising:

at said remote station,

a memory;

first means for generating high resolution data representative of said optically imaged scene to be displayed and storing said data in said memory;

second means for selectively accessing said memory to read out therefrom a first portion of said data as high resolution image information and a second portion of said data as low resolution image information; and

third means for transmitting signals corresponding to said first portion of said data containing said high resolution image information and said second portion of said data containing said low resolution image information to said local station; and

at said local station,

display means for visually displaying the images representative of said first and second portions of said data;

fourth means for sensing the location on said display means the visual observation by the operator and, in response thereto, generating signals identifying that portion as requiring high resolution image information; and

fifth means for transmitting signals representative of the signals generated by said fourth means to said remote station to cause said second means to selectively access said memory to read out therefrom as high resolution image information that portion of said data corresponding to said prescribed portion of said scene.

41. A system according to claim 40, wherein said second portion of said data read out from said memory by said second means as low resolution image information corresponds to a portion or said scene adjacent to that to which said high resolution image information corresponds.

42. A system according to claim 40 further including, at said local station,

sixth means for receiving the signals transmitted to said local station by said third means and causing said high resolution image information to be coupled to said display means, whereby a high resolution image of said portion of said scene is displayed thereby.

43. A system according to claim 42, wherein said sixth means includes means for causing said low resolution image information to be coupled to said display means, whereby a low resolution image of a portion of said scene adjacent to the high resolution portion thereof is displayed by said display means.

44. A system according to claim 40, wherein said fourth means comprises means, responsive to the observation by an operator of a location of interest on said display means, for causing the signals generated thereby to be representative of a delimited portion of said scene associated with said location of interest on said display means at which a high resolution image is to be displayed.

45. A system according to claim 44, wherein said fourth means comprises oculometer means and head movement sensing means, respectively responsive to the eye and head movement of said operator, for generating signals representative of the location of observation by said operator on said display means at which said high resolution image is to be displayed.

46. A system according to claim 40, wherein said system comprises, at said remote station, optical sensor means for generating output signals representative of the entirety of the scene optically imaged thereby, and said first means includes means, coupled to said optical sensor means, for generating digital data representative of said output signals and updating the contents of said memory with said digital data.

47. A system according to claim 46, wherein said second portion of said data that is read out from said memory by said second means as low resolution image information corresponds to a portion of said scene adjacent to that to which said high resolution image information corresponds.