Stabilization and display of remote images

• US 10,178,316 B2
• Filed: 07/17/2015
• Issued: 01/08/2019
• Est. Priority Date: 07/17/2014
• Status: Active Grant

• Current Assignee: Elbit Systems Limited
• Original Assignee: Elbit Systems Limited
• Inventors: Shamir, Hanan, Ashkenazi, Asaf, Tell, Eliyahu
• Primary Examiner(s): Nasri, Maryam A
• Application Number:
  US15/326,673
• Publication Number:
  US 20170208251A1
• Time in Patent Office: 1,271 Days
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64U 20/87 Mounting of imaging devices, e.g. mounting of gimbals
• CPC Class Codes: G01C 21/1656 with passive imaging devices, e.g. cameras
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/30181 Earth observation
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/685 performed by mechanical compensation
• CPC Class Codes: H04N 25/531 by controlling rolling shutters in CMOS SSIS
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

Apparatus for remote imaging of a terrestrial area, the apparatus comprising: a camera array having a focal length f and a photosensor comprising photosensor pixels characterized by a photosensor pixel pitch P on which light from the terrestrial area is imaged from an operating distance A from the terrestrial area to acquire an image of the terrestrial area; an orthographic image of the terrestrial area having image pixels that image features in the terrestrial area; a terrain map that provides elevation for features in the terrestrial area imaged on the image pixels of the orthographic image; a controller that registers the image of the terrestrial area to the orthographic image responsive to the terrain data; wherein elevation provided by the terrain map has an uncertainty Δβ that satisfies a constraint P≥Δe·f·sin α/A, where a is a maximum oblique angle at which the camera array images the terrestrial area so that registration of the image of the terrestrial area to the orthographic image has an accuracy better than or about equal to the pixel pitch.

Claims

1. Apparatus for remote imaging of at least one region of interest (ROI) in a terrestrial area, the apparatus comprising:

a camera array having a focal length f and a photo sensor comprising pixels characterized by a pixel pitch P that acquires a sequence of images of the terrestrial area, the images acquired from different 3D spatial locations and camera orientations;

an inertial measurement unit (IMU) and satellite navigation system (GNSS) receiver that acquire data responsive to the 3D spatial locations and/or orientations of the camera array during acquisition of the sequence of images;

a data base comprising a terrain map that provides elevation data having an uncertainty Δ

e that satisfies a constraint P≥

Δ

e·

f·

sin α

/A, where A is a maximum distance of the camera array from the terrestrial area and a is a maximum oblique angle at which the camera array images the terrestrial area; and

a controller that;

registers at least one image of the sequence of images to an orthographic image of the terrestrial area or another image of the sequence of images to determine corrections to the IMU and/or GNSS data;

uses the corrected IMU and/or GNSS data to crop each image of the sequence to an ROI cropped image portion of the image that images an ROI of the at least one ROI; and

registers each ROI cropped image portion to an orthographic image of the terrestrial area responsive to elevation data in the terrain image of the terrestrial area to stabilize the ROI cropped image portion so that a same feature in each of the ROI cropped image portions is located at a same location to within a distance of pitch P in each of the registered ROI cropped image portions.

2. The apparatus according to claim 1 wherein P is less than or equal to about 15 μ

m (micrometers).

3. The apparatus according to claim 2 wherein P is less than or equal to about 10 μ

m.

4. The apparatus according to claim 3 wherein P is less than or equal to about 3 μ

m.

5. The apparatus according to claim 1 wherein A is greater than or equal to about 500 m (meters).

6. The apparatus according to claim 5 wherein A is greater than or equal to about 5,000 m.

7. The apparatus according to claim 6 wherein A is greater than or equal to about 10,000 m.

8. The apparatus according to claim 1 wherein a number of the photosensor pixels is equal to or greater than about 10⁹.

9. The apparatus according to claim 8 wherein a number of the photosensor pixels is equal to or greater than about 5×

10⁹.

10. The apparatus according to claim 1 wherein the controller registers at least 1 of every 10 images of the terrestrial area acquired by the camera array to the orthographic image to determine corrections to the IMU and/or GNSS data.

11. The apparatus according to claim 10 wherein the controller registers at least 1 of every 5 images of the terrestrial area acquired by the camera array to the orthographic image to determine corrections to the IMU and/or GNSS data.

12. The apparatus according to claim 11 wherein the controller registers at least 1 of every 3 images of the terrestrial area acquired by the camera array to the orthographic image to determine corrections to the IMU and/or GNSS data.

13. The apparatus according to claim 1 wherein the camera array acquires images of the terrestrial area at a frequency greater than or equal to about 3 Hz.

14. The apparatus according to claim 1 wherein the controller samples data provided by the IMU at a frequency greater than or equal to about 100 Hz.

15. The apparatus according to claim 14 wherein the controller samples data provided by the IMU at a frequency greater than or equal to about 200 Hz.

16. The apparatus according to claim 1 wherein the controller samples data provided by the GNSS receiver at a frequency greater than or equal to about 1 Hz.

17. The apparatus according to claim 1 wherein the camera array comprises a camera that operates in a rolling shutter mode.

18. The apparatus according to claim 1 and comprising a database comprising data that the IMU and/or GNSS acquires.

19. The apparatus according to claim 1 wherein the camera array is stabilized by a two axis gimbal system.

20. The apparatus according to claim 1 wherein the camera array is not stabilized by gimbal system.