Automated video imagery database generation using photogrammetry

• US 5,259,037 A
• Filed: 02/07/1991
• Issued: 11/02/1993
• Est. Priority Date: 02/07/1991
• Status: Expired due to Term

• Current Assignee: L-3 Communications Corporation (L3Harris Technologies, Inc.)
• Original Assignee: Hughes Training, Inc. (Rtx Corporation)
• Inventors: Plunk, Gerald W.
• Primary Examiner(s): Boudreau, Leo H.
• Assistant Examiner(s): Johns, Andrew W.
• Application Number:
  US07/651,819
• Time in Patent Office: 999 Days
• Field of Search: 382/1, 382/44, 382/45, 364/443, 364/458, 364/456, 358/103, 356/12, 356/20, 434/38
• US Class Current: 382/154
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
• CPC Class Codes: G06T 15/10 Geometric effects
• CPC Class Codes: G09B 29/003 Maps
• CPC Class Codes: G09B 9/301 by computer-processed or -generated image
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

A method for converting forward looking video or motion picture imagery into a downlooking database suitable for use in an image generation system to generate realtime perspective images for simulation purposes. A digital video camera photographs a sequence of forward looking frames along a course including several known ground control points. Digital data from the digital video camera is read into a computer. The initial position and the position of ground control points is read into the computer. A Kalman filter technique provides continuous estimates of exact camera location in x, y, z coordinates, and in roll, pitch and yaw orientation. A correlation filter determines corresponding pixels in adjacent frames. Range from the camera to the pixel surfaces is calculated to determine the x, y and z position thereof. A modified mathematical stereophotogrammetry technique is used. Radiometric data and elevation data is extracted from the lower portion of the frame for each pixel. After the radiometric and elevation data have been established for each pixel in each frame, the imagery is warped onto the elevation data to generate a top down view. The database is then ready for use in an image generation system.

Claims

1. A method for generating a downlooking three dimensional photographic database from oblique sequential imagery comprising the following steps:

acquiring oblique forward looking sequential imagery data with respect to a vehicle moving over a terrain of interest for a plurality of frames;

calculating the position and attitude of the vehicle in x, y, z and roll, pitch and yaw at the time each frame was made;

determining corresponding pixels in adjacent frames;

calculating range to corresponding pixels in each frame;

using range information for corresponding pixels and photogrametric calculations to generate elevation data for each pixel in each frame; and

determining radiometric data for each pixel in each frame using only radiometric data extracted from the lower portion of each frame; and

storing the radiometric data and elevation data for each pixel to provide a three dimensional database suitable for use in an image generation system.

2. The method of claim 1 wherein the step of acquiring oblique forward looking sequential imagery frames comprises the step of:

acquiring oblique forward looking sequential imagery data with respect to a vehicle moving over a terrain of interest; and

digitizing the oblique forward looking sequential imagery frames.

3. The method of claim 1 wherein the step of acquiring oblique forward looking sequential imagery frames comprises the step of:

digitally recording oblique forward looking sequential imagery frames from a vehicle moving over the terrain of interest.

4. The method of claim 1 wherein the step of determining corresponding pixels in adjacent frames comprises the step of:

determining corresponding pixels in adjacent frames by use of a correlation filter.

5. The method of claim 1 wherein the step of calculating the position of the vehicle comprises the step of:

estimating the exact position of the vehicle at the time each frame was made in x, y, z and roll, pitch and yaw using Kalman filter processing.

6. The method of claim 1 wherein the step of acquiring oblique forward looking imagery comprises the step of:

recording oblique forward looking sequential imagery frames from a vehicle moving over the terrain of interest using a single recording camera.

7. A method for generating a downlooking three dimensional photographic database form oblique sequential imagery comprising the following steps:

acquiring oblique forward looking sequential imagery data with respect to a vehicle moving over a terrain of interest using a single recording camera;

digitizing the oblique forward looking sequential imagery frames;

calculating the position and attitude of the vehicle in x, y, z and roll, pitch and yaw at the time each frame was made;

determining corresponding pixels in adjacent frames using a correlation filter;

calculating range to corresponding pixels in each frame;

using range data for corresponding pixels and photogrammetric calculations to generate elevation data for each pixel in each frame;

processing only radiometric data extracted from the lower portion of each frame to generate radiometric data for each pixel in the frame; and

storing the elevation data and radiometric data for each pixel to generate a downlooking three dimensional database suitable for use in an image generation system.

8. The method claim 7 wherein the step of calculating position comprises the step of:

estimating the exact position of the vehicle at the time each frame was made in x, y, z and roll, pitch and yaw using Kalman filter processing.

9. The method of claim 7 wherein the step of acquiring sequential frames comprises the step of:

recording oblique forward looking sequential imagery frames from a vehicle moving over the terrain of interest using a single recording camera.

10. A method for generating a downlooking three dimensional database from oblique sequential imagery comprising the following steps:

making oblique forward looking sequential imagery frames from a vehicle moving over a terrain of interest using a single camera for a plurality of frames;

digitizing the oblique forward looking sequential imagery;

loading the digitized oblique forward looking sequential imagery into a computer;

calculating the position of the vehicle in x, y, z and roll, pitch and yaw from known ground control positions at the time each frame was made;

estimating the exact position of the vehicle in x, y, z and roll, pitch and yaw using extended Kalman filtering techniques at the time each frame was made;

determining corresponding pixels in adjacent frames by use of a correlation filter;

calculating range to corresponding pixels in each frame

using range data for each frame and photogrammetry calculations to determine elevation data for each pixel in each frame;

generating radiometric data for each pixel in each frame using only radiometric data extracted from the lower portion of each frame; and

storing the radiometric data and the elevation data for each pixel to generate a top down view thus providing a downlooking three dimensional database suitable for use in an image generation system.