METHOD AND APPARATUS FOR THREE-DIMENSIONAL IMAGE RECONSTRUCTION

• US 20120069019A1
• Filed: 09/22/2010
• Published: 03/22/2012
• Est. Priority Date: 09/22/2010
• Status: Active Grant

• Current Assignee: Raytheon Company (Rtx Corporation)
• Original Assignee: Raytheon Company (Rtx Corporation)
• Inventors: RICHARDS, Chet L.
• Granted Patent: US 8,610,708 B2
• Time in Patent Office: Days
• US Class Current: 345/421
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/536 from perspective effects, e.g. by using vanishing points

Abstract

In accordance with various aspects of the disclosure, a method and apparatus is disclosed for constructing a three-dimensional model of a scene captured as a video stream including obtaining a video stream of a scene by a first camera; locating a horizon with respect to the scene with a second camera; and constructing a three-dimensional model of the scene based upon the obtained video stream and the located horizon.

Claims

1. A method for constructing a three-dimensional model of a scene captured as a video stream, comprising:

obtaining a video stream of a scene by a first camera;

locating a horizon with respect to the scene with a second camera; and

in a processor in communication with a memory having instructions stored therein which, when executed, constructing a three-dimensional model of the scene based upon the obtained video stream and the located horizon.

2. The method according to claim 1, wherein the second camera includes an infrared sensor that is configured to penetrate haze and other obscurants.

3. The method according to claim 1, further comprising:

co-mounting the first and the second camera on a common support structure.

4. The method according to claim 1, wherein the second camera is configured to image an area outside of a field of view of the first camera.

5. The method according to claim 1, further comprising:

finding a range to an object within the scene by measuring an image position of the object with respect to the located horizon.

6. The method according to claim 1, further comprising:

determining a vanishing point in a nadir direction.

7. The method according to claim 6, further comprising:

correlating a vertical edge within the scene with a motion derived nadir vanishing point to refine an estimated camera pose.

8. The method according to claim 6, wherein the vanishing point in the nadir direction is determined based on a downward tilt angle of the first camera and a roll angle about a line of sight of the first camera.

9. The method according to claim 8, further comprising:

measuring slopes, locations and divergences of vertical edges within the video stream of the scene to determine the downward tilt angle of the first camera, the roll angle about the line of sight of the first camera, and the scene horizon.

10. The method according to claim 9, further comprising:

determining a distance to an object within the scene on a ground plane at a known relative altitude based on a focal length of the first camera and the scene horizon, the downward tilt angle of the first camera, and the roll angle about a line of sight of the first camera.

11. The method according to claim 10, further comprising:

determining different range features of other objects within the scene based on the determined distance to the object.

12. The method according to claim 11, further comprising:

determining a physical height of the object based on a height of the image of the object and the downward tilt angle of the first camera.

13. A method for mounting two cameras on an aerial vehicle, the method comprising:

arranging a first camera to maintain an essentially horizontal orientation with respect to a horizon to determine a line of the horizon; and

arranging a second camera relative to the first camera to image an object at ground level.

14. The method according to claim 13, further comprising:

mounting the second camera in a mounting assembly, wherein the mounting assembly is configured to allow the second camera to move in at least one degree of freedom relative to the first camera.

15. The method according to claim 14, wherein the at least one degree of freedom comprises movement about a depression angle with regard to a pivot point of the mounting assembly, or movement about a vertical axis of the second camera, or both.

16. The method according to claim 14, wherein both the first and the second camera are arranged together in the mounting assembly.

17. The method according to claim 13, wherein the first camera includes an infrared camera.

18. A computer-implemented method for determining a distance from a camera mounted to an aerial vehicle and locations within an image captured by the camera in a processor in communication with a memory having instructions stored therein which, when executed, perform the method comprising:

determining a location of a horizon with respect to a center of the camera, wherein the camera includes a lens with a focal length, the camera arranged at a height above a ground plane and a tilt angle with respect to the horizon;

determining a horizon line based on the determined location of the horizon;

determining a tilt of the horizon line;

determining a location of a ground point in the image; and

determining, by a distance determining computer, the distance from the height of the camera, the focal length of the lens of the camera, the tilt angle and the location of the ground point in the image.

19. The computer-implemented method according to claim 18, wherein the horizon line is outside a field of view of the camera.