Method for position and location detection by means of virtual reference images

• US 9,857,178 B2
• Filed: 03/04/2015
• Issued: 01/02/2018
• Est. Priority Date: 03/05/2014
• Status: Active Grant

• Current Assignee: Airbus Defence and Space GmbH
• Original Assignee: Airbus DS GMBH
• Inventors: Ahrns, Ingo, Haskamp, Christoph
• Primary Examiner(s): Dang, Hung
• Assistant Examiner(s): Wendmagegn, Girumsew
• Application Number:
  US14/638,128
• Publication Number:
  US 20150253140A1
• Time in Patent Office: 1,035 Days
• Field of Search: 348116
• CPC Class Codes: G01C 21/005 with correlation of navigation data from several sources, e.g. map or contour matching G01C21/30 takes precedence
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/24 specially adapted for cosmonautical navigation
• CPC Class Codes: G01C 21/30 Map- or contour-matching
• CPC Class Codes: G01C 21/38 Electronic maps specially adapted for navigation; Updating thereof
• CPC Class Codes: G01C 21/3826 Terrain data
• CPC Class Codes: G01C 21/3852 Data derived from aerial or satellite images
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 20/13 Satellite images
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof

Abstract

A method describes the architecture of a vision-supported navigation system in a terrain using available topographical maps, which is based on already existing methods for detecting prominent terrain features. An offline process used in this case creates a feature database from existing topographical maps, which is subsequently used in an online process for recognizing features that have been seen. In this case, virtual reference images of the expected situation are created using computer graphical methods, which are analyzed using methods for feature extraction, and a feature catalog made up of vision-based feature vectors and the associated 3D coordinates of the localized features is derived from the same. These data are stored in a flight system in the form of a model database, for example, and enable navigation close to a reference trajectory and for a planned lighting situation. High localization characteristics and a correspondingly high navigation accuracy are achieved by means of the possibility of using any desired, not necessarily lighting-invariant, feature extractors.

Claims

1. A method of detecting a position and a location of a camera with respect to a topographical map, comprising the steps:

a) obtaining existing topographical map data;

b) obtaining a predicted camera position and location that a camera is predicted to have during a planned trajectory;

c) producing a local 3D model from the existing topographical map data, wherein the local 3D model contains only such localized map features of the existing topographical map data that would be imaged by the camera at the predicted camera position and location;

d) producing a virtual image from the local 3D model;

e) in an off-line process, producing a feature database from the virtual image; and

f) in an online process, using the feature database to recognize actual features that have been imaged by the camera during an actual trajectory.

2. The method according to claim 1, wherein at least the step f) is carried out on-board a flight system, which flies along the actual trajectory.

3. The method according to claim 1, wherein the step d) further comprises producing a sequence of a plurality of the virtual images, which further involves storing a respective corresponding 3D coordinate in a reference coordinate system for each pixel of each one of the virtual images.

4. The method according to claim 1, wherein the step d) comprises producing the virtual image with a computer-graphical method based on the local 3D model evaluated for an expected situation, and wherein the step e) comprises analyzing the virtual image by feature extraction to produce the feature database containing vision-based feature vectors and associated 3D coordinates of localized features of the virtual image based on the localized map features.

5. The method according to claim 4, wherein the expected situation comprises a predicted lighting situation that is predicted to exist at the predicted camera position and location on the planned trajectory.

6. The method according to claim 1, wherein the feature database is stored in a flight system, and further comprising navigating the flight system along the actual trajectory to follow closely along the planned trajectory.