Method for determining a location from images acquired of an environment with an omni-directional camera

• US 8,249,302 B2
• Filed: 06/30/2009
• Issued: 08/21/2012
• Est. Priority Date: 06/30/2009
• Status: Expired due to Fees

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Ramalingam, Srikumar, Brand, Matthew, Bouaziz, Sofien
• Primary Examiner(s): Werner, Brian P
• Application Number:
  US12/495,655
• Publication Number:
  US 20100329542A1
• Time in Patent Office: 1,148 Days
• Field of Search: None
• US Class Current: 382/104
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/162 involving graph-based methods
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06V 20/39 Urban scenes
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects

Abstract

A location and orientation in an environment is determined by first acquiring a real omni-directional image of an unknown skyline in the environment. A set of virtual omni-directional images of known skylines are synthesized from a 3D model of the environment, wherein each virtual omni-directional image is associated with a known location and orientation. The real omni-directional image with each virtual omni-directional images to determine a best matching virtual omni-directional image with the associated known location and orientation.

Claims

1. A method for determining a location and orientation in an environment, comprising, a processor for performing steps of the method, comprising the steps of:

acquiring a real omni-directional image of an unknown skyline in the environment from an unknown location and unknown orientation in the environment;

synthesizing a set of virtual omni-directional images from a 3D model of the environment, wherein each virtual omni-directional image is associated with a known skyline, a known location and a known orientation; and

comparing the real omni-directional image with each virtual omni-directional images to determine a best matching virtual omni-directional image with the associated known location and orientation that correspond to the unknown location and orientation.

2. The method of claim 1, further comprising:

extracting features from pixels in the real and virtual images, and the comparing is based on the features.

3. The method of claim 1, wherein the environment is urban.

4. The method of claim 1, wherein the omni-directional camera is arranged in a vehicle, and the 3D model is acquired in real time using a communication network as the virtual images are generated.

5. The method of claim 1, wherein the synthesizing is performed in real time from the 3D model acquired in real-time using a network.

6. The method of claim 2, wherein the features include pixel intensities.

7. The method of claim 1, further comprising:

segmenting the real image using the features into foreground and back ground pixels, and the background pixels correspond to a sky, and wherein the foreground pixels adjacent to the back ground pixels define a skyline.

8. The method of claim 1, further comprising:

calibrating the omni-directional camera using a generic imaging model.

9. The method of claim 1, wherein the synthesizing further comprises:

generating a set of perspective images for each virtual image from the 3D model; and

combining the perspective images to form the virtual omni directional image.

10. The method of claim 1, wherein each virtual omni-directional image is a binary image.

11. The method of claim 2, wherein the comparing uses a chamfer distance.

12. The method of claim 1, wherein each virtual omni-directional image is synthesized at multiple resolutions.

13. The method of claim 2, wherein the features include color, gradients, lines and vanishing points.

14. The method of claim 8, wherein the generic calibration model can be applied to arbitrary combinations of cameras, mirrors and lenses.

15. The method of claim 1, wherein the set of virtual omni-directional images are synthesized in real time.

16. The method of claim 1, wherein the environment is an urban setting.

17. The method of claim 1, wherein the environment is indoor.

18. The method of claim 1, wherein the real omni-directional image is acquired using visible light.

19. The method of claim 7, wherein the segmenting is performed using a graphcut.

20. The method of claim 8, wherein the synthesizing uses a generic projection.

21. A system for determining a location and orientation in an environment, comprising:

an omni-directional camera configured to acquire a real omni-directional image of an unknown skyline in the environment from an unknown location and unknown orientation in the environment;

means for synthesizing a set of virtual omni-directional images from a 3D model of the environment, wherein each virtual omni-directional image is associated with a known skyline, a known location, and a known orientation; and

means for comparing the real omni-directional image with each virtual omni-directional images to determine a best matching virtual omni-directional image with the associated known location and known orientation that correspond to the unknown location and unknown orientation.