Method for producing a digital photo wherein at least some of the pixels comprise position information, and such a digital photo

• US 10,264,239 B2
• Filed: 06/29/2011
• Issued: 04/16/2019
• Est. Priority Date: 06/29/2010
• Status: Active Grant

• Current Assignee: CycloMedia Technology B.V.
• Original Assignee: CycloMedia Technology B.V.
• Inventors: Beers, Bart, van den Heuvel, Franciscus
• Primary Examiner(s): Rahaman, Mohammed S
• Assistant Examiner(s): Lee, Jimmy S
• Application Number:
  US13/704,456
• Publication Number:
  US 20130141549A1
• Time in Patent Office: 2,848 Days
• Field of Search: 348 46
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G03B 37/04 with cameras or projectors providing touching or overlapping fields of view
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: H04N 13/264 using the relative movement of objects in two video frames or fields

Abstract

Method for producing a digital photo comprising pixels, wherein at least some of the pixels comprise position information, the method comprising of: taking a photo with a known geometry of the optical system with which the photo is taken; recording the position from which the photo has been taken; recording the direction in which the photo has been taken; providing a three-dimensional model comprising points which are comprised by at least an outer surface of an object in the field of view of the position where the photo has been taken, wherein the points comprise position information; relating a pixel on the one hand and a corresponding point in the three-dimensional model on the other; and recording, in a manner associated with the pixel, the position information of the corresponding pixel.

Claims

1. A method for producing a digital photo comprising pixels, wherein at least some of the pixels comprise position information, the method comprising of:

a) taking a first photo with a known geometry of the optical system with which the first photo is taken;

b) recording the position from which the first photo has been taken;

c) recording the direction in which the first photo has been taken;

d) using a LIDAR device for providing a three-dimensional model comprising points, each point comprising position information, and wherein the points each correspond to a respective reflection point on an outer surface of an object from which reflection point a laser beam from the LIDAR device has been reflected back to the LIDAR device;

e) for each pixel in the photo;

determining, in dependence of the position and direction recorded for that first photo, a line that runs through said each pixel and the position from which the first photo has been taken;

searching for a point in the three-dimensional model that relates to the same respective reflection point as said each pixel, based on a distance between such point and the line;

if a point in the three-dimensional model is found that relates to the same reflection point, calculating a distance to be associated with said each pixel using the distance between the reflection point that corresponds to the point found in said three-dimensional model and the position from which the first photo has been taken; and

storing the distance in association with said pixel of said first photo.

2. The method as claimed in claim 1, wherein said searching and finding said point using said line comprises:

determining which point of the three-dimensional model is at the shortest distance to the position from which the photo has been taken while the distance from this point to the line lies below a threshold value;

or

reconstructing a plane using points of the three-dimensional model that are in the vicinity of the line and determining the point as the intersection of the line and the reconstructed plane.

3. The method as claimed in claim 1, wherein the method comprises using a first and second photo camera for recording respective photos according to step a), and comprises performing step b) and c) for each recorded photo, wherein the digital photo is constructed by stitching the photos obtained by the first and second photo camera.

4. The method as claimed in claim 3, wherein the field of view of the digital photo comprises a field of vision of 360 degrees.

5. The method as claimed in claim 4, wherein the first and second camera are mounted on a vehicle, wherein the photos are taken from the vehicle while the vehicle is in motion;

wherein the first and second camera are controlled such that, when the second photo camera takes a photo, the entrance pupil of the second photo camera is situated as close as possible to the position where the entrance pupil of the first photo camera was situated when it took a photo at a preceding point in time.

6. The method as claimed in claim 1, the method further comprising:

extracting a corresponding pair of object characteristics from the photo and the three-dimensional model;

comparing the position of the extracted object characteristics in the photo to the position of the corresponding extracted object characteristics in the three-dimensional model;

determining a relative position error between the photo and the three-dimensional model based on said comparison; and

correcting the position from which the photo has been taken and/or the position information of the three-dimensional model based on said relative position error.

7. The method as claimed in claim 6, wherein correction of the position comprises correcting the position data of both the photo and the three-dimensional model on the basis of a weighting.

8. The method as claimed in claim 7, wherein the weighting is determined by reliability information of the position determinations of the photo and/or the three-dimensional model.

9. The method as claimed in claim 5, wherein the vehicle is a car.

10. The method as claimed in claim 1, wherein the LIDAR device is arranged on a vehicle and wherein the points are collected while the vehicle is in motion.

11. The method as claimed in claim 10, wherein the photo is taken from a vehicle other than the vehicle on which the LIDAR device is arranged.

12. The method as claimed in claim 1, each pixel comprising a plurality of color channels and a depth channel, in which depth channel said distance is indicated.