Bundle adjustment based on image capture intervals

• US 9,466,107 B2
• Filed: 05/15/2015
• Issued: 10/11/2016
• Est. Priority Date: 05/18/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Lin, Michael Hongmai, Reinhardt, Tilman
• Primary Examiner(s): BEKELE, MEKONEN T
• Application Number:
  US14/713,012
• Publication Number:
  US 20150248759A1
• Time in Patent Office: 515 Days
• Field of Search: 382/154, 382/103, 382/285
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/20228 Disparity calculation for image-based rendering
• CPC Class Codes: G06T 2207/30181 Earth observation
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: G06V 2201/12 Acquisition of 3D measurements of objects

Abstract

Methods, systems, and computer program products are provided for determining camera parameters and three dimensional locations of features from a plurality of images of a geographic area. These include, determining a correlation between a pose of a first camera and a pose of a second camera, generating one or more constraints incorporating the correlation, and determining at least one of camera parameters and three dimensional locations of features using a plurality of constraints including the generated one or more constraints. The first camera and the second camera have substantially rigid positions and poses relative to each other. A strength of the correlation is based at least upon a time interval between respective image captures by the first camera and the second camera.

Claims

1. A computer-implemented method for determining three dimensional locations of features in images of an area, comprising:

receiving, by one or more computing devices, data associated with a pose of a first camera and a pose of a second camera;

determining, by the one or more computing devices, a time interval between respective image captures by the first camera and the second camera;

determining, by the one or more computing devices, a three dimensional location of a feature using a plurality of constraints, at least one of the plurality of constraints being based on the time interval and the relative poses of the first and second cameras; and

in response to a request for the three dimensional location of the feature, providing, by the one or more computing devices, bundle adjusted information associated with the three dimensional location of the feature.

2. The computer-implemented method of claim 1, wherein a relationship between the pose of the first camera and the pose of the second camera is associated with a decreasing function of a duration of the time interval.

3. The computer-implemented method of claim 1, further comprising receiving, by the one or more computing devices, position and timestamp information for each of a plurality of images captured by the first and second cameras.

4. The computer-implemented method of claim 1, wherein the first and second cameras are arranged within a camera rosette, the camera rosette including a plurality of cameras.

5. The computer-implemented method of claim 4, wherein the respective image captures by the first and second cameras occur in one iteration of image captures by plurality of cameras in the camera rosette.

6. The computer-implemented method of claim 4, wherein the respective image captures by the first and second cameras occur with the camera rosette located at different positions along a travel path of the camera rosette.

7. The computer-implemented method of claim 1, further comprising generating at least one constraint relating the time interval to an estimated pose adjustment.

8. The computer-implemented method of claim 7, wherein the at least one constraint is utilized to determine the three dimensional location of the feature.

9. The computer-implemented method of claim 1, wherein positions and poses of the first and second cameras are substantially rigid relative to each other.

10. A system for determining three dimensional locations of features in images of an area, the system comprising:

one or more computing devices including one or more processors and associated memory, the memory storing instructions that, when implemented by the one or more processors, configuring the one or more computing devices to;

receive data associated with a pose of a first camera and a pose of a second camera;

determine a time interval between respective image captures by the first camera and the second camera;

determine a three dimensional location of a feature using a plurality of constraints, at least one of the constraints being based on the time interval and the relative poses of the first and second cameras; and

in response to a request for the three dimensional location of the feature, provide bundle adjusted information associated with the three dimensional location of the feature.

11. The system of claim 10, further comprising a camera rosette including a plurality of cameras communicatively coupled to the one or more computing devices, wherein the plurality of cameras of cameras includes the first and second cameras.

12. The system of claim 11, wherein the respective image captures by the first and second cameras occur in one iteration of image captures by the plurality of cameras in the camera rosette or the respective image captures by the first and second cameras occur with the camera rosette located at different positions along a travel path of the camera rosette.

13. The system of claim 11, wherein the one or more computing devices are further configured to determine a relationship between the pose of the first camera and the pose of the second camera, the relationship being associated with a decreasing function of a duration of the time interval.

14. The system of claim 10, wherein the one or more computing devices are further configured to receive position and timestamp information for each of a plurality of images captured by the first and second cameras.

15. The system of claim 10, wherein the one or more computing devices are configured to generate at least one constraint relating the time interval to an estimated pose adjustment.

16. The system of claim 10, wherein positions and poses of the first and second cameras are substantially rigid relative to each other.

17. A tangible, non-transitory computer-readable medium storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations, comprising;

receiving data associated with a pose of a first camera and a pose of a second camera;

determining a time interval between respective image captures by the first camera and the second camera;

determining camera parameters using a plurality of constraints, at least one of the constraints being based on the time interval and the relative poses of the first and second cameras; and

in response to a request for the camera parameters, providing bundle adjusted information associated with the camera parameters.

18. The computer-readable medium of claim 17, Wherein determining the camera parameters comprises determining a pose adjustment for the first camera based on the time interval.

19. The computer-readable medium of claim 18, wherein the operations further comprise determining a three dimensional location of one or more features corresponding to the determined pose adjustment for the first camera.

20. The computer-readable medium of claim 17, wherein the operations further comprise determining a relationship between the pose of the first camera and the pose of the second camera, the relationship being associated with a decreasing function of a duration of the time interval.