Camera alignment method using correspondence between multi-images

• US 9,576,365 B2
• Filed: 07/29/2014
• Issued: 02/21/2017
• Est. Priority Date: 02/18/2014
• Status: Expired due to Fees

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Jang, Kyung Ho
• Primary Examiner(s): Rao, Andy
• Assistant Examiner(s): EDWARDS, TYLER B
• Application Number:
  US14/446,281
• Publication Number:
  US 20150237330A1
• Time in Patent Office: 938 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/20072 Graph-based image processing
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

Provided is a method of aligning a camera using correspondence information between multi-images. The camera alignment method using correspondences between multi-images includes defining a correspondence relation between images photographed in the multi-camera system, estimating an initial position of the camera using the correspondence relation between the images and a Structure From Motion (SFM) algorithm, redefining a changed correspondence relation between the images as a result of the estimation of the initial position of the camera using a bundle edge to generate an optimal edge, and correcting the position of the camera based on the optimal edge.

Claims

1. A method of aligning a position of a camera in a multi-camera system photographing multi-images, the method comprising:

defining a correspondence relation between a plurality of images photographed in the multi-camera system;

estimating respective initial positions of the camera and a three-dimensional point using the correspondence relation between the plurality of images and a Structure From Motion (SFM) algorithm;

determining projected two-dimensional points by projecting the three-dimensional point onto first and second images among the plurality of images, respectively, the first and second images including respective original two-dimensional points;

determining Euclidean distances between the projected two-dimensional points and the original two-dimensional points of the first and second images, respectively; and

when a sum of the Euclidean distances is less than or equal to a reference value, generating an optimal edge using a bundle edge that connects a first vertex corresponding to the first image and a second vertex corresponding to the second image, and correcting the position of the camera based on the optimal edge.

2. The method of claim 1, wherein defining the correspondence relation comprises generating a graph including a plurality of vertices and a plurality of edges, the plurality of vertices corresponding to the plurality of images, and each of the edges representing a relation between the plurality of images,

wherein the plurality of vertices includes the first and second vertices.

3. The method of claim 1, wherein defining the correspondence relation comprises:

extracting feature points from the first image and the second image;

comparing the feature point of the first image with the feature point of the second image to calculate a number of valid correspondences between the first and second images; and

if the number of valid correspondences is greater than a preset threshold, generating an edge between the first vertex and the second vertex.

4. The method of claim 2, wherein

the bundle edge connects the first vertex and the second vertex when an edge between the first vertex and the second vertex defined in the graph is broken as a result of the estimation of the initial position of the camera.

5. The method of claim 2, wherein

the bundle edge connects the first vertex and the second vertex when an edge between the first vertex and the second vertex defined in the graph is not connected as a result of the estimation of the initial position of the camera.

6. The method of claim 1, wherein correcting the position of the camera comprises:

(a) calculating the three-dimensional point, the three-dimensional point being shared by the first and second vertices, the first and second vertices being connected to the optimal edge;

(b) calculating camera parameter information for each of the first and second images using the three-dimensional point;

(c) calculating the sum of the Euclidean distances between the projected two-dimensional points and the original two-dimensional points of the first and second images, respectively using the camera parameter information; and

(d) recursively performing operations (a) to (c) until the sum of the Euclidean distances is equal to or less than the preset threshold.

7. The method of claim 1, further comprising:

generating, when the sum of the Euclidean distances is greater than the reference value, a first optimal edge connected to the first vertex and a second optimal edge connected to the second vertex.

8. The method of claim 7, wherein the generating comprises:

when the sum of the Euclidean distances is greater than the reference value, disconnecting the bundle edge that connects the first vertex and the second vertex, generating the first optimal edge and the second optimal edge, and correcting the position of the camera based on the first optimal edge and the second optimal edge.

9. An apparatus for aligning a position of a camera in a multi-camera system photographing multi-images, the apparatus comprising:

at least one processor; and

a non-volatile memory configured to store a code executed by the processor,

wherein the processor is implemented to;

define a correspondence relation between a plurality of images photographed in the multi-camera system;

estimate respective initial positions of the camera and a three-dimensional point using the correspondence relation between the plurality of images and a Structure From Motion (SFM) algorithm; and

determine projected two-dimensional points by projecting the three-dimensional point onto first and second images among the plurality of images, respectively, the first and second images including respective original two-dimensional points;

determine Euclidean distances between the projected two-dimensional points and the original two-dimensional points of the first and second images, respectively; and

when a sum of the Euclidean distances is less than or equal to a reference value, generate an optimal edge using a bundle edge that connects a first vertex corresponding to the first image and a second vertex corresponding to the second image, and correct the position of the camera based on the optimal edge.

10. The apparatus of claim 9, wherein the processor is implemented to define the correspondence relation by generating a graph including a plurality of vertices and a plurality of edges, the plurality of vertices corresponding to the plurality of images, and each of the plurality of edges representing a relation between the plurality of images,

wherein the plurality of vertices includes the first and second vertices.

11. The apparatus of claim 10, wherein

the bundle edge connects the first vertex and the second vertex when an edge between the first vertex and the second vertex defined in the graph is broken as a result of the estimation of the initial position of the camera.

12. The apparatus of claim 10, wherein

the bundle edge connects the first vertex and the second vertex when an edge between the first vertex and the second vertex defined in the graph is not connected as a result of the estimation of the initial position of the camera.

13. The apparatus of claim 10, wherein the processor is implemented to generate, when the sum of the Euclidean distances is greater than the reference value, a first optimal edge connected to the first vertex and a second optimal edge connected to the second vertex.

14. The apparatus of claim 13, wherein the processor is implemented to, when the sum of the Euclidean distances is greater than the reference value, disconnect the bundle edge that connects the first vertex and the second vertex, generate the first optimal edge and the second optimal edge, and correct the position of the camera based on the first optimal edge and the second optimal edge.