APPARATUS AND METHOD FOR CAMERA TRACKING

• US 20140241576A1
• Filed: 12/10/2013
• Published: 08/28/2014
• Est. Priority Date: 02/28/2013
• Status: Abandoned Application

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: YU, Jung-Jae, KIM, Hae-Dong, KIM, Hye-Sun, BAN, Yun-Ji, KIM, Myung-Ha, BYON, Joo-Hee, JANG, Ho-Wook, NAM, Seung-Woo, JANG, Kyung-Ho
• Application Number:
  US14/102,096
• Publication Number:
  US 20140241576A1
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/55 from multiple images

Abstract

A camera tracking apparatus including a sequence image input unit configured to obtain one or more image frames by decoding an input two-dimensional image, a two-dimensional feature point tracking unit configured to obtain a feature point track by extracting feature points from respective image frames obtained by the sequence image input unit, and comparing the extracted feature points with feature points extracted from a previous image frame, to connect feature points determined to be similar, and a three-dimensional reconstruction unit configured to reconstruct the feature point track obtained by the two-dimensional feature point tracking unit.

Claims

1. A camera tracking apparatus comprising:

a sequence image input unit configured to obtain one or more image frames by decoding an input two-dimensional image;

a two-dimensional feature point tracking unit configured to obtain a feature point track by extracting feature points from each of the image frames obtained by the sequence image input unit, and by comparing the extracted feature points with feature points extracted from a previous image frame to connect feature points that are determined to be similar; and

a three-dimensional reconstruction unit configured to reconstruct the feature point track obtained by the two-dimensional feature point tracking unit.

2. The camera tracking apparatus of claim 1, wherein the two-dimensional feature point tracking unit extracts feature points, and connects feature points discovered to be similar to each other by performing matching comparing a descriptor that represents a shape of a feature point to distinguish feature points from one another.

3. The camera tracking apparatus of claim 1, wherein the two-dimensional feature point tracking unit connects only pairs of feature points corresponding to inliers, not pairs of feature points corresponding to outliers, by calculating a fundamental matrix and a homography matrix.

4. The camera tracking apparatus of claim 1, wherein the two-dimensional feature point tracking unit divides an input image into a plurality of blocks, and adds new features needed to keep the number of feature tracks in each block bigger than a predefined minimum value.

5. The camera tracking apparatus of claim 1, wherein the two-dimensional feature point tracking unit, in a case in which a feature point track is disconnected and then after several frames feature points coincident with the disconnected feature point track are reobserved, reconnects the feature points that are classified as inliers among the reobserved feature points in consideration of a cumulative homography matrix.

6. The camera tracking apparatus of claim 1, further comprising a three-dimensional reconstruction preparation unit configured to adjust an option for three-dimensional reconstruction and designate a parameter value.

7. The camera tracking apparatus of claim 6, wherein the three-dimensional reconstruction preparation unit edits the feature point track obtained by the two-dimensional feature point tracking unit according to user input, wherein an error graph of quantitative results of the two-dimensional feature point tracking unit are displayed on a screen, and unnecessary feature point tracks are selected and removed according to user input.

8. The camera tracking apparatus of claim 6, wherein the three-dimensional reconstruction preparation unit edits the feature point track obtained by the two-dimensional feature point tracking unit according to user input, wherein an editing user interface is displayed on a screen, and a plurality of feature points are connected through group matching according to user input.

9. The camera tracking apparatus of claim 1, wherein the three-dimensional reconstruction unit comprises:

a key frame selection unit configured to extract a key frame from one or more frames at intervals of a predetermined number of frames;

an initial section reconstruction unit configured to perform three-dimensional reconstruction on an initial section formed of two first key frames;

a sequential section reconstruction unit configured to expand the three-dimensional reconstruction in a key frame section following the initial section;

a camera projection matrix calculation unit configured to calculate camera projection matrixes of remaining intermediate frames except for the key frame; and

a three-dimensional reconstruction adjustment unit configured to obtain camera projection matrixes and reconstruction three-dimensional point coordinates of entire frames that minimize a total reprojection error.

10. A camera tracking method comprising:

obtaining one or more image frames by decoding an input two-dimensional image;

tracking a feature point track by extracting feature points from each of the obtained image frames, and by comparing the extracted feature points with feature points extracted from a previous image frame to connect feature points that are determined to be similar; and

reconstructing the obtained feature point track.

11. The camera tracking method of claim 10, further comprising:

adjusting an algorithm parameter value that is to be used in the tracking of the feature point track; and

generating a mask region.

12. The camera tracking method of claim 10, wherein in the tracking of the feature point track, feature points that are not connected, among feature points detected from a current frame, are added to a new feature point track that starts from the current frame.

13. The camera tracking method of claim 10, further comprising:

preparing for three-dimensional reconstruction by adjusting an option for the three-dimensional reconstruction and designating a parameter value.

14. The camera tracking method of claim 13, wherein in the preparing of the three-dimensional reconstruction, the feature point track obtained in the tracking of the feature point track is edited according to user input, wherein an editing user interface is displayed on a screen if the feature point track is disconnected, and a plurality of feature points are connected through group matching according to user input.

15. The camera tracking method of claim 10, wherein the preparing of the three-dimensional reconstruction comprises:

extracting a key frame from one or more frames at intervals of a predetermined number of frames;

performing three-dimensional reconstruction on an initial section formed of two first key frames;

expanding the three-dimensional reconstruction in a key frame section following the initial section;

calculating camera projection matrixes of remaining intermediate frames except for the key frame; and

obtaining camera projection matrixes and reconstruction three-dimensional point coordinates of entire frames that minimize a total reprojection error.