Image-based localization feature point registration apparatus, method and computer-readable medium

• US 8,498,472 B2
• Filed: 12/27/2010
• Issued: 07/30/2013
• Est. Priority Date: 01/04/2010
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Yoon, Suk June, Roh, Kyung Shik, Hyung, Seung Yong, Ahn, Sung Hwan
• Primary Examiner(s): CHAWAN, SHEELA C
• Application Number:
  US12/979,036
• Publication Number:
  US 20110164832A1
• Time in Patent Office: 946 Days
• Field of Search: 382/115, 382/118, 382/154, 382/181, 382/231, 382/294, 382/203, 382/209, 382/228, 382/285, 345/419, 345/427
• US Class Current: 382/154
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20164 Salient point detection; Corner detection
• CPC Class Codes: G06T 2207/30248 Vehicle exterior or interior
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

An image-based localization feature point registration apparatus includes a camera to capture an image, a feature point extractor to extract a feature point from the captured image, a calculator to calculate depth information about the feature point according to whether the feature point is one of two-dimensional (2D) and a three-dimensional (3D) corner, and a feature point register to register 3D coordinates of the feature point based on the depth information about the feature point and image coordinates of the feature point.

Claims

1. An image-based localization feature point registration apparatus, comprising:

a camera to capture an image;

a feature point extractor to extract a feature point from the captured image;

a calculator to perform a first calculation to obtain depth information about the feature point if the feature point is a two-dimensional (2D) corner and to perform a second calculation, different from the first calculation, to obtain depth information about the feature point if the feature point is a three-dimensional (3D) corner; and

a feature point register to register 3D coordinates of the feature point based on the depth information about the feature point and image coordinates of the feature point.

2. The feature point registration apparatus according to claim 1, wherein the feature point extractor acquires 3D information about a window including the feature point.

3. The feature point registration apparatus according to claim 2, further comprising a corner identifier to determine whether the feature point is one of the 2D and the 3D corner based on the 3D information about the window including the feature point.

4. The feature point registration apparatus according to claim 3, wherein if the feature point is a 2D corner, the calculator calculates an average depth of an entire area of the window as the depth information about the feature point.

5. The feature point registration apparatus according to claim 3, wherein if the feature point is a 3D corner, the calculator extracts a foreground area from the image and calculates an average depth of the foreground area as the depth information about the feature point.

6. The feature point registration apparatus according to claim 3, wherein the calculator separates a foreground area and a background area from the image by calculating a depth distribution of the window.

7. An image-based localization feature point registration method, comprising:

capturing an image by a camera;

extracting, by a processor, a feature point from the captured image and acquiring three-dimensional (3D) information about a window including the feature point;

determining, by the processor, whether the feature point is one of a two-dimensional (2D) and a 3D corner based on the 3D information about the window;

calculating, by the processor, depth information about the feature point using a first calculation if the feature point is a 2D corner; and

calculating, by the processor, depth information about the feature point using a second calculation, different from the first calculation, if the feature point is a 3D corner; and

registering, by the processor, 3D coordinates of the feature point based on the depth information about the feature point and image coordinates of the feature point.

8. The feature point registration method according to claim 7, wherein if the feature point is a 2D corner, the calculating comprises calculating an average depth of an entire area of the window as the depth information about the feature point.

9. The feature point registration method according to claim 7, further comprising, if the feature point is a 3D corner, separating a foreground area and a background area from the image according to a depth distribution of the window.

10. The feature point registration method according to claim 9, wherein if the feature point is the 3D corner, the calculating comprises calculating an average depth of the foreground area as the depth information about the feature point.

11. At least one non-transitory computer readable medium comprising computer readable instructions that control at least one processor to implement a method, comprising:

capturing an image by a camera;

extracting a feature point from the captured image and acquiring three-dimensional (3D) information about a window including the feature point;

determining whether the feature point is one of a two-dimensional (2D) and a 3D corner based on the 3D information about the window;

calculating depth information about the feature point using a first calculation if the feature point is a 2D corner; and

calculating depth information about the feature point using a second calculation, different from the first calculation, if the feature point is a 3D corner; and

registering 3D coordinates of the feature point based on the depth information about the feature point and image coordinates of the feature point.

12. The at least one non-transitory computer readable medium comprising computer readable instructions that control at least one processor to implement the method of claim 11, wherein if the feature point is a 2D corner, the calculating comprises calculating an average depth of an entire area of the window as the depth information about the feature point.

13. The at least one non-transitory computer readable medium comprising computer readable instructions that control at least one processor to implement the method of claim 11, further comprising, if the feature point is a 3D corner, separating a foreground area and a background area from the image according to a depth distribution of the window.

14. The at least one non-transitory computer readable medium comprising computer readable instructions that control at least one processor to implement the method of claim 13, wherein if the feature point is the 3D corner, the calculating comprises calculating an average depth of the foreground area as the depth information about the feature point.