APPARATUS AND METHOD FOR ESTIMATING CAMERA POSE

• US 20150235378A1
• Filed: 02/02/2015
• Published: 08/20/2015
• Est. Priority Date: 02/17/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: RHEE, Seon Min, LEE, Yong Beom, LEE, Hyong Euk, PARK, Seung In
• Granted Patent: US 9,916,689 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

An apparatus for estimating a camera pose includes an image acquisition unit to acquire a photographed image, a motion sensor to acquire motion information of the apparatus for estimating the camera pose, a static area detector to detect a static area of the photographed image based on the photographed image and the motion information, and a pose estimator to estimate a camera pose based on the detected static area.

Claims

1. An apparatus for estimating a camera pose, the apparatus comprising:

an image acquisition unit to acquire a photographed image;

a motion sensor to acquire motion information of the apparatus for estimating the camera pose;

a static area detector to detect a static area of the photographed image based on the photographed image and the motion information; and

a pose estimator to estimate the camera pose based on the detected static area.

2. The apparatus of claim 1, wherein the image acquisition unit acquires a first color image and a second color image.

3. The apparatus of claim 2, wherein the pose estimator matches corresponding points between the first color image and the second color image, and estimates a rotation value from the motion information.

4. The apparatus of claim 3, wherein the pose estimator calibrates the estimated rotation value to correspond to a disposition in which the camera is at a position identical to the motion sensor.

5. The apparatus of claim 3, wherein the pose estimator calculates an essential matrix based on a matching relationship between the matched corresponding points and the estimated rotation value, and estimates a translation value based on the calculated essential matrix.

6. The apparatus of claim 5, wherein the static area detector detects the static area based on the estimated rotation value and the estimated translation value.

7. The apparatus of claim 6, wherein the static area detector applies the estimated rotation value and the estimated translation value to the corresponding point of the first color image, and detects a predicted position of the corresponding point of the second color image.

8. The apparatus of claim 7, wherein the static area detector detects an area in which a difference between the predicted position of the corresponding point of the second color image and an actual position of the corresponding point of the second color image is less than a predetermined threshold value to be a static area.

9. The apparatus of claim 1, wherein the image acquisition unit acquires a first depth image and a second depth image.

10. The apparatus of claim 9, wherein the pose estimator selects a pair of three-dimensional (3D) points from the first depth image and the second depth image that satisfies the motion information acquired from the motion sensor.

11. The apparatus of claim 10, wherein the pose estimator estimates a translation value based on the selected pair of 3D points.

12. The apparatus of claim 10, wherein the static area detector applies the rotation value and the translation value to a 3D point of the first depth image, and estimates a predicted position of a 3D point of the second depth image.

13. The apparatus of claim 12, wherein the static area detector detects an area in which a difference between the predicted position of the 3D point of the second depth image and an actual position of a 3D point of the second depth image is less than a predetermined threshold value to be a static area.

14. A method of controlling an apparatus for estimating a camera pose, the method comprising:

acquiring a photographed image;

acquiring motion information of the apparatus for estimating the camera pose;

detecting, by a processor, a static area of the photographed image based on the photographed image and the motion information; and

estimating the camera pose based on the detected static area.

15. The method of claim 14, wherein the acquiring of the photographed image comprises:

acquiring at least one of;

a color image pair comprising a first color image and a second color image, and

a depth image pair comprising a first depth image and a second depth image.

16. The method of claim 15, wherein the estimating of the camera pose is based on at least one of:

corresponding points of the first color image and the second color image, and

a pair of three-dimensional (3D) points from a static area of the first depth image and the second depth image.

17. The method of claim 14, further comprising:

outputting a control signal corresponding to the estimated camera pose.

18. The method of claim 17, further comprising:

executing an application that provides an augmented reality (AR) service,

wherein the outputting of the control signal corresponding to the estimated camera pose comprises;

providing an AR service corresponding to the estimated camera pose.

19. The method of claim 17, further comprising:

displaying a panoramic image having a size greater than a display screen of the apparatus for estimating the camera pose,

wherein the outputting of the control signal corresponding to the estimated camera pose comprises;

translating the displayed panoramic image corresponding to the estimated camera pose, and displaying the translated panoramic image.

20. The method of claim 19, wherein the translating of the displayed panoramic image corresponding to the estimated camera pose comprises:

translating the displayed panoramic image corresponding to translation information of the estimated camera pose.