APPARATUS FOR EXTRACTING CAMERA MOTION, SYSTEM AND METHOD FOR SUPPORTING AUGMENTED REALITY IN OCEAN SCENE USING THE SAME

• US 20080075358A1
• Filed: 05/02/2007
• Published: 03/27/2008
• Est. Priority Date: 09/27/2006
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Yu, Jung Jae, Lee, In Ho, Park, Chang Joon, Kim, Jae Hean
• Granted Patent: US 8,077,906 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

A system and method for supporting augmented reality in an ocean scene is provided. According to the system and method, camera rotation information is extracted from horizontal line information of a sea extracted from an original image including a focal length and at least two first natural feature point tracking data existing on a sea plane and a plane parallel to the sea plane, using long-distance background features; camera movement is sequentially calculated from the camera rotation information and at least two natural feature point tracking data existing on the sea plane of the original image; and a computer graphics object is placed on a ground plane by importing camera position data, and a rendering is performed as moving the camera. Accordingly, the camera rotation and the camera movement can be accurately calculated, assuming that the camera focal length information is known at the time of capturing a moving image with a sea image for a background, and thus the CG/real scene synthesis with the sea for a background can be accomplished more realistically and accurately during the production of videos such as movie and drama.

Claims

1. An apparatus for extracting a camera motion, comprising:

a camera rotation information extraction means for acquiring a tilt and a yaw of a camera by extracting a horizontal line of a sea from an original image, and acquiring a pan of the camera using the acquired yaw and long-distance background features of at least two first natural feature point tracking data existing in the original image; and

a camera movement calculation means for calculating the camera movement from rotation angles obtained from the tilt, yaw, and pan of the camera and at least two second natural feature point tracking data existing on a sea plane of the original image.

2. The apparatus of claim 1, wherein the first natural feature points exist on a plane parallel to a sea plane.

3. The apparatus of claim 1, wherein the extraction of the horizontal line of the sea is performed through a Canny edge operation and a Radon transform.

4. The apparatus of claim 1, wherein the camera movement calculation means comprises:

a means for acquiring a camera position at a first frame on the basis of one natural feature point p that exists on a sea plane at the first frame, using the tilt, yaw, and pan of the camera;

a means for acquiring spatial coordinates of another natural feature point q that exists on the sea plane, using the camera position at the first frame; and

a means for acquiring the camera movement at the following frame, using the spatial coordinates of the natural feature points p and q and the natural feature points at the respective frame images.

5. A system for supporting augmented reality in an ocean scene using an apparatus for extracting a camera motion, the system comprising:

a camera rotation information extraction means for extracting camera rotation information from horizontal line information of a sea extracted from an original image and at least two first natural feature point tracking data existing on a plane parallel to a sea plane, using long-distance background features;

a camera movement calculation means for calculating camera movement from the camera rotation information and at least two second natural feature point tracking data existing on the sea plane of the original image; and

a means for placing a computer graphics object on a ground plane by importing camera position data, and performing a rendering according to the movement of the camera.

6. The system of claim 5, wherein the camera rotation information extraction means comprises:

a means for extracting the horizontal line and the first natural feature point tracking points from the image;

a means for acquiring a tilt and a yaw of the camera from a normal vector of the sea plane; and

a means for acquiring a pan of the camera using the tilt and the yaw of the camera and the tracking points.

7. The system of claim 5, wherein the camera movement calculation means comprises:

a means for acquiring a camera position at a first frame on the basis of one natural feature point p that exists on a sea plane at the first frame, using the camera rotation information;

a means for acquiring spatial coordinates of another natural feature point q that exists on the sea plane, using the camera position at the first frame; and

a means for acquiring the camera movement at the following frame, using the spatial coordinates of the natural feature points p and q and the natural feature points at the respective frame images.

8. A method of supporting augmented reality in an ocean scene using an apparatus for extracting a camera motion, the method comprising:

(a) extracting camera rotation information from horizontal line information of a sea extracted from an original image and at least two first natural feature point tracking data existing on a sea plane and a plane parallel to the sea plane, using long-distance background features;

(b) calculating camera movement from the camera rotation information and at least two second natural feature point tracking data existing on the sea plane of the original image; and

(c) placing a computer graphics object on a ground plane by importing camera position data, and performing a rendering according to the movement of the camera.

9. The method of claim 8, wherein the step (a) comprises:

extracting the horizontal line information from the input image;

acquiring a tilt and a yaw of the camera from a normal vector of the sea plane; and

acquiring a pan of the camera using the tilt and the yaw of the camera and the externally inputted natural feature point tracking data.

10. The method of claim 8, wherein the step of extracting the horizontal line information comprises:

designating a region of interest (ROI) with respect to the input image;

transforming the designated ROI into a grayscale;

obtaining edges by performing a Canny edge operation; and

extracting the horizontal line information through a Radon transform.

11. The method of claim 8, wherein the step (b) comprises:

acquiring a camera position at a first frame on the basis of one natural feature point p that exists on a sea plane at the first frame, using the camera rotation information;

acquiring spatial coordinates of another natural feature point q that exists on the sea plane, using the camera position at the first frame; and

acquiring the camera movement at the following frame, using the spatial coordinates of the natural feature points p and q and the natural feature points at the respective frame images.