MARKERLESS AUGMENTED REALITY SYSTEM AND METHOD USING PROJECTIVE INVARIANT

• US 20110090252A1
• Filed: 10/15/2010
• Published: 04/21/2011
• Est. Priority Date: 10/20/2009
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd. (Samsung Group)
• Original Assignee: Samsung Electronics Co. Ltd. (Samsung Group)
• Inventors: Ahn, Sung Hwa, YOON, Suk June, Hyung, Seung Yong, Roh, Kyung Shik
• Granted Patent: US 8,791,960 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments

Abstract

Disclosed herein are a markerless augmented reality system and method for extracting feature points within an image and providing augmented reality using a projective invariant of the feature points. The feature points are tracked in two images photographed while varying the position of an image acquisition unit, a set of feature points satisfying a plane projective invariant is obtained from the feature points, and augmented reality is provided based on the set of feature points. Accordingly, since the set of feature points satisfies the plane projective invariant even when the image acquisition unit is moved and functions as a marker, a separate marker is unnecessary. In addition, since augmented reality is provided based on the set of feature points, a total computation amount is decreased and augmented reality is more efficiently provided.

Claims

1. A markerless augmented reality system receiving an image from an image device, the markerless augmented reality system comprising:

an image processing unit to separate the image acquired by the image device and to extract feature points;

a plane extraction unit to extract a plane using a plane projective invariant of the feature points; and

an augmented reality provision unit to provide augmented reality based on the extracted plane.

2. The markerless augmented reality system according to claim 1, further comprises a storage unit to store two images taken by changing a position of the image device.

3. The markerless augmented reality system according to claim 1, wherein the image device is a single camera to acquire image information of an object at different positions.

4. The markerless augmented reality system according to claim 1, wherein the image processing unit equally separates an initial image I_t acquired at a position before the image device is moved and extracts the feature points.

5. The markerless augmented reality system according to claim 4, wherein the image processing unit further includes a feature point tracking unit to track the extracted feature points in a next image I_t+1 acquired at a position to which the image device is moved.

6. The markerless augmented reality system according to claim 5, wherein the plane extraction unit obtains a set of feature points satisfying the plane projective invariant from the tracked feature points and extracts the plane.

7. The markerless augmented reality system according to claim 6, wherein the set of feature points is obtained by generating a set of feature points in descending order of index values thereof and computing the plane projective invariant.

8. The markerless augmented reality system according to claim 6, wherein the augmented reality provision unit provides the augmented reality based on the set of feature points satisfying the plane projective invariant.

9. A markerless augmented reality system using a projective invariant receiving an image from an image device, the markerless augmented reality system comprising:

an image processing unit to extract and track feature points from the plurality of images acquired by the image device;

a plane extraction unit to obtain a set of feature points satisfying a plane projective invariant from the tracked feature points and to extract a plane; and

an augmented reality provision unit to provide augmented reality based on the set of feature points satisfying the plane projective invariant.

10. The markerless augmented reality system of claim 9, wherein the plurality of images photographed at a different positions of the image device.

11. The markerless augmented reality system of claim 9, further comprising:

a storage unit to store the plurality of images.

12. The markerless augmented reality system according to claim 9, wherein the plurality of images includes an initial image I_t acquired at a position before the image device is moved and a next image I_t+1 acquired at a position to which the image device is moved.

13. The markerless augmented reality system according to claim 9, wherein the image processing unit equally separates the initial image I_t, extracts the feature points and tracks the extracted feature points in the next image I_t+1.

14. The markerless augmented reality system according to claim 9, wherein the set of feature points is obtained by generating a set of feature points in descending order of index values thereof and computing the plane projective invariant.

15. A markerless augmented reality method using a projective invariant, the markerless augmented reality method comprising:

photographing an object using an image device and acquiring an initial image I_t;

equally separating the acquired initial image I_t and extracting feature points;

moving the image device and acquiring a next image l_t+1;

tracking the extracted feature points in the next and obtaining a set of feature points satisfying a plane projective invariant; and

providing augmented reality based on the set of feature points satisfying the plane projective invariant.

16. The markerless augmented reality method according to claim 15, wherein the image device acquires a plurality of images photographed at different positions.

17. The markerless augmented reality method according to claim 15, wherein the set of feature points is obtained by generating a set of feature points in descending order of index values thereof and computing the plane projective invariant.

18. The markerless augmented reality method according to claim 15, wherein the obtaining of the set of feature points satisfying the plane projective invariant includes computing the plane projective invariant using a world coordinate of a feature point expressed on an object plane and an image coordinate of a feature point expressed on an image plane in correspondence therewith, obtaining the set of feature points satisfying the plane projective invariant, and extracting a plane.

19. The markerless augmented reality method according to claim 15, wherein the image device is a camera.