REAL-TIME INTERACTIVE AUGMENTED REALITY SYSTEM AND METHOD AND RECORDING MEDIUM STORING PROGRAM FOR IMPLEMENTING THE METHOD

• US 20110216090A1
• Filed: 12/14/2010
• Published: 09/08/2011
• Est. Priority Date: 03/03/2010
• Status: Abandoned Application

• Current Assignee: Gwangju Institute of Science and Technology
• Original Assignee: Gwangju Institute of Science and Technology
• Inventors: Baek, Woon Hyuk, Kim, Ki Young, Park, Young Min, WOO, Woon Tack
• Application Number:
  US12/968,070
• Publication Number:
  US 20110216090A1
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T

Abstract

The present invention relates to a real-time interactive system and method regarding an interactive technology between miniatures in real environment and digital contents in virtual environment, and a recording medium storing a program for performing the method. An exemplary embodiment of the present invention provides a real-time interactive augmented reality system including: an input information acquiring unit acquiring input information for an interaction between real environment and virtual contents in consideration of a planned story; a virtual contents determining unit determining the virtual contents according to the acquired input information; and a matching unit matching the real environment and the virtual contents by using an augmented position of the virtual contents acquired in advance. According to the present invention, an interaction between the real environment and the virtual contents can be implemented without tools, and improved immersive realization can be obtained by augmentation using natural features.

Claims

1. An interactive augmented reality system comprising:

an input information acquiring unit acquiring input information for an interaction between real environment and virtual contents in consideration of planned story;

a virtual contents determining unit determining the virtual contents according to the acquired input information; and

a matching unit matching the real environment and the virtual contents by using an augmented position of the virtual contents acquired in advance.

2. The interactive augmented reality system according to claim 1, further comprising:

a virtual contents authoring unit authoring the virtual contents on the basis of GUI (graphical user interface).

3. The interactive augmented reality system according to claim 2, wherein:

the virtual contents authoring unit includes a virtual contents aligning unit aligning the virtual contents to be proportion to the sizes of objects in the real environment when authoring the virtual contents.

4. The interactive augmented reality system according to claim 3, wherein:

the virtual contents aligning unit includes

an augmented position setting unit setting the augmented position,

a virtual contents loading unit loading the virtual contents to the set augmented position,

a scale setting unit determining a scale of the loaded virtual contents, and

a pose setting unit determining the pose of the virtual contents on which the scale setting has been performed.

5. The interactive augmented reality system according to claim 1, wherein:

the matching unit includes an interaction analyzing unit analyzing the interaction between the real environment and the virtual contents on the basis of depth information of at least one object positioned in the real environment.

6. The interactive augmented reality system according to claim 1, further comprising:

a position computing unit computing the augmented position on the basis of features in an acquired image.

7. The interactive augmented reality system according to claim 6, further comprising:

a first feature matching unit comparing a restored image regarding the real environment and the acquired image and performing feature matching through the comparison, and

a second feature matching unit comparing the previous image of the acquired image and the acquired image and performing feature matching through the comparison,

wherein, when the position computing unit computes the augmented position, the first feature matching unit and the second feature matching unit are driven at the same time.

8. The interactive augmented reality system according to claim 7, wherein:

the first feature matching unit includes

an object recognizing unit primarily recognizing at least one object having a scale equal to or greater than a reference value in the acquired image and secondarily recognizing at least one object a scale less than the reference value in the acquired image,

a comparing unit adding the number of objects primarily recognized to the number of objects secondarily recognized and comparing the number of recognized objects with a predetermined limit value,

a recognition control unit making the object recognizing unit recognize objects in the acquired image one more time if the number of recognized objects is greater than the limit value,

a pose computing unit computing poses of the recognized objects by using the restored image if the number of recognized objects is not greater than the limit value, and

an object information acquiring unit acquiring information on objects corresponding to inliers among the objects on which the pose computation has been performed according to a coordinate system generating algorithm based on RANSAC (random sample consensus).

9. The interactive augmented reality system according to claim 7, wherein:

the second feature matching unit includes

a determining unit determining whether an object acquired in advance is in the acquired image,

a feature matching unit performing feature matching on the basis of an SIFT (scale invariant feature transform) algorithm if an object acquired in advance is in the acquired image,

a comparing unit comparing the number of matched features with a reference value,

a pose computing unit computing a pose of an image acquiring device for acquiring images by using a previous image if the number of matched features is equal to or greater than the reference value,

an image acquisition control unit performs a function of making the image acquiring device acquire an image again if any object acquired in advance is not in the acquired image or the number of matched features is less than the reference value, and

an object information acquiring unit acquiring information on objects corresponding to inliers in the acquired image by the image acquiring device on which the pose computation has been performed.

10. The interactive augmented reality system according to claim 6, wherein:

the position computing unit includes

a feature detecting unit detecting the features by using color information of the acquired image or selectively detecting the features in uniform spaces on the basis of observation probabilities, and

a tracking unit tracking the position of an image acquiring device for acquiring images by using edge information of objects or 2D information of the objects in addition to the features.

11. The interactive augmented reality system according to claim 1, wherein:

the matching unit uses depth information of an acquired image or context information of the acquired image when matching the virtual contents.

12. A method of implementing an interaction in augmented reality comprising:

(a) acquiring input information for an interaction between real environment and virtual contents in consideration of a planned story;

(b) determining the virtual contents according to the acquired input information; and

(c) matching the real environment and the virtual contents by using an augmented position of the virtual contents acquired in advance.

13. The method of implementing an interaction in augmented reality according to claim 12, further comprising:

(b1) authoring the virtual contents on the basis of GUI (graphical user interface) between the (b) and the (c).

14. The method of implementing an interaction in augmented reality according to claim 13, wherein:

the (b1) aligns the virtual contents to be proportion to the sizes of objects in the real environment when authoring the virtual contents

15. The method of implementing an interaction in augmented reality according to claim 14, wherein:

the aligning of the virtual contents in the (b1) includes

(b11) setting the augmented position,

(b12) loading the virtual contents to the set augmented position,

(b13) determining a scale of the loaded virtual contents, and

(b14) determining the pose of the virtual contents on which the scale setting has been performed.

16. The method of implementing an interaction in augmented reality according to claim 12, further comprising:

(b1) computing the augmented position on the basis of features in an acquired image between the (b) and the (c).

17. The method of implementing an interaction in augmented reality according to claim 16, wherein:

the (b1) includes

(b11) comparing a restored image regarding the real environment and the acquired image and performing feature matching through the comparison, and

(b12) a second feature matching unit comparing the previous image of the acquired image and the acquired image and performing feature matching through the comparison, and

computes the augmented position.

18. The method of implementing an interaction in augmented reality according to claim 17, wherein:

the (b11) includes

(b11a) primarily recognizing at least one object having a scale equal to or greater than a reference value in the acquired image and secondarily recognizing at least one object a scale less than the reference value in the acquired image,

(b11b) adding the number of objects primarily recognized to the number of objects secondarily recognized and comparing the number of recognized objects with a predetermined limit value,

(b11c) making the object recognizing unit recognize objects in the acquired image one more time if the number of recognized objects is greater than the limit value, and computing poses of the recognized objects by using the restored image if the number of recognized objects is not greater than the limit value, and

(b11d) acquiring information on objects corresponding to inliers among the objects on which the pose computation has been performed according to a coordinate system generating algorithm based on RANSAC (random sample consensus).

19. The method of implementing an interaction in augmented reality according to claim 17, wherein:

the (b12) includes

(b12a) determining whether an object acquired in advance is in the acquired image,

(b12b) performing feature matching on the basis of an SIFT (scale invariant feature transform) algorithm if an object acquired in advanced is in the acquired image, and making the image acquiring device for acquiring images acquire an image again if any object acquired in advance is not in the acquired image,

(b12c) comparing the number of matched features with a reference value,

(b12d) computing a pose of the image acquiring device by using the previous image if the number of matched features is equal to or greater than the reference value, and making the image acquiring device acquire an image again if the number of matched features is less than the reference value, and

(b12e) acquiring information on objects corresponding to inliers in the acquired image by the image acquiring device on which the pose computation has been performed.