Tabletop, mobile augmented reality system for personalization and cooperation, and interaction method using augmented reality

• US 8,823,697 B2
• Filed: 02/11/2009
• Issued: 09/02/2014
• Est. Priority Date: 02/12/2008
• Status: Active Grant

• Current Assignee: Intellectual Discovery Co., Ltd.
• Original Assignee: Gwangju Institute of Science and Technology
• Inventors: Woo, Woon Tack, Na, Se Won
• Primary Examiner(s): McDowell, Jr., Maurice L
• Assistant Examiner(s): Zhai, Kyle
• Application Number:
  US12/867,420
• Publication Number:
  US 20100315418A1
• Time in Patent Office: 2,029 Days
• Field of Search: 345/633, 345/419
• US Class Current: 345/419
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A tabletop, mobile augmented reality system for personalization and cooperation and an interaction method using augmented reality is presented. More particularly, the tabletop, mobile augmented reality system for personalization and cooperation adds a mobile interface for providing a personal user space to a table interface for providing a cooperative space for users such that information on a table can be brought to a personal mobile device and controlled directly or manipulated personally by a user, allows for fairness in the right to control the table, and permits users to share the result of personalization.

Claims

1. A mobile interface apparatus, cooperating with a table top interface apparatus, for providing an augmented reality, comprising:

a camera module;

a display module;

a communication module communicating with the table top interface apparatus, wherein the table top interface apparatus comprises a table top display device providing a cooperative space on which at least one real object is disposed; and

a control module configured to;

detect, via the camera module, position information and orientation information of the real object,

display, via the display module, an image of a virtual object onto the real object according to the position information and the orientation information, wherein the table top interface apparatus displays, via the table top display device, an image of a background scene related to the virtual object on the cooperative space,

receive, via the communication module, manipulation information being used for modifying the virtual object from the table top interface apparatus, and

adjusting the image of the virtual object based on the manipulation information, wherein the table top interface apparatus modifies the image of the background scene considering the adjusted image of the virtual object,

wherein a spatial coordinate system of the mobile interface apparatus and a spatial coordinate system of the table top interface apparatus are matched by exchanging coordinate data of the real object between the mobile interface apparatus and the table top interface apparatus under temporal synchronization.

2. The apparatus of claim 1, wherein the manipulation information includes at least one of size information, height information and rotation information of the virtual object.

3. The apparatus of claim 1, wherein the control module displays, via the display module, the image of the virtual object at a specific view point determined based on the orientation information, and wherein the control module performs at least one of a size change, a height change and a rotation of the displayed image of the virtual object based on the manipulation information.

4. The apparatus of claim 1, wherein the table top display device is a touch display panel receiving a touch input on the cooperative space from a user, and wherein the table top interface apparatus displays, via the touch display panel, at least one soft key on the cooperative space and obtains the manipulation information based on the touch input to the soft key.

5. The apparatus of claim 1, wherein the table top interface apparatus further comprises a camera detecting a touch gesture on the cooperative space from a user, and wherein the table top interface apparatus displays, via the table top display device, at least one soft key on the cooperative space and obtains the manipulation information based on the touch gesture to the soft key.

6. The apparatus of claim 1, wherein the virtual object is a building and the background scene is a map.

7. A method for providing an augmented reality, performed in a mobile interface apparatus cooperating with a table top interface apparatus, comprising:

detecting position information and orientation information of at least one real object, wherein the real object is disposed on a cooperative space provided by a table top display device of the table top interface apparatus;

displaying an image of a virtual object onto the real object according to the position information and the orientation information, wherein the table top apparatus displays, via the table top display device, an image of a background scene related to the virtual object on the cooperative space;

receiving manipulation information being used for modifying the virtual object from the table top interface apparatus; and

adjusting the image of the virtual object based on the manipulation information, wherein the table top interface apparatus modifies the image of the background scene considering the adjusted image of the virtual object,

wherein a spatial coordinate system of the mobile interface apparatus and a spatial coordinate system of the table top interface apparatus are matched by exchanging coordinate data of the real object between the mobile interface apparatus and the table top interface apparatus under temporal synchronization.

8. The method of claim 7, wherein the manipulation information includes at least one of a size information, a height information and a rotation information of the virtual object.

9. The method of claim 7, wherein the table top display device is a touch display panel receiving a touch input on the cooperative space from a user, and wherein the table top interface apparatus displays, via the touch display panel, at least one soft key on the cooperative space and obtains the manipulation information based on the touch input to the soft key.

10. The method of claim 7, wherein the table top interface apparatus further comprises a camera detecting a touch gesture on the cooperative space from a user, and wherein the table top interface apparatus displays, via the table top display device, at least one soft key on the cooperative space and obtains the manipulation information based on the touch gesture to the soft key.

11. A table top interface apparatus, cooperating with at least one mobile interface apparatus, for providing an augmented reality, comprising:

a table top display panel providing a cooperative space on which at least one real object is disposed;

a communication module communicating with the at least one mobile interface apparatus, wherein the at least one mobile interface apparatus detects position information and orientation information of the real object and displays an augmented image including an image of a virtual object onto the real object based on the position information and orientation information; and

a control module configured to;

display, via the table top display panel, an image of a background scene related to the virtual object on the cooperative space,

obtain manipulation information being used for modifying the virtual object;

transmit, via the communication module, the manipulation information to the at least one mobile interface apparatus, wherein the at least one mobile interface apparatus adjusts the image of the virtual object based on the manipulation information, and

modifying the image of the background scene considering the adjusted image of the virtual object,

wherein a spatial coordinate system of the mobile interface apparatus and a spatial coordinate system of the table top interface apparatus are matched by exchanging coordinate data of the real object between the mobile interface apparatus and the table top interface apparatus under temporal synchronization.

12. The apparatus of claim 11, wherein the manipulation information includes at least one of size information, height information and an rotation information of the virtual object.

13. The apparatus of claim 11, wherein the table top display panel is a touch display panel receiving a touch input on the cooperative space from a user, and wherein the at least one mobile interface apparatus displays, via the touch display panel, at least one soft key on the cooperative space and obtains the manipulation information based on the touch input to the soft key.

14. The apparatus of claim 11, wherein the control module receives, via the communication module, the manipulation information from one of the at least one mobile interface apparatus.