METHOD AND MOBILE APPARATUS FOR DISPLAYING AN AUGMENTED REALITY

• US 20120216149A1
• Filed: 09/23/2011
• Published: 08/23/2012
• Est. Priority Date: 02/18/2011
• Status: Abandoned Application

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Choi, Seung-eok, Ju, Hak-soo, Kim, Sin-ae, KANG, Nam-wook, Ryu, Jong-hyun
• Application Number:
  US13/242,935
• Publication Number:
  US 20120216149A1
• Time in Patent Office: Days
• US Class Current: 715/848
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A mobile apparatus and method for displaying an Augmented Reality (AR) in the mobile apparatus. The mobile apparatus captures an image of a current environment of the mobile apparatus, displays the image, detects mapping information corresponding to the current environment from among mapping information stored in the mobile apparatus, maps a three-dimensional (3D) Graphical User Interface (GUI) of detected mapping information onto the displayed image, based on a relative location relationship between the detected mapping information, and adjusts a display status of the 3D GUI, while maintaining the relative location relationship between the detected mapping information.

Claims

1. A method for displaying an Augmented Reality (AR) in a mobile apparatus, the method comprising the steps of:

capturing, by the mobile apparatus, an image of a current environment of the mobile apparatus;

displaying the image;

detecting mapping information corresponding to the current environment from among mapping information stored in the mobile apparatus;

mapping a three-dimensional (3D) Graphical User Interface (GUI) of detected mapping information onto the displayed image, based on a relative location relationship between the detected mapping information; and

adjusting a display status of the 3D GUI, while maintaining the relative location relationship between the detected mapping information.

2. The method of claim 1, wherein adjusting the display status of the 3D GUI comprises:

receiving a user manipulation input from a user of the mobile apparatus; and

adjusting the display status of the 3D GUI according to the user manipulation.

3. The method of claim 1, wherein adjusting the display status of the 3D GUI comprises:

receiving a touch and drag user manipulation input on a screen of the mobile apparatus; and

rotating the 3D GUI according to a direction of the touch and drag user manipulation input so as to adjust a display location of the detected mapping information.

4. The method of claim 1, wherein adjusting the display status of the 3D GUI comprises:

receiving a user manipulation input from a user of the mobile apparatus that changes a location or direction of the mobile apparatus; and

adjusting the display status of the 3D GUI according to the user manipulation input.

5. The method of claim 1, wherein the mapping information includes at least one of geographical information in an area corresponding to the current environment, search information in relation to the geographical information, related information obtained in relation to an activity performed in the area corresponding to the current environment.

6. The method of claim 5, wherein the related information includes at least one of use information related to credit card activity in the area corresponding to the current environment, image data related to a photograph taken within the area, and message information, Social Networking Service (SNS) information, and e-mail information related to messages, SNS activity, and e-mails that have been transmitted or received within the area, and text or image file information related to text or image files that have been made or read within the area.

7. The method of claim 1, wherein detecting the mapping information corresponding to the current environment comprises detecting mapping information from among the stored mapping information, which has location coordinates belonging to a location range identifying the area corresponding to the current environment.

8. The method of claim 7, wherein mapping the 3D GUI of the detected mapping information comprises:

comparing the location coordinates of the detected mapping information to set the relative location relationship;

arranging other mapping information at a display direction and distance that is determined according to the relative location relationship, based on a location of one of the detected mapping information so as to form the 3D GUI; and

mapping the 3D GUI to the current environment.

9. The method of claim 8, wherein adjusting the display status of the 3D GUI comprises:

receiving a user manipulation input from a user of the mobile apparatus; and

rotating, reducing, and enlarging the 3D GUI, while maintaining the display direction and distance between the mapping information.

10. A mobile apparatus for displaying an Augmented Reality (AR), the mobile apparatus comprising:

a camera that captures an image of a current environment of the mobile apparatus;

a display that displays the image of the current environment along with a three-dimensional (3D) GUI of detected mapping information;

a memory that stores mapping information;

a Graphical User Interface (GUI) processor that detects mapping information corresponding to the current environment from among the mapping information stored in the memory and maps the 3D GUI of the detected mapping information on the current environment, based on a relative location relationship between the detected mapping information; and

a controller that controls the GUI processor to adjust a display status of the 3D GUI, while maintaining the relative location relationship between the detected mapping information.

11. The mobile apparatus of claim 10, further comprising a input device that receives a user manipulation input from a user of the mobile apparatus,

wherein user manipulation input commands the controller to control the GUI processor to adjust the display status of the 3D GUI according to the user manipulation input.

12. The mobile apparatus of claim 11, wherein the input device comprises a touch screen,

wherein when the user manipulation input includes a touch and drag on the touch screen of the mobile apparatus, the controller portion controls the GUI processor to rotate the 3D GUI according to a direction of the touch and drag so as to adjust a display location of the detected mapping information.

13. The mobile apparatus of claim 10, wherein when a user changes a location or direction of the mobile apparatus, the controller controls the GUI processor to adjust the display status of the 3D GUI according to a change of the location or direction.

14. The mobile apparatus of claim 10, wherein the mapping information comprises at least one of:

geographical information in an area corresponding to the current environment;

search information in relation to the geographic information; and

related information in relation to an activity performed in the area corresponding to the current environment.

15. The mobile apparatus of claim 14, wherein the related information comprises at least one of”

use information related to credit card activity in the area corresponding to the current environment;

image data related to a photograph taken within the area; and

message information, Social Networking Service (SNS) information, and e-mail information related to messages, SNS activity, and e-mails that have been transmitted or received within the area, and text or image file information related to text or image files that have been made or read within the area.

16. The mobile apparatus of claim 10, wherein the GUI processor detects mapping information that has location coordinates belonging to a location range identifying the area corresponding to the current environment, compares the location coordinates of the detected mapping information to set the relative location relationship, and arranges other mapping information at a display direction and distance that is determined according to the relative location relationship, based on a location of one of the detected mapping information so as to form the 3D GUI.

17. The mobile apparatus of claim 16, further comprising an input device that receives a user manipulation input from a user of the mobile apparatus,

wherein when the input device receives the user manipulation input, the controller rotates, reduces, and enlarges the 3D GUI, while maintaining the display direction and distance between the mapping information.