METHOD FOR PROVIDING AUGMENTED REALITY, SERVER FOR SAME, AND PORTABLE TERMINAL

• US 20120122491A1
• Filed: 06/29/2010
• Published: 05/17/2012
• Est. Priority Date: 07/30/2009
• Status: Active Grant

• Current Assignee: SK Planet Co., Ltd. (SK Telecom Co., Ltd.)
• Original Assignee: SK Planet Co., Ltd. (SK Telecom Co., Ltd.)
• Inventors: Kim, Namhoon, Im, Jungbin
• Granted Patent: US 9,130,999 B2
• Time in Patent Office: Days
• US Class Current: 455/456.3
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

The present disclosure provides a method for providing an augmented reality, to server and portable terminal for same. Augmented reality providing server provides an augmented reality with information overlaid on captured video and includes server communication unit for inter-working with portable terminal; and server control unit for receiving selected menu information from portable terminal, generating first menu information data attained by extracting data that matches selected menu information by comparing the selected menu information with pre-stored location-associated information, and transmitting first menu information data to portable terminal, wherein the server control unit generates the first menu information data by selecting menus that match the selected menu information among a plurality of menus included in the location-associated information. In an embodiment, fast augmented reality operation can be obtained since portable terminal may receive and output information corresponding to menu selection immediately with a composition onto acquired image information.

Claims

1. An augmented reality providing server for providing an augmented reality with information overlaid on a captured video, the server comprising:

a server communication unit for inter-working with a portable terminal; and

a server control unit for receiving a selected menu information from the portable terminal, generating a first menu information data attained by extracting data that matches the selected menu information by comparing the selected menu information with a location-associated information that have been previously stored, and transmitting a generated first menu information data to the portable terminal, wherein the server control unit generates the first menu information data by selecting menus that match the selected menu information among a plurality of menus included in the location-associated information.

2. The augmented reality providing server of claim 1, wherein the server control unit transmits a real picture image information included in the location-associated information to the portable terminal.

3. The augmented reality providing server of claim 2, wherein the server control unit transmits the first menu information attained by extracting one or more kinds of information that matches the selected menu information among a latitude-longitude coordinate information corresponding to the real picture image information, point of interest (POI) information, advertisement information, name information, phone number information, promotion information, Internet link information, and tagging information, to the portable terminal.

4. The augmented reality providing server of claim 1, wherein the server control unit calculates a current location information of the portable terminal by using a position information received from the portable terminal, generates a third menu information data by extracting data that matches the current location information and the selected menu information among the location-associated information that is related with the range within a preset radius from the current location information, and sends the third menu information data to the portable terminal.

5. The augmented reality providing server of claim 1, wherein the server control unit calculates a current location information of the portable terminal based on a location information of a base station synchronized with the portable terminal, generates a fourth menu information data by extracting data that matches the current location information and the selected menu information among the location-associated information that is related with the range within a preset radius from the current location information, and sends the fourth menu information data to the portable terminal.

6. The augmented reality providing server of claim 1, wherein the server control unit compares a feature point received from the portable terminal with a pre-stored location-associated information for extracting a local information data matched with the feature point, and further transmits the local information data to the portable terminal.

7. The augmented reality providing server of claim 1, wherein the server control unit extracts a feature point from an image information from the portable terminal, extracts a local information data matched with an extracted feature point by comparing the extracted feature point to the location-associated information, and further transmits the local information data to the portable terminal, and

the portable terminal transmits the image information to the server communication unit by sending only a part of the captured video as the image information.

8. The augmented reality providing server of claim 7, wherein the image information is information attained from extracting selected one or more frames including initial frame 1 or frame 2, a key frame, and a video frame upon occurrence of a specific event deviating from a preset condition.

9. The augmented reality providing server of claim 7, wherein the server control unit extracts the feature point based on one or more kinds of information selected from a color information, edge information, outline information, texture information, text information, and contour information depending on a specified angle with respect to a specific object, which are contained in the image information.

10. The augmented reality providing server of claim 6, wherein the server control unit transmits the local information data containing one or more kinds of information including a name information, phone number information, POI information, advertisement information, promotion information, Internet link information, and tagging information, to the portable terminal.

11. The augmented reality providing server of claim 6, further comprising a server storage unit for storing the location-associated information including one or more kinds of information including a real picture image information, latitude-longitude coordinate information corresponding to the real picture image information, point of interest (POI) information, and advertisement information.

12. The augmented reality providing server of claim 6, wherein the server control unit calculates a current location information of the portable terminal by using a position information received from the portable terminal, and extracts the local information data based on a calculated current location information.

13. The augmented reality providing server of claim 6, wherein the server control unit extracts the local information data based on a location information of a base station synchronized with the portable terminal.

14. A portable terminal provided with an augmented reality by inter-working with an augmented reality providing server, the terminal comprising:

a display unit for displaying an application under execution;

a terminal communication unit for inter-working with the augmented reality providing server;

a camera module for acquiring an image information corresponding to a still image or a video;

a key input unit for receiving an input of a user command; and

a terminal control unit for transmitting a menu information selected by the user command to the augmented reality providing server, downloading a first menu information data corresponding to the menu information from the augmented reality providing server, and displaying the image information combined with the first menu information data through the display unit.

15. The portable terminal of claim 14, wherein the terminal control unit combines the acquired image information with one or more kinds of information matching the selected menu information among a latitude-longitude coordinate information corresponding to a real picture image information to the portable terminal, point of interest (POI) information, advertisement information, name information, phone number information, promotion information, internet link information, and tagging information.

16. The portable terminal of claim 14, wherein the terminal control unit extracts a feature point from the acquired image information, generates a second menu information data attained from extracting a data matched with an extracted feature point by comparing the extracted feature point to the downloaded first menu information data, and combines a generated second menu information data with the acquired image information.

17. The portable terminal of claim 16, wherein the terminal control unit extracts the feature point based on one or more kinds of information selected from a color information, edge information, outline information, texture information, text information, and contour information depending on a specified angle with respect to a specific object, which are contained in the acquired image information.

18. The portable terminal of claim 16, further comprising a GPS module for generating a position information based on a navigation data included in a radio wave, wherein the terminal control unit transmits a the position information to the augmented reality providing server, downloading a third menu information data corresponding to the menu information and the position information from the augmented reality providing server, and combines a downloaded third menu information data with the image information.

19. The portable terminal of claim 14, wherein the terminal control unit extracts a feature point from the acquired image information, transmits an extracted feature point to the augmented reality providing server, receives a local information data corresponding to the feature point from the augmented reality providing server, and further comprises outputting the image information combined with the local information data through the display unit.

20. The portable terminal of claim 19, wherein the terminal control unit extracts the feature point based on one or more kinds of information selected from a color information, edge information, outline information, texture information, text information, and contour information depending on a specified angle with respect to a specific object, which are contained in the image information.

21. The portable terminal of claim 14, further comprising transmitting the acquired image information to the augmented reality providing server, receiving a local information data corresponding to a transmitted image information from the augmented reality providing server, and outputting the acquired image information combined with a received local information data through the display unit.

22. The portable terminal of claim 21, wherein the terminal control unit extracts selected one or more frames including initial frame 1 or frame 2, a key frame, and a video frame upon occurrence of a specific event deviating from a preset condition from the acquired image information, and transmits an extracted frame or frames to the augmented reality providing server.

23. The portable terminal of claim 19, further comprising a GPS module for generating a position information based on a navigation data included in a radio wave, wherein the terminal control unit transmits a generated position information to the augmented reality providing server, receives a current location information corresponding to a transmitted position information from the augmented reality providing server, and combines the received local information data with the acquired image information based on a received current location information.

24. The portable terminal of claim 19, wherein the terminal control unit combines the acquired image information with the local information data containing one or more kinds of information including a name information, phone number information, POI information, advertisement information, promotion information, internet link information, and tagging information.

25. A method for providing an augmented reality with information overlaid on a captured video, the method comprising:

receiving a selected menu information from a portable terminal;

generating a first menu information data attained by extracting data that matches the selected menu information by comparing the selected menu information with a location-associated information that has been previously stored;

transmitting a real picture image information included in the location-associated information to the portable terminal; and

transmitting the first menu information data with an incorporation of one or more kinds of information that matches the selected menu information among a latitude-longitude coordinate information corresponding to the real picture image information, point of interest (POI) information, advertisement information, name information, phone number information, promotion information, internet link information, and tagging information, to the portable terminal.

26. An augmented reality providing server for providing an augmented reality with information overlaid on a captured video, the server comprising:

a server communication unit for inter-working with a portable terminal; and

a server control unit for receiving a selected local information from the portable terminal, generating a first local information data attained by extracting data that matches the selected local information by comparing the received selected local information with a location-associated information that have been previously stored, and transmitting the first local information data to the portable terminal,

wherein the server control unit generates the first local information data containing all of a plurality of menus included in a specified locale after specifying the selected local information from a plurality of locales contained in the location-associated information.

27. The augmented reality providing server of claim 26, wherein the server control unit transmits a real picture image information of a locale corresponding to the selected local information from the location-associated information to the portable terminal.

28. The augmented reality providing server of claim 26, wherein the server control unit transmits the first menu information data containing one or more kinds of information from a latitude-longitude coordinate information corresponding to the real picture image information of the locale corresponding to the selected local information, point of interest (POI) information, advertisement information, name information, phone number information, promotion information, internet link information, and tagging information, to the portable terminal.

29. A portable terminal for downloading augmented reality information by inter-working with an augmented reality providing server, the terminal comprising:

a display unit for displaying an application under execution;

a terminal communication unit for inter-working with the augmented reality providing server;

a camera module for acquiring an image information corresponding to a still image or a video;

a key input unit for receiving an input of a user command; and

a terminal control unit for transmitting a local information selected by the user command to the augmented reality providing server, downloading a first local information data corresponding to the local information from the augmented reality providing server, and displaying the image information combined with the first local information data through the display unit.

30. The portable terminal of claim 29, wherein the terminal control unit combines the acquired image information with one or more kinds of information from a latitude-longitude coordinate information corresponding to a real picture image information of the locale corresponding to the selected local information contained in the downloaded second local information data, point of interest (POI) information, advertisement information, name information, phone number information, promotion information, internet link information, and tagging information.

31. The portable terminal of claim 29, further comprising a GPS module for generating a position information based on a navigation data included in a radio wave, wherein the terminal control unit transmits a generated position information to the augmented reality providing server, receives a current location information corresponding to a transmitted position information from the augmented reality providing server, and combines a received second local information data with the acquired image information based on a received current location information.

32. The portable terminal of claim 29, wherein the terminal control unit extracts a feature point from the acquired image information, generates a second local information data attained from extracting a data matched with an extracted feature point by comparing the extracted feature point to the downloaded first menu information data, and combines a generated second local information data with the acquired image information.

33. The portable terminal of claim 32, wherein the terminal control unit extracts the feature point based on one or more kinds of information selected from a color information, edge information, outline information, texture information, text information, and contour information depending on a specified angle with respect to a specific object, which are contained in the acquired image information.

34. A method for providing an augmented reality information by an inter-working portable terminal, the method comprising:

receiving a selected local information from the portable terminal;

generating a first local information data attained by extracting data that matches a received selected local information by comparing the received selected local information with a location-associated information that have been previously stored;

transmitting a real picture image information of a locale corresponding to the selected local information among the location-associated information to the portable terminal; and

transmitting the first local information data with an incorporation of one or more kinds of information among a latitude-longitude coordinate information corresponding to the real picture image information of the locale corresponding to the selected local information, point of interest (POI) information, advertisement information, name information, phone number information, promotion information, internet link information, and tagging information, to the portable terminal.