APPARATUS TO PROVIDE AUGMENTED REALITY SERVICE USING LOCATION-BASED INFORMATION AND COMPUTER-READABLE MEDIUM AND METHOD OF THE SAME

• US 20110165893A1
• Filed: 12/23/2010
• Published: 07/07/2011
• Est. Priority Date: 01/04/2010
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: HYUNG, Seung Yong, Ahn, Sung Hwan, Roh, Kyung Shik, Yoon, Suk June
• Granted Patent: US 8,644,859 B2
• Time in Patent Office: Days
• US Class Current: 455/457
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: H04W 4/029 Location-based management or tracking services

Abstract

An augmented reality (AR) service apparatus includes a camera to capture an actual image, a controller to receive feature point information about the captured image from at least one of a plurality of base stations (BSs), detect a location of the camera by matching data of feature points with data of the image, and provide location-based information in a same direction as the captured image according to the location of the camera, and a display to realize an AR service by combining the captured image with the location-based information under control of the controller.

Claims

1. An apparatus to provide an Augmented Reality (AR) service using location-based information in a user terminal communicating with a plurality of Base Stations (BSs), the apparatus comprising:

a camera to capture an actual image;

a controller to receive feature point information about the captured image from at least one of the plurality of BSs, detect a location of the camera by matching data of feature points to data of the image, and provide location-based information in a same direction as the captured image according to the location of the camera; and

a display to provide an AR service by combining the captured image with the location-based information under control of the controller.

2. The apparatus according to claim 1, wherein the plurality of BSs segment an area where the user terminal is located into cells and manage feature point information and location-based information about the cells.

3. The apparatus according to claim 1, further comprising a transceiver to communicate with the plurality of BSs, wherein the controller searches for a connectable BS among the plurality of BSs through the transceiver, receives location information about all found BSs, and detects a location of the user terminal based on the location information.

4. The apparatus according to claim 3, wherein the controller requests transmission of image patch data corresponding to the feature points from a BS according to the location of the user terminal.

5. The apparatus according to claim 4, wherein the controller receives the image patch data corresponding to the feature points from the BS and matches the image patch data with the captured image in a one-to-one correspondence.

6. The apparatus according to claim 5, wherein when the feature points are matched, the controller acquires information about a current location and position of the camera by tracking the feature points.

7. A method of providing an Augmented Reality (AR) service using location-based information in a user terminal communicating with a plurality of Base Stations (BSs), the method comprising:

capturing an actual image by a camera;

detecting, by a processor, a location of the user terminal by searching for connectable BSs among the plurality of BSs;

requesting, by the processor, feature point data of the captured image to a BS according to the location of the user terminal;

receiving, by the processor, the feature point data from the BS and detecting a location of the camera by matching the feature point data with data of the captured image; and

providing, by the processor, an AR service by combining the captured image with location-based information in a same direction according to the location of the camera.

8. The method according to claim 7, wherein the plurality of BSs segment an area where the user terminal is located into cells and manage feature point information and location-based information about the cells.

9. The method according to claim 8, wherein the detection of a location of the user terminal comprises requesting location information to all found BSs, and upon receipt of the location information, detecting the location of the user terminal based on the location information and times of arrival of the location information.

10. The method according to claim 9, wherein the requesting of feature point data of the captured image comprises identifying a cell where the user terminal is located and requesting the feature point data of the captured image from a BS covering the cell.

11. The method according to claim 10, wherein the detection of a location of the camera comprises acquiring information about a current location and posture of the camera by receiving image patch data corresponding to feature points from the BS and matching the image patch data to the captured image in a one-to-one correspondence.

12. The method according to claim 7, wherein the providing of an AR service comprises providing a three-dimensional AR service by combining the image captured by the camera with location-based information received from the BS.

13. At least one non-transitory computer readable medium comprising computer readable instructions that control at least one processor to implement a method, comprising:

capturing an actual image by a camera;

detecting a location of a user terminal by searching for connectable base stations (BSs) among a plurality of BSs;

requesting feature point data of the captured image from a BS according to the location of the user terminal;

receiving the feature point data from the BS and detecting a location of the camera by matching the feature point data with data of the captured image; and

providing an augmented reality (AR) service by combining the captured image with location-based information in a same direction according to the location of the camera.

14. The at least non-transitory one computer readable medium comprising computer readable instructions that control at least one processor to implement the method of claim 13, wherein

the plurality of BSs segment an area where the user terminal is located into cells and manage feature point information and location-based information about the cells.

15. The at least one non-transitory computer readable medium comprising computer readable instructions that control at least one processor to implement the method of claim 14, wherein the detection of a location of the user terminal comprises requesting location information to all found BSs, and upon receipt of the location information, detecting the location of the user terminal based on the location information and times of arrival of the location information.

16. The at least one non-transitory computer readable medium comprising computer readable instructions that control at least one processor to implement the method of claim 15, wherein the requesting of feature point data of the captured image comprises identifying a cell where the user terminal is located and requesting the feature point data of the captured image from a BS covering the cell.

17. The at least one non-transitory computer readable medium comprising computer readable instructions that control at least one processor to implement the method of claim 16, wherein the detection of a location of the camera comprises acquiring information about a current location and posture of the camera by receiving image patch data corresponding to feature points from the BS and matching the image patch data to the captured image in a one-to-one correspondence.

18. The at least one non-transitory computer readable medium comprising computer readable instructions that control at least one processor to implement the method of claim 13, wherein the providing of an AR service comprises providing a three-dimensional AR service by combining the image captured by the camera with location-based information received from the BS.