Terminal and method for providing augmented reality

• US 8,750,559 B2
• Filed: 09/01/2011
• Issued: 06/10/2014
• Est. Priority Date: 12/13/2010
• Status: Active Grant

• Current Assignee: MEC Resources, LLC (Three Affiliated Tribes of the Fort Berthold Reservation, North Dakota)
• Original Assignee: Pantech Co. Ltd. (Pantech Incorporated)
• Inventors: Sung, Young Ju, HA, Yong Gu
• Primary Examiner(s): Drennan, Barry
• Assistant Examiner(s): Harandi, Siamak
• Application Number:
  US13/224,053
• Publication Number:
  US 20120148106A1
• Time in Patent Office: 1,013 Days
• Field of Search: None
• US Class Current: 382/100
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 2370/16 Use of wireless transmission of display information

Abstract

A method for providing augmented reality includes acquiring a real-world image including an object; transmitting terminal information, in which the terminal information includes a location information of a terminal and an original retrieval distance; receiving object information corresponding to the object, in which the object information is based on the transmitted terminal information; and overlapping the received object information over the corresponding object in the real-world image. A terminal to perform the methods described herein includes a location information providing unit, an information transmitting/receiving unit, an image processing unit, and a user view analyzing unit.

Claims

1. A terminal to provide augmented reality, the terminal comprising:

a processor;

a location information providing unit to provide location information of a terminal at a physical location;

an information transmitting/receiving unit to transmit terminal information corresponding to the physical location, and to receive object information retrieved based on the terminal information corresponding to the physical location;

an image processing unit to overlap the object information on a real-world image, wherein the real-world image comprises an object corresponding to the object information; and

a user view analyzing unit to calculate, via the processor, a virtual movement location of the terminal based on a zooming level, to transmit terminal information corresponding to the virtual movement location, and to receive object information retrieved based on the terminal information corresponding to the virtual movement location.

2. The terminal of claim 1, wherein the terminal information comprises at least one of location information of the terminal, an original retrieval distance, and azimuth information of the terminal.

3. The terminal of claim 1, wherein the location information providing unit provides at least one of cellular network-based location information and GPS information-based location information.

4. The terminal of claim 1, further comprising a camera unit having a zooming operation to obtain the real-world image, wherein if the zooming level of the camera unit is changed, the user view analyzing unit resets the terminal information according to the virtual movement location, and a new retrieval distance is calculated based on the virtual movement location.

5. The terminal of claim 4, wherein the new retrieval distance is calculated by subtracting the virtual movement distance from an original retrieval distance and the terminal information is reset based on the virtual movement location and the new retrieval distance.

6. The terminal of claim 4, wherein the information transmitting/receiving unit extracts feature information of the object displayed on the real-world image corresponding to the changed zooming level, transmits the feature information along with the reset terminal information corresponding to the virtual movement location, and receives object information based on the reset terminal information and the feature information of the object, and

the image processing unit matches the received object information to a location of a corresponding object to be displayed.

7. The terminal of claim 1, wherein the virtual movement distance is calculated according to a reference value, wherein the reference value is obtained by dividing an original retrieval distance by the total number of zooming levels.

8. The terminal of claim 1, wherein the user view analyzing unit calculates a distance between the location of the terminal and the object based on the location information of the object and sets the calculated distance as a reference distance,

wherein the location of the terminal is an image capturing location or a virtual movement location, and the location information of the object is comprised in the received object information.

9. The terminal of claim 1, wherein the virtual movement distance is calculated according to a reference value, wherein the reference value is obtained by dividing an original retrieval distance by a number of the zoom levels.

10. The terminal of claim 1, wherein the real-world image is a real-time image.

11. The terminal of claim 8, wherein the user view analyzing unit calculates the virtual movement location of the terminal based on the reference distance and the zooming level,

the information transmitting/receiving unit transmits the virtual movement location of the terminal and feature information of objects extracted from the real-world image, and receives object information based on the feature information of objects, and

the image processing unit matches the received information to a location of a corresponding object to be displayed.

12. A terminal to provide augmented reality, the terminal comprising:

a processor;

a location information providing unit to provide location information of the terminal at a physical location;

an information transmitting/receiving unit to transmit the location information of the terminal and feature information of an object displayed on a real-world image, and to request and receive object information based on the location information of the terminal and the feature information of the object;

an object recognition determining unit to obtain a distance between the terminal and the object based on a location information of the object and to set the obtained distance as a reference distance, wherein the location information of the object is comprised in the received object information;

an image processing unit to overlap the object information received on the real-world image; and

a user view analyzing unit to calculate, via the processor, a virtual movement location of the terminal based on the reference distance and a zooming level, to transmit the virtual movement location of the terminal and the feature information of the object, and to receive object information corresponding to the virtual movement location of the terminal.

13. The terminal of claim 12, wherein, if the number of objects displayed on the real-world image and corresponding received object information is more than one, the object recognition determining unit obtains an average distance between the terminal and the objects based on the location information of the objects comprised in the received object information, and sets the average distance as the reference distance.

14. The terminal of claim 12, wherein, if the object information requested is not available according to the result of the object recognition determining unit, the user view analyzing unit does not change the location information of the terminal even if the zooming level of the camera is changed.

15. The terminal of claim 12, wherein the real-world image is a real-time image.

16. A method for providing augmented reality, the method comprising:

acquiring a real-world image comprising an object;

transmitting terminal information, wherein the terminal information comprises a location information of a terminal and an original retrieval distance;

receiving object information corresponding to the object, wherein the object information is based on the transmitted terminal information;

overlapping the received object information over the corresponding object in the real-world image;

changing a zooming level of the real-world image for providing a second real-world image, wherein the second real-world image comprises a second object;

calculating a virtual movement location of the terminal, a virtual movement distance between the second object and the terminal, and a second retrieval distance according to the changed zooming level; and

resetting the terminal information according to the virtual movement location for providing a second terminal information.

17. The method of claim 16, further comprising:

transmitting the second terminal information;

receiving second object information based on the second terminal information; and

overlapping the second object information over the second object in the second real-world image.

18. The method of claim 16, further comprising:

changing a zooming level of the real-world image for providing a second real-world image, wherein the second real-world image comprises a second object;

extracting feature information of the second object displayed on the second real-world image;

calculating a virtual movement location of the terminal;

resetting the terminal information according to the virtual movement location for providing a second terminal information;

transmitting the second terminal information and the feature information;

receiving second object information based on the second terminal information and the feature information;

overlapping the second object information over the second object in the second real-world image; and

displaying the overlapped image such that the second object information is matched with a location of the corresponding second object.

19. The method of claim 16, wherein the real-world image is a real-time image.

20. A method for providing augmented reality, the method comprising:

acquiring a real-world image comprising an object using a terminal;

transmitting location information of the terminal and feature information of the object;

receiving object information corresponding to the object based on the location information of the terminal and the feature information of the object;

calculating a distance between the terminal and the object based on location information of the object comprised in the received object information;

setting the calculated distance as a reference distance;

changing a zooming level of the real-world image for providing a second real-world image, wherein the second real-world image comprises a second object; and

calculating a virtual movement location of the terminal based on the reference distance and the changed zooming level.

21. The method of claim 20, further comprising:

extracting feature information of the second object from the second real-world image;

transmitting the virtual movement location of the terminal and the feature information of the second object;

requesting and receiving second object information based on the virtual movement location of the terminal and the feature information of the second object; and

overlapping the second object information over the second object in the second real-world image, wherein the second object information overlaps the second object at a location of the corresponding second object.

22. The method of claim 20, further comprising:

changing a zooming level of the real-world image for providing a second real-world image, wherein the second real-world image comprises a second object;

calculating a virtual movement location of the terminal based on the reference distance and the changed zooming level;

extracting feature information of the second object displayed on the second real world image;

transmitting the virtual movement location of the terminal and feature information of objects;

requesting second object information based on the virtual movement location of the terminal and the feature information of the second object; and

if the requested second object information is not available, receiving object information based on the location information of the terminal and the feature information of the second object.

23. The method of claim 20, wherein the real-world image is a real-time image.

24. A method for providing augmented reality, the method comprising:

transmitting terminal information, wherein the terminal information comprises a location information of the terminal and an original retrieval distance;

receiving object information corresponding to an object, wherein the object information is based on the transmitted terminal information;

overlapping the received object information over the corresponding object in the real-world image;

calculating a distance between the terminal and the object based on location information of the object comprised in the received object information;

setting the calculated distance as a reference distance;

changing a zooming level of the real-world image for providing a second real-world image, wherein the second real-world image comprises a second object;

calculating a virtual movement distance and a virtual movement location of the terminal based on the reference distance;

setting a second retrieval distance, wherein the second retrieval distance is calculated by taking a difference in value between the virtual movement distance and the original retrieval distance; and

resetting the terminal information based on the virtual movement location and the second retrieval distance.

25. The method of claim 24, further comprising:

transmitting the reset terminal information;

requesting and receiving second object information based on the reset terminal information; and

overlapping the second object information over the second object in the second real-world image.

26. The method according to claim 24, further comprising:

changing a zooming level of the real-world image for providing a second real-world image, wherein the second real-world image comprises a second object;

calculating the virtual movement location of the terminal based on the reference distance and the changed zooming level;

extracting feature information of the second object from the second real-world image;

transmitting the virtual movement location of the terminal and the feature information of the second object;

receiving second object information based on the virtual movement location of the terminal and the feature information of the second object; and

overlapping the second object information over the second object in the second real-world image, wherein the second object information overlaps the second object at a location of the corresponding second object.

27. The method of claim 24, wherein the real-world image is a real-time image.

28. A terminal to provide augmented reality, the terminal comprising:

a processor, comprising;

a location information providing unit to provide location information of a terminal at a physical location,

an information transmitting/receiving unit to transmit terminal information corresponding to the physical location, and to receive object information retrieved based on the terminal information corresponding to the physical location, and

an image processing unit to overlap the object information on a real-world image, wherein the real-world image comprises an object corresponding to the object information; and

a user view analyzing unit to calculate, via the processor, a virtual movement location of the terminal based on a zooming level, and to transmit terminal information corresponding the virtual movement location, and to receive object information retrieved based on the terminal information corresponding to the virtual movement location.