Apparatus and method for providing augmented reality using synthesized environment map

• US 8,654,151 B2
• Filed: 08/02/2011
• Issued: 02/18/2014
• Est. Priority Date: 12/17/2010
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Pantech Corporation (Ideahub, Inc.)
• Inventors: KIM, Chung-Hyun
• Primary Examiner(s): AMIN, JWALANT B
• Application Number:
  US13/196,753
• Publication Number:
  US 20120154425A1
• Time in Patent Office: 931 Days
• Field of Search: None
• US Class Current: 345/633
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 2320/0261 in the context of movement of objects on the screen or movement of the observer relative to the screen
• CPC Class Codes: G09G 2340/10 Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 5/14 Display of multiple viewports

Abstract

An apparatus and method for providing Augmented Reality (AR) corresponding to objects that are hidden from view of a user. The apparatus includes a first extraction unit to extract a first environment map corresponding to a location of a terminal; a second extraction unit to determine an expanded area enclosing the location of the terminal and to extract a second environment map corresponding to the expanded area; and a synthesization unit to generate a synthesized environment map by synthesizing the first environment map and the second environment map.

Claims

1. An apparatus to provide Augmented Reality (AR), comprising:

a first extraction unit to extract a first environment map corresponding to a location of a terminal;

a second extraction unit to determine an expanded area enclosing the location of the terminal and to extract a second environment map included in the expanded area; and

a synthesization unit to generate a synthesized environment map by synthesizing the first environment map and the second environment map,

wherein the second environment map is smaller than the expanded area.

2. The apparatus of claim 1, further comprising:

an environment map database to store a plurality of environment maps each having a center and environment data corresponding to the center, the first environment map and the second environment map extracted from among the plurality of environment maps.

3. The apparatus of claim 2, wherein a center of the first environment map corresponds to the location of the terminal.

4. The apparatus of claim 2, wherein the expanded area has a center that corresponds to the location of the terminal, and the second environment map is located within the expanded area.

5. The apparatus of claim 2, wherein the synthesization unit is further configured to detect a synthesization target area from the first environment map and the second environment map based on the location of the terminal, a location of a center of the second environment map, and a viewing angle with respect to the second environment map from the location of the terminal.

6. The apparatus of claim 5, wherein the synthesization unit is further configured to adjust a size and/or a transparency of the synthesization target areas of the first environment map and the second environment map based on a distance between the location of the terminal and the center of the second environment map.

7. The apparatus of claim 6, wherein the synthesization unit is further configured to project the synthesization target area of the second environment map onto the synthesization target area of the first environment map.

8. The apparatus of claim 1, further comprising:

a location detection unit to detect the location of the terminal based on at least one of location information received from the terminal and image data captured by the terminal.

9. The apparatus of claim 1, further comprising:

an environment map transmission unit to transmit the synthesized environment map to the terminal.

10. The apparatus of claim 1, wherein the first environment map and the second environment map include at least one of a cylinder map, a planar map, a spherical map, and a cubic map.

11. The apparatus of claim 1, wherein the second extraction unit extracts a plurality of second environment maps corresponding to the expanded area, and selects at least one of the plurality of second environment maps to generate the synthesized environment map.

12. The apparatus of claim 1, wherein a center of the first environment map does not coincide with a center of the second environment map.

13. A method for providing Augmented Reality (AR), comprising:

extracting a first environment map corresponding to a location of a terminal;

determining an expanded area enclosing the location of the terminal;

extracting a second environment map included in the expanded area; and

generating a synthesized environment map by synthesizing the first environment map and the second environment map,

wherein the second environment map is smaller than the expanded area.

14. An apparatus to provide Augmented Reality (AR), comprising:

an environment map database to store a first environment map and a second environment map, the first environment map and the second environment map each having a center and environment data corresponding to the center;

an extraction unit to extract the first environment map corresponding to a location of a terminal, to determine an expanded area enclosing the location of the terminal, and to extract the second environment map included in the expanded area; and

a synthesization unit to generate a synthesized environment map by synthesizing the first environment map and the second environment map,

wherein the second environment map is smaller than the expanded area.

15. The apparatus of claim 14, wherein a center of the first environment map corresponds to the location of the terminal.

16. The apparatus of claim 14, wherein the expanded area has a center that corresponds to the location of the terminal, and a center of the second environment map is located within the expanded area.

17. The apparatus of claim 14, wherein the synthesization unit is further configured to detect a synthesization target area from the first environment map and the second environment map based on the location of the terminal, a location of a center of the second environment map, and a viewing angle with respect to the second environment map from the location of the terminal.

18. The apparatus of claim 17, wherein the synthesization unit is further configured to adjust a size and/or a transparency of the synthesization target areas of the first environment map and the second environment map based on a distance between the location of the terminal and the center of the second environment map.

19. The apparatus of claim 18, wherein the synthesization unit is further configured to project the synthesization target area of the second environment map onto the synthesization target area of the first environment map.

20. The apparatus of claim 14, further comprising:

a location detection unit to detect the location of the terminal based on at least one of location information received from the terminal and image data captured by the terminal.

21. The apparatus of claim 14, further comprising:

an environment map transmission unit to transmit the synthesized environment map to the terminal.

22. The apparatus of claim 14, wherein the first environment map and the second environment map include at least one of a cylinder map, a planar map, a spherical map, and a cubic map.

23. The apparatus of claim 14, wherein the extraction unit extracts a plurality of second environment maps corresponding to the expanded area from the environment map database, and selects at least one of the plurality of second environment maps to generate the synthesized environment map.