System and method for overlaying two-dimensional map data on a three-dimensional scene

• US 9,183,666 B2
• Filed: 03/15/2013
• Issued: 11/10/2015
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Kontkanen, Janne
• Primary Examiner(s): Caschera, Antonio A
• Application Number:
  US13/834,474
• Publication Number:
  US 20140267279A1
• Time in Patent Office: 970 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics

Abstract

A method for augmenting a three-dimensional (3D) scene with two-dimensional (2D) map data includes receiving 3D map data having a 3D geometry data and imagery texture data. For a specified perspective of a virtual camera in a 3D coordinate system, geometry of the 3D scene is rendered using the 3D geometry data, wherein the specified perspective includes a position and orientation relative to the 3D scene. A map texture is generated using the 2D map data in accordance with the perspective of the virtual camera in the 3D coordinate system, and the imagery texture and map texture are applied to the rendered geometry.

Claims

1. A method in a computing device for augmenting three-dimensional (3D) scenes with two-dimensional (2D) map data, the method comprising:

receiving 3D map data for generating a textured 3D scene, the 3D map data including 3D geometry data and imagery texture data;

for a specified perspective of a virtual camera in a 3D coordinate system, rendering geometry of the 3D scene using the 3D geometry data, wherein the specified perspective includes a position and orientation relative to the 3D scene;

generating a map texture using 2D map data in accordance with the perspective of the virtual camera in the 3D coordinate system; and

applying by one or more processors (i) the imagery texture and (ii) the map texture to the rendered geometry corresponding to a same geographic area to overlay the 2D map data on the 3D scene.

2. The method of claim 1, wherein the 2D map data is parameterized in a coordinate system associated with projecting a spherical surface onto a flat surface.

3. The method of claim 2, wherein the coordinate system in which the 2D map data is parameterized is one of (i) Mercator or (ii) plate carré

e.

4. The method of claim 2, wherein the 3D coordinate system is Cartesian.

5. The method of claim 2, wherein generating the map texture includes determining a perspective of the virtual camera relative to a plane with which the 2D map data is associated, including calculating:

(i) a target point within the 2D map data,

(ii) a target distance between the virtual camera and the target point,

(iii) tilt of the virtual camera, and

(iv) rotation of the virtual camera.

6. The method of claim 5, wherein:

the 3D geometry data includes (i) terrain layer geometry data (ii) buildings and vegetation layer geometry data, and

calculating the target point includes determining the intersection between a direction of flight (DOF) vector and terrain layer.

7. The method of claim 5, wherein calculating the target distance includes transforming the position of the virtual camera in the 3D coordinate system to the system in which the 2D map data is parameterized.

8. The method of claim 5, wherein generating the map texture includes applying a curvature factor to the target distance to account for the curvature of the Earth.

9. The method of claim 1, wherein:

the 3D geometry data includes (i) a first layer geometry data (ii) a second layer geometry data;

rendering the geometry of the 3D scene includes;

(i) in a first instance, rendering a first layer using the first layer geometry data and rendering the second layer using the second layer geometry data, and

(ii) in a second instance, re-rendering the first layer using the first layer geometry data; and

wherein applying the imagery texture and the map texture includes;

(i) applying the imagery texture to the rendered first layer and the second layer before the first layer is re-rendered, and

(ii) applying the map texture to re-rendered first layer.

10. The method of claim 9, wherein applying the map texture to the re-rendered first layer includes applying a depth offset to raise the map texture above the bottom of the second layer.

11. The method of claim 9, wherein the first layer is a terrain layer and the second layer is a buildings and vegetation layer.

12. The method of claim 1, wherein:

the 3D geometry data includes (i) first layer geometry data (ii) second layer geometry data; and

rendering the geometry of the 3D scene includes;

(i) in a first instance, rendering a first layer using the first layer geometry data and not rendering a second layer, and

(ii) in a second instance, rendering the second layer using the second layer geometry data;

applying the imagery texture and the map texture includes;

(i) applying the imagery texture and the map texture to the first layer before the second layer is rendered, and

(ii) applying the imagery texture to the second layer.

13. The method of claim 12, wherein the first layer is a terrain layer and the second layer is a buildings and vegetation layer.

14. The method of claim 1, wherein generating the map texture includes:

rendering the 2D map data to a frame buffer of a graphics pipeline, and

copying the frame buffer into a texture.

15. The method of claim 1, wherein the 2D map data includes vector-based 2D geometry data and map texture data.

16. The method of claim 1, wherein the imagery texture is photographic texture.

17. The method of claim 1, wherein generating the map texture includes:

rendering map elements included in the 2D map data into a texture, and

rendering areas free of map elements into transparent portions of the texture.

18. The method of claim 1, wherein generating the map texture includes:

rendering map elements of a first type included in the 2D map data into the texture, and

rendering map elements of a second type included in the 2D map data into transparent portions of the texture.

19. The method of claim 18, wherein the first type includes at least one of:

(i) roads,

(ii) labels, and

(iii) point of interest (POI) indicators.

20. A computing device comprising:

a user interface including a display device;

one or more processors;

a tangible computer-readable medium storing thereon instructions that, when executed on the one or more processors, cause the computing device to;

receive 2D map data that specifies elements of a digital 2D map, wherein the 2D map data is parameterized in a coordinate system associated with projecting a spherical surface onto a flat surface;

receive 3D map data for generating a textured 3D scene, the 3D map data including 3D geometry data and imagery texture data;

receive an indication of a perspective of a virtual camera relative to the 3D scene in a 3D coordinate system;

using the 2D map data, generate a map texture for application to the 3D scene in accordance with the perspective of the virtual camera; and

apply the map texture and the imagery texture data to the 3D scene, at a location corresponding to a same geographic area, to overlay the 2D map data on the 3D scene.

21. The computing device of claim 20, wherein to generate the map texture, the instructions cause the computing device to calculate:

(i) a target point within the 2D map data,

(ii) a target distance between the virtual camera and the target point,

(iii) tilt of the virtual camera, and

(iv) rotation of the virtual camera.

22. The computing device of claim 21, wherein:

the 3D geometry data includes (i) terrain layer geometry data (ii) buildings and vegetation layer geometry data, and

to calculate the target point, the instructions cause the computing device to determine the intersection between a direction of flight (DOF) vector and terrain layer.

23. The computing device of claim 21, wherein:

to generate the map texture, the instructions cause the computing device to apply a curvature factor to the target distance to account for the curvature of the Earth.

24. The computing device of claim 20, further comprising:

a graphics processor;

wherein the instructions include a shader program executable on the graphics processor to blend the map texture over the 3D scene.

25. The computing device of claim 20, wherein to generating the map texture, the instructions cause the computing device to:

render map elements of a first type included in the 2D map data into the texture, and

render map elements of a second type included in the 2D map data into transparent portions of the texture.

26. The computing device of claim 20, wherein the coordinate system in which the 2D map data is parameterized is one of (i) Mercator or (ii) plate carré

e.

27. A tangible non-transitory computer-readable medium storing thereon instructions that, when executed by one or more processors, cause the one or more processors to:

receive 3D map data that includes 3D geometry data and imagery texture data;

for a specified perspective of a virtual camera in a 3D coordinate system, render geometry of the 3D scene using the 3D geometry data, wherein the specified perspective includes a position and orientation relative to the 3D scene;

cause a map texture to be generated using the 2D map data in accordance with the perspective of the virtual camera in the 3D coordinate system; and

cause (i) the imagery texture and (ii) the map texture to be applied to the rendered geometry corresponding to a same geographic area to overlay the 2D map data on the 3D scene.