Photorealistic Rendering of Scenes with Dynamic Content

• US 20150097834A1
• Filed: 10/08/2013
• Published: 04/09/2015
• Est. Priority Date: 10/08/2013
• Status: Active Grant

• Current Assignee: HERE Global B.V. (Nokia Corporation)
• Original Assignee: HERE Global B.V. (Nokia Corporation)
• Inventors: Chen, Xin, Ma, Xiang, Dou, Hang
• Granted Patent: US 9,892,550 B2
• Time in Patent Office: Days
• US Class Current: 345/426
• CPC Class Codes: G06T 15/60 Shadow generation
• CPC Class Codes: G06T 17/05 Geographic models

Abstract

Methods for rendering three-dimensional photo meshes having dynamic content include: (a) detecting a shadow in a three-dimensional photo mesh; (b) removing the shadow from the three-dimensional photo mesh to form a modified photo mesh having a shadow-free texture; (c) simulating a real-time condition in the modified photo mesh; and (d) rendering an image that shows an effect of the real-time condition. Systems for rendering three-dimensional photo meshes having dynamic content are described.

Claims

1. A method comprising:

detecting, by a processor, a shadow in a three-dimensional photo mesh;

removing, by the processor, the shadow from the three-dimensional photo mesh to form a modified photo mesh having a shadow-free texture;

simulating, by the processor, a real-time condition in the modified photo mesh; and

rendering, by the processor, an image that shows an effect of the real-time condition.

2. The method of claim 1 wherein the three-dimensional photo mesh comprises a tiled texture.

3. The method of claim 1 wherein the three-dimensional photo mesh comprises a non-tiled texture.

4. The method of claim 1 wherein the three-dimensional photo mesh depicts a city model.

5. The method of claim 1 wherein a texture of the three-dimensional photo mesh comprises a plurality of polygons.

6. The method of claim 5 wherein the detecting of the shadow comprises:

determining a position of a light source;

creating a directional shadow map based on the determined position;

testing light visibility of a polygon of the texture based on the directional shadow map;

mapping a vertex of the polygon to unique coordinates of the texture;

generating a shadow mask representing a shadowed area in the texture; and

outputting the shadow mask as an image.

7. The method of claim 6 wherein the determining of the position of the light source is based on a position of the shadow in the three-dimensional photo mesh.

8. The method of claim 6 wherein the determining of the position of the light source is based on knowledge of a time stamp of the texture of the three-dimensional photo mesh.

9. The method of claim 6 wherein the image of the shadow mask comprises a plurality of colors, and wherein each of the colors independently represents information selected from the group consisting of presence of a shadow, non-presence of a shadow, type of shadow, and combinations thereof.

10. The method of claim 1 wherein removing of the shadow comprises:

separating a shadow region from a non-shadow region in a texture of the three-dimensional photo mesh;

compensating for illumination in the shadow region based on a neighboring non-shadow pixel;

compensating for color and/or texture in the shadow region based on a neighboring non-shadow pixel; and

smoothing a boundary between the shadow region and the non-shadow region.

11. The method of claim 1 wherein the simulating of the real-time condition comprises:

relighting a scene to generate a shadow map for the modified photo mesh.

12. The method of claim 1 wherein the real-time condition is variable based on a parameter selected from the group consisting of time of day, day of year, season of year, weather conditions, and combinations thereof.

13. The method of claim 1 wherein the real-time condition comprises lighting, weather, traffic, or combinations thereof.

14. An apparatus comprising:

at least one processor; and

at least one memory including computer program code for one or more programs, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following;

detect a shadow in a three-dimensional photo mesh;

remove the shadow from the three-dimensional photo mesh to form a modified photo mesh having a shadow-free texture;

simulate a real-time condition in the modified photo mesh; and

render an image that shows an effect of the real-time condition.

15. The apparatus of claim 14 wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus to perform at least the following:

determine a position of a light source;

create a directional shadow map based on the determined position;

test light visibility of a polygon of a texture of the three-dimensional photo mesh based on the directional shadow map;

map a vertex of the polygon to unique coordinates of the texture;

generate a shadow mask representing a shadowed area in the texture; and

output the shadow mask as an image.

16. The apparatus of claim 14 wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus to perform at least the following:

separate a shadow region from a non-shadow region in a texture of the three-dimensional photo mesh;

compensate for illumination in the shadow region based on a neighboring non-shadow pixel;

compensate for color and/or texture in the shadow region based on a neighboring non-shadow pixel; and

smooth a boundary between the shadow region and the non-shadow region.

17. The apparatus of claim 14 wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus to perform at least the following:

relight a scene to generate a shadow map for the modified photo mesh.

18. The apparatus of claim 14 wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus to perform at least the following:

determine a position of a light source;

create a directional shadow map based on the determined position;

test light visibility of a polygon of a texture of the three-dimensional photo mesh based on the directional shadow map;

map a vertex of the polygon to unique coordinates of the texture;

generate a shadow mask representing a shadowed area in the texture;

output the shadow mask as an image;

separate a shadow region from a non-shadow region in the texture;

compensate for illumination in the shadow region based on a neighboring non-shadow pixel;

compensate for color and/or texture in the shadow region based on a neighboring non-shadow pixel;

smooth a boundary between the shadow region and the non-shadow region; and

relight a scene to generate a shadow map for the modified photo mesh

19. An apparatus comprising:

means for detecting a shadow in a three-dimensional photo mesh;

means for removing the shadow from the three-dimensional photo mesh to form a modified photo mesh having a shadow-free texture;

means for simulating a real-time condition in the modified photo mesh; and

means for rendering an image that shows an effect of the real-time condition.

20. In a non-transitory computer-readable storage medium having stored therein data representing instructions executable by a programmed processor, the storage medium comprising instructions for:

detecting a shadow in a three-dimensional photo mesh;

removing the shadow from the three-dimensional photo mesh to form a modified photo mesh having a shadow-free texture;

simulating a real-time condition in the modified photo mesh; and

rendering an image that shows an effect of the real-time condition.