Systems and method for navigating between oblique views of a map

• US 8,994,738 B1
• Filed: 03/21/2012
• Issued: 03/31/2015
• Est. Priority Date: 10/04/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Hernandez Esteban, Carlos, Seitz, Steven Maxwell, Simpson, Matthew Robert
• Primary Examiner(s): Tseng, Charles
• Application Number:
  US13/426,508
• Time in Patent Office: 1,105 Days
• Field of Search: 345/475
• US Class Current: 345/475
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images

Abstract

System and method for rendering a sequence of images corresponding to a sequence of camera poses of a target area to generate an animation representative of a progression of camera poses are provided. An initial image and an associated initial depthmap of a target area captured from an initial camera pose, and a final image and an associated final depthmap of the target area captured from a final camera pose are identified. A plurality of intermediate images representing a plurality of intermediate camera poses directed at the target are produced by performing interpolation on the initial image, the initial depthmap, the final image and the final depthmap. Each intermediate image is associated with a point along the navigational path between the initial and the final camera poses. An animation of the plurality of intermediate images produces a transition of views between the initial camera pose and the final camera pose.

Claims

1. A machine-implemented method of rendering a sequence of images corresponding to a sequence of camera poses of a target area to generate an animation representative of a progression of camera poses along a navigational path, the method comprising:

identifying, with one or more processors, an initial image and an associated initial depthmap of a target area captured from an initial camera pose;

identifying, with the one or more processors, a final image and an associated final depthmap of the target area captured from a final camera pose;

rendering, with the one or more processors, an initial mesh model based on the initial image and the initial depthmap; and

rendering, with the one or more processors, a final mesh model based on the final image and the final depthmap,

performing, with the one or more processors, interpolation on the initial image, the initial depthmap, the final image and the final depthmap in order to produce a plurality of intermediate images representing a plurality of intermediate camera poses directed at the target, each intermediate image being associated with a point along the navigational path between the initial camera pose and the final camera pose, wherein an animation of the plurality of intermediate images produces a transition of views between the initial camera pose and the final camera pose, and wherein producing the plurality of intermediate images comprises;

rendering a first intermediate image based on the initial mesh model;

rendering a second intermediate image based on the final mesh model; and

blending the first intermediate image and second intermediate image by assigning a first opacity to the first intermediate image and a second opacity to the second intermediate image, and combining the first intermediate image and the second intermediate image.

2. The machine-implemented method of claim 1, wherein depthmaps associated with the initial image and the final image comprise a plurality of pixels, and wherein each pixel of each depthmap comprises a depth value associated with the respective initial image or final image.

3. The machine-implemented method of claim 1, wherein the identified initial image and the identified final image are oblique view images representing views at a same angle to the surface plane of the target location, wherein the same angle is greater than 0°

and less than 90°

.

4. The machine-implemented method of claim 3, wherein each of the plurality of intermediate images corresponds to an intermediate camera pose of a specific heading angle, and wherein each of the plurality of intermediate images represents a view at a same angle to the surface plane as the initial image and the final image.

5. The machine-implemented method of claim 1, wherein the animation comprises providing for display each of the plurality of intermediate images in an order according to a progression of heading angles of the plurality of intermediate images.

6. The machine-implemented method of claim 5, wherein the progression of angles progresses from a heading angle associated with the initial image to a heading angle associated with the final image.

7. The machine-implemented method of claim 5, wherein the progression of angles progresses from a heading angle associated with the final image to a heading angle associated with the initial image.

8. The machine-implemented method of claim 1, further comprising applying a timing function to control the production of each of the plurality of intermediate images by determining a corresponding camera pose heading angle for each intermediate image to be produced.

9. A non-transitory machine-readable medium comprising instructions stored therein, which when executed by a system, cause the system to perform operations comprising:

receiving a first oblique view image corresponding to an initial camera pose of a target location;

receiving a first depthmap associated with the first oblique view, wherein the depthmap comprises a plurality of pixels, each pixel comprising a depth value associated with the first oblique view image;

receiving a second oblique view image corresponding to a final camera pose of the target location, the second oblique view image having a different heading from the first oblique view image;

receiving a second depthmap associated with the second oblique view, wherein the depthmap comprises a plurality of pixels, each pixel comprising a depth value associated with the second oblique view image; and

producing a plurality of intermediate images corresponding to a plurality of intermediate camera poses of the target area by;

rendering a first intermediate image based on the received first oblique view image and the first depth map;

rendering a second intermediate image based on the received second oblique view image and the second depth map; and

blending the first intermediate image and second intermediate image by assigning a first opacity to the first intermediate image and a second opacity to the second intermediate image, and combining the first intermediate image and the second intermediate image.

10. The non-transitory machine-readable medium of claim 9, wherein the received first oblique view image and the received second oblique view image represent a view at a 45°

angle to a surface plane of the target location.

11. The non-transitory machine-readable medium of claim 10, wherein the plurality of intermediate images represents a view at a 45°

angle to a surface plane of the target location.

12. The non-transitory machine-readable medium of claim 9, wherein the first and second opacities are determined based on the specific heading angle of the corresponding camera pose.

13. The non-transitory machine-readable medium of claim 9, wherein a change in the first opacity is negatively correlated to a change in the second opacity through a progression of the plurality of intermediate images.

14. The non-transitory machine-readable medium of claim 13, wherein an intermediate image with a heading angle closer to a heading angle of the first oblique view image will have a higher first opacity and a lower second opacity than a first opacity and a second opacity of an intermediate image with a heading angle closer to a heading angle of the second oblique view image.

15. The non-transitory machine-readable medium of claim 9, wherein an animation of the plurality of camera poses produces a transition of views between the initial camera pose and the final camera pose.

16. The non-transitory machine-readable medium of claim 9, further comprising applying a timing function to control the production of each of the plurality of intermediate images by determining a corresponding camera pose heading angle for each intermediate image to be produced.

17. A system for generating a sequence of images corresponding to a sequence of camera poses of a target area to generate an animation representative of a progression of camera poses along a navigational path, the system comprising:

one or more processors; and

a machine-readable medium comprising instructions stored therein, which when executed by the one or more processors, cause the one or more processors to perform operations comprising;

receiving an initial oblique view image and an associated initial depthmap of a target area corresponding to an initial camera pose;

receiving a final oblique view image and an associated final depthmap of the target area corresponding to a final camera pose, wherein the initial camera pose is different from the final camera pose; and

rendering an initial mesh model based on the initial image and the initial depthmap; and

rendering a final mesh model based on the final image and the final depthmap,

performing interpolation on the initial oblique view image, the initial depthmap, the final oblique view image, and the final depthmap in order to produce a plurality of intermediate images corresponding to a plurality of intermediate camera poses of the target area, each of the plurality of camera poses representing a heading angle along a navigational path between a heading angle associated with the initial camera pose and a heading angle associated with the final camera pose, wherein an animation of the plurality of intermediate images produces a transition of views between the initial camera pose and the final camera pose, and wherein producing each of the plurality of intermediate images comprises;

rendering a first intermediate image based on the initial mesh model;

rendering a second intermediate image based on the final mesh model; and

blending the first intermediate image and second intermediate image by assigning a first opacity to the first intermediate image and a second opacity to the second intermediate image, and combining the first intermediate image and the second intermediate image.

18. The system of claim 17, wherein each of the initial and final depthmaps comprises a plurality of pixels, and wherein each pixel of the depthmap comprises a depth value associated with the respective initial image or final image.