Blending between street view and earth view

• US 9,898,857 B2
• Filed: 07/07/2016
• Issued: 02/20/2018
• Est. Priority Date: 07/17/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Dillard, Scott Edward, Parker, Evan Hardesty, Jones, Michael Cameron
• Primary Examiner(s): BROOME, SAID A
• Application Number:
  US15/204,448
• Publication Number:
  US 20160321837A1
• Time in Patent Office: 593 Days
• Field of Search: None
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 15/503 Blending, e.g. for anti-aliasing
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 7/90 Determination of colour characteristics

Abstract

In one aspect, computing device(s) may determine a plurality of fragments for a three-dimensional (3D) model of a geographical location. Each fragment of the plurality of fragments may correspond to a pixel of a blended image and each fragment has a fragment color from the 3D model. The one or more computing devices may determine geospatial location data for each fragment based at least in part on latitude information, longitude information, and altitude information associated with the 3D model. For each fragment of the plurality of fragments, the one or more computing devices may identify a pixel color and an image based at least in part on the geospatial location data, determine a blending ratio based on at least one of a position and an orientation of a virtual camera, and generate the blended image based on at least the blending ratio, the pixel color, and the fragment color.

Claims

1. A method for displaying images in order to provide an appearance of jumping between the images, the method comprising:

receiving, by one or more processors of a user input device, a request to transition from a first street-level image to a second street-level image via an aerial view of a transitional image;

determining a transition path of a virtual camera between the first street-level image and the second street-level image such that the transition path corresponds to the virtual camera ascending from the first street-level image at an initial point in the transition path and thereafter descending down and into the second street-level image wherein the transitional image corresponds to a location of the virtual camera along the transition path between a location of the first street-level image and a location of the second street-level image, such that when the virtual camera is at the initial point in the transition path, the virtual camera is oriented towards the first street-level image and tilted upward with respect to ground;

generating the transitional image corresponding to the aerial view by;

rasterizing fragments of a three dimensional (3D) model onto pixels of a display of the user input device,

identifying pixels of the first street-level image that correspond to locations of the rasterized fragments by extending lines-of-sight from the virtual camera at the initial point through the identified pixels and to each of the rasterized fragments,

identifying a blending ratio for blending fragments of the 3D model with the identified pixels,

wherein the blending ratio defines a color contribution from each of the identified pixels and the rasterized fragments, and

wherein the color contribution from the identified pixels being greater than the color contribution from the rasterized fragments when the virtual camera is aligned with the street-level images and the color contribution from the rasterized fragments increases as the virtual camera becomes less coincident with the street-level images, and

using the blending ratio to generate the transitional image; and

displaying each of the first street-level image, the transitional image, and the second street-level image on the display as a sequence in order to provide

     1) the appearance of jumping between the first street-level image and the second street-level image and

     2) spatial context between the first street-level image and the second street-level image.

2. The method of claim 1, further comprising:

generating a set of transitional images including the transitional image, wherein each given transitional image of the set is associated with a location of the virtual camera along the transition path and each given transitional image of the set is generated based on a blending ratio corresponding to the location of the virtual camera for the given transitional image of the set; and

displaying the set of transitional images in sequence according to the locations associated with each given transitional image of the set along the transition path with the transitional image in order to provide the appearance of jumping between the first street-level image and the second street-level image.

3. The method of claim 2, wherein for each given transitional image of the set the blending ratio further corresponds to an orientation of the virtual camera at the location associated with the given transitional image of the set.

4. The method of claim 1, further comprising:

generating a second transitional image using a second blending ratio different from the blending ratio; and

displaying the second transitional images in sequence with the transitional image according to associated locations of the second transitional images and the transitional image along the transition path in order to provide the appearance of jumping between the first street-level image and the second street-level image.

5. The method of claim 1, wherein identifying the identified pixels is further based on a distance along the transition path between the first street-level image and the location.

6. The method of claim 1, wherein identifying the identified pixels is further based on which of the first street-level image and second street-level image are closest along the transition path to the location.

7. The method of claim 1, wherein determining the transition path includes determining an arc shape of the transition path according to a distance between a location of the first street-level image and a location of the second street-level image.

8. The method of claim 1, wherein using the blending ratio includes determining a color contribution for each pixel of the display.

17. The method of claim 1, wherein the transitional image is located at an apex of the transition path.

18. The method of claim 17, wherein the transitional image is an aerial view of the 3D model.

19. The method of claim 1, wherein the virtual camera ascends up and away from the first street-level image and into the transitional image.

20. The method of claim 1, wherein the virtual camera descends from the transitional image and down and into the second street-level image.

21. The method of claim 1, wherein when the color contribution from the identified pixels is greater than the color contribution from the rasterized fragments the virtual camera is aligned perpendicular with the street-level images.

9. A system for displaying images in order to provide an appearance of jumping between the images, the system comprising:

a display; and

one or more processors configured to;

receive a request to transition from a first street-level image to a second street-level image via an aerial view of a transitional image;

determine a transition path of a virtual camera between the first street-level image and the second street-level image such that the transition path corresponds to the virtual camera ascending from the first street-level image at an initial point in the transition path and thereafter descending down and into the second street-level image wherein the transitional image corresponds to a location of the virtual camera along the transition path between a location of the first street-level image and a location of the second street-level image, such that when the virtual camera is at the initial point in the transition path, the virtual camera is oriented towards the first street-level image and tilted upward with respect to ground and;

generate the transitional image corresponding to the aerial view by;

rasterize fragments of a three dimensional (3D) model onto pixels of the display,

identify pixels of the first street-level image that correspond to locations of the rasterized fragments by extending lines-of-sight from the virtual camera at the initial point through the identified pixels and to each of the rasterized fragments,

determine a blending ratio for blending fragments of the 3D model with the identified pixels,

wherein the blending ratio defines a color contribution from each of the identified pixels and the rasterized fragments, and

wherein the color contribution from the identified pixels being greater than the color contribution from the rasterized fragments when the virtual camera is aligned with the street-level images and the color contribution from the rasterized fragments increases as the virtual camera becomes less coincident with the street-level images, and

using the blending ratio to generate the transitional image; and

display each of the first street-level image, the transitional image, and the second street-level image on the display as a sequence in order to provide

     1) the appearance of jumping between the first street-level image and the second street-level image and

     2) spatial context between the first street-level image and the second street-level image.

10. The system of claim 9, wherein the one or more processors are further configured to:

generate a set of transitional images including the transitional image, wherein each given transitional image of the set is associated with a location of the virtual camera along the transition path and each given transitional image of the set is generated based on a corresponding blending ratio corresponding to the location of the virtual camera for the given transitional image of the set; and

display the set of transitional images in sequence according to the locations associated with each given transitional image of the set along the transition path with the transitional image in order to provide the appearance of jumping between the first street-level image and the second street-level image.

11. The system of claim 10, wherein for each given transitional image of the set the corresponding blending ratio further corresponds to an orientation of the virtual camera at the location associated with the given transitional image of the set.

12. The system of claim 9, wherein the one or more processors are further configured to:

generate a second transitional image using a second blending ratio different from the blending ratio; and

display the second transitional images in sequence with the transitional image according to associated locations of the second transitional images and the transitional image along the transition path in order to provide the appearance of jumping between the first street-level image and the second street-level image.

13. The system of claim 9, wherein the one or more processors are further configured to identify the identified pixels further based on a distance along the transition path between the first street-level image and the location.

14. The system of claim 9, wherein the one or more processors are further configured to identify the identified pixels further based on which of the first street-level image and second street-level image are closest along the transition path to the location.

15. The system of claim 9, wherein the one or more processors are further configured to determine the transition path by also determining an arc shape of the transition path according to a distance between a location of the first street-level image and a location of the second street-level image.

16. The system of claim 11, wherein the one or more processors are further configured to use the blending ratio by also determining a color contribution for each pixel of the display.

22. The system of claim 9, wherein when the color contribution from the identified pixels is greater than the color contribution from the rasterized fragments the virtual camera is aligned perpendicular with the street-level images.