Using image content to facilitate navigation in panoramic image data

• US 10,163,263 B2
• Filed: 03/16/2017
• Issued: 12/25/2018
• Est. Priority Date: 02/27/2008
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Zhu, Jiajun, Filip, Daniel Joseph, Vincent, Luc
• Primary Examiner(s): Nguyen, Kimbinh T
• Application Number:
  US15/460,727
• Publication Number:
  US 20170186229A1
• Time in Patent Office: 649 Days
• Field of Search: 345427, 345419, 345426
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 15/06 Ray-tracing
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 2200/32 involving image mosaicing
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images

Abstract

The technology uses image content to facilitate navigation in panoramic image data. Aspects include providing a first image including a plurality of avatars, in which each avatar corresponds to an object within the first image, and determining an orientation of at least one of the plurality of avatars to a point of interest within the first image. A viewport is determined for a first avatar in accordance with the orientation thereof relative to the point of interest, which is included within the first avatar'"'"'s viewport. In response to received user input, a second image is selected that includes at least a second avatar and the point of interest from the first image. A viewport of the second avatar in the second image is determined and the second image is oriented to align the second avatar'"'"'s viewpoint with the point of interest to provide navigation between the first and second images.

Claims

1. A computer-implemented method for image navigation, the method comprising:

providing, by one or more computing devices, a first image including a plurality of avatars, each avatar corresponding to an object within the first image, the first image being geo-coded to a first one of the plurality of avatars;

determining, by the one or more computing devices, an orientation of at least one of the plurality of avatars to a point of interest within the first image;

determining, by the one or more computing devices, a viewport of the first avatar in accordance with the orientation of the first avatar to the point of interest within the first image, the point of interest within the first image being included within the viewport of the first avatar, the viewport presenting only a portion of the first image;

in response to received user input, selecting by the one or more computing devices a second image, the second image including at least a second one of the avatars from the first image and the point of interest from the first image;

determining, by the one or more computing devices, a viewport of the second avatar in the second image, the viewport of the second avatar in the second image including the point of interest from the first image, the viewport of the second avatar in the second image presenting only a portion of the second image; and

orienting, by the one or more computing devices, the second image to align the viewpoint of the second avatar in the second image with the point of interest in order to provide navigation between the first image and the second image.

2. The computer-implemented method of claim 1, wherein the first image and the second image comprise parts of a panoramic image, and a portion of the panoramic image displayed in the second avatar'"'"'s viewport contains a set of content that corresponds to a set of content displayed in the viewport of the first avatar.

3. The computer-implemented method of claim 1, further comprising generating a transition between the first avatar and another one of the plurality of avatars, the transition including at least one intermediate image for avatar positions between the first avatar and the other avatar.

4. The computer-implemented method of claim 3, further comprising orienting the at least one intermediate image to face the point of interest.

5. The computer-implemented method of claim 1, wherein:

the first image corresponds to a first geolocation and the second image corresponds to a second geolocation; and

a position of the second avatar is a same position as a closest available panoramic image to the point of interest.

6. The computer-implemented method of claim 1, wherein the point of interest is determined in response to a user selection in the first image.

7. The computer-implemented method of claim 1, wherein the one or more processors use a virtual model to navigate between the first and second images.

8. The computer-implemented method of claim 1, wherein the first and second avatars are associated with objects located on different lanes of a street.

9. The computer-implemented method of claim 1, wherein the first and second images are of nearby locations having different perspectives.

10. The computer-implemented method of claim 1, wherein at least one of the first and second images comprises a photographic image taken from a street level.

11. The computer-implemented method of claim 1, wherein at least some features of the first image are not included in the second image.

12. The computer-implemented method of claim 1, wherein the first and second images were captured by a same camera.

13. The computer-implemented method of claim 12, wherein the first and second images were captured by the same camera at different points in time.

14. A system, comprising:

a memory; and

one or more one or more computing devices, operatively coupled to the memory, configured to execute a navigation controller that;

provides a first image including a plurality of avatars, each avatar corresponding to an object within the first image, the first image being geo-coded to a first one of the plurality of avatars;

determines an orientation of at least one of the plurality of avatars to a point of interest within the first image;

determines a viewport of the first avatar in accordance with the orientation of the first avatar to the point of interest within the first image, the point of interest within the first image being included within the viewport of the first avatar, the viewport presenting only a portion of the first image;

in response to received user input, selects a second image, the second image including at least a second one of the avatars from the first image and the point of interest from the first image;

determines a viewport of the second avatar in the second image, the viewport of the second avatar in the second image including the point of interest from the first image, the viewport of the second avatar in the second image presenting only a portion of the second image; and

orients the second image to align the viewpoint of the second avatar in the second image with the point of interest in order to provide navigation between the first image and the second image.

15. The system of claim 14, wherein the navigation controller is operative to generate a transition between the first avatar and another one of the plurality of avatars, the transition including at least one intermediate image for avatar positions between the first avatar and the other avatar.

16. The system claim 15, wherein the navigation controller is operative to orient the at least one intermediate image to face the point of interest.

17. The system of claim 14, wherein the point of interest is determined in response to a user selection in the first image.

18. A non-transitory computer-readable medium storing computer program instructions, which, when executed by one or more computing devices, cause the one or more computing devices to perform a method comprising:

providing a first image including a plurality of avatars, each avatar corresponding to an object within the first image, the first image being geo-coded to a first one of the plurality of avatars;

determining an orientation of at least one of the plurality of avatars to a point of interest within the first image;

determining a viewport of the first avatar in accordance with the orientation of the first avatar to the point of interest within the first image, the point of interest within the first image being included within the viewport of the first avatar, the viewport presenting only a portion of the first image;

in response to received user input, selecting a second image, the second image including at least a second one of the avatars from the first image and the point of interest from the first image;

determining a viewport of the second avatar in the second image, the viewport of the second avatar in the second image including the point of interest from the first image, the viewport of the second avatar in the second image presenting only a portion of the second image; and

orienting the second image to align the viewpoint of the second avatar in the second image with the point of interest in order to provide navigation between the first image and the second image.

19. The non-transitory computer-readable medium of claim 18, wherein the method further comprises generating a transition between the first avatar and another one of the plurality of avatars, the transition including at least one intermediate image for avatar positions between the first avatar and the other avatar.

20. The non-transitory computer-readable medium of claim 19, wherein the method further comprises orienting the at least one intermediate image to face the point of interest.