Selecting time-distributed panoramic images for display

• US 9,972,121 B2
• Filed: 04/22/2014
• Issued: 05/15/2018
• Est. Priority Date: 04/22/2014
• Status: Expired due to Fees

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Li, Wenfeng, Hutchison, Allen
• Primary Examiner(s): Faulk, Devona E
• Assistant Examiner(s): BEARD, CHARLES LLOYD
• Application Number:
  US14/258,594
• Publication Number:
  US 20150302633A1
• Time in Patent Office: 1,484 Days
• CPC Class Codes: G06T 15/10 Geometric effects
• CPC Class Codes: G06T 3/04 Context-preserving transformations, e.g. by using an importance map panospheric to cylindrical image transformations G06T3/12
• CPC Class Codes: G09G 2340/045 Zooming at least part of an image, i.e. enlarging it or shrinking it
• CPC Class Codes: G09G 2360/122 Tiling
• CPC Class Codes: G09G 2380/10 Automotive applications
• CPC Class Codes: G09G 5/14 Display of multiple viewports

Abstract

Images may be automatically annotated with place identifiers based on textual metadata associated with the images. The quality of these place identifiers may be scored based on a number of different factors. For example, these annotations may also include confidence and topicality values indicative of the relationships between the place identifier and the text as well as topics of the text of the textual metadata. Images that also have additional location information that includes GPS coordinates or a location specified by a user, may be used to evaluate the quality of a given place identifier, combination of confidence and topicality values, and the overall annotation system. These valuations may, in turn, be used to score the quality of such automatic annotations of other images that do not have such additional location information.

Claims

1. A computer implemented method for selecting time-distributed panoramas for display, the method comprising:

accessing, by one or more computing devices, a plurality of panoramic images, each panoramic image being associated with multi-dimensional location information defined in at least two dimensions, a one-dimensional (1D) snap location along a path at least partially traveled during image capture, three dimensional (3D) geometry data corresponding to a surface of an object depicted in the panoramic image, and time information identifying when the panoramic image was captured;

for a first panoramic image of the plurality, querying the plurality of panoramic images and selecting, by the one or more computing devices, a second panoramic image of the plurality such that a second 1D snap location associated with the second panoramic image is closest to a first 1D snap location associated with the first panoramic image relative to 1D snap locations information associated with others of the plurality of panoramic images, at least a portion of a geographic area depicted in the first panoramic image is also depicted in the second panoramic image, and first time information associated with the first panoramic image and second time information associated with the second panoramic image indicate that the first panoramic image and the second panoramic image were captured on different days;

determining whether the second 1D snap location is within a threshold range of the first 1D snap location;

providing, by the one or more computing devices, the first panoramic image for display with an indication that the second panoramic image is available when the second 1D snap location is within the threshold range of the first 1D snap location;

determining, by the one or more computing devices, a zoom level for the second panoramic image based on first 3D geometry data and first multi-dimensional location information of the first panoramic image, and second 3D geometry data and second multi-dimensional location information of the second panoramic image, such that the zoom level for the second panoramic image is consistent with a zoom level of the first panoramic image; and

providing, by the one or more computing devices, the second panoramic image for display with the zoom level for the second panoramic image.

2. The method of claim 1, further comprising:

receiving view port information identifying a viewing angle of a display of the first panoramic image and a zoom level of the display of the first panoramic image,

wherein determining the zoom level for the second panoramic image further includes;

identifying a first distance to a three dimensional point on a surface of an object in the viewing angle of the display of the first panoramic image based on the first 3D geometry data, the first multi-dimensional location information, and the zoom level of the display of the first panoramic image;

identifying a second distance to the three dimensional point on the surface of the object in the second panoramic image based on the first location information and the second 3D geometry data; and

determining the zoom level for the second panoramic image using at least the first distance and the second distance.

3. The method of claim 2, wherein determining the zoom level for the second panoramic image is further based on a ratio of the first distance to the second distance.

4. The method of claim 2, wherein the second distance is a linear distance between the three dimensional point and the second location information.

5. The method of claim 2, wherein when the zoom level of the display of the first panoramic image indicates that the first panoramic is not zoomed in, the first distance is a linear distance between the first location of the first panoramic image and the three dimensional point on the surface of the object in the viewing angle of the display of the first panoramic image.

6. The method of claim 2, wherein the view port information further identifies a viewing angle of a display of the first panoramic image, and the method further comprises:

determining a viewing angle of the second panoramic image based on the viewing angle; and

the viewing angle for the second panoramic image is provided with the second panoramic image.

7. The method of claim 1, wherein the 1D snap location associated with each of the plurality of panoramic images is defined as a point on a line along a central portion of roadway.

8. The system of claim 2, wherein the 1D snap location associated with each particular panoramic image of the plurality of panoramic images is defined relative to a point on a line along a central portion of road.

9. A system comprising one or more computing devices, the one or more computing devices being configured to:

access a plurality of panoramic images, each panoramic image being associated with multi-dimensional location information defined in at least two dimensions, a one-dimensional (1D) snap location along a path at least partially traveled during image capture, three dimensional (3D) geometry data corresponding to a surface of an object depicted in the panoramic image, and time information identifying when the panoramic image was captured;

for a first panoramic image of the plurality, query the plurality of panoramic images and select a second panoramic image of the plurality such that a second 1D snap location associated with the second panoramic image is closest to a first 1D snap location associated with the first panoramic image relative to 1D snap locations associated with others of the plurality of panoramic images, at least a portion of a geographic area depicted in the first panoramic image is also depicted in the second panoramic image, and first time information associated with the first panoramic image and second time information associated with the second panoramic image indicate that the first panoramic image and the second panoramic image were captured on different days;

determine whether the second 1D snap location is within a threshold range of the first 1D snap location;

provide the first panoramic image for display with an indication that the second panoramic image is available when the second 1D snap location is within the threshold range of the first 1D snap location;

determine a zoom level for the second panoramic image based on first 3D geometry data and first multi-dimensional location information of the first panoramic image, and second 3D geometry data and second multi-dimensional location information of the second panoramic image, such that the zoom level for the second panoramic image is consistent with a zoom level of the first panoramic image; and

provide the second panoramic image for display with the zoom level for the second panoramic image.

10. The system of claim 9, wherein the one or more computing devices are further configured to:

receive view port information identifying a viewing angle of a display of the first panoramic image and a zoom level of the display of the first panoramic image, and

determine the zoom level for the second panoramic image further by;

identifying a first distance to a three dimensional point on a surface of an object in the viewing angle of the display of the first panoramic image based on the first 3D geometry data, the first multi-dimensional location information, and the zoom level of the display of the first panoramic image;

identifying a second distance to the three dimensional point on the surface of the object in the second panoramic image based on the first location information and the second 3D geometry data; and

determining the zoom level for the second panoramic image using at least the first distance and the second distance.

11. The system of claim 10, wherein determining the zoom level for the second panoramic image is further based on a ratio of the first distance to the second distance.

12. The system of claim 10, wherein the second distance is a linear distance between the three dimensional point and the second location information.

13. The system of claim 10, wherein when the zoom level of the display of the first panoramic image indicates that the first panoramic is not zoomed in, the first distance is a linear distance between the first location of the first panoramic image and the three dimensional point on the surface of the object in the viewing angle of the display of the first panoramic image.

14. The system of claim 10, wherein the view port information further identifies a viewing angle of a display of the first panoramic image, and the one or more computing devices are further configured to:

determine a viewing angle of the second panoramic image based on the viewing angle; and

wherein the viewing angle for the second panoramic image is provided with the second panoramic image.

15. A non-transitory computer-readable storage medium on which computer readable instructions of a program are stored, the instructions, when executed by one or more processors, cause the one or more processors to perform a method, the method comprising:

accessing a plurality of panoramic images, each panoramic image being associated with multi-dimensional location information defined in at least two dimensions, a one-dimensional (1D) snap location along a path at least partially traveled during image capture, three dimensional (3D) geometry data corresponding to a surface of an object depicted in the panoramic image, and time information identifying when the panoramic image was captured;

for a first panoramic image of the plurality, query the plurality of panoramic images and select a second panoramic image of the plurality such that a second 1D snap location associated with the second panoramic image is closest to a first 1D snap location associated with the first panoramic image relative to 1D snap locations associated with others of the plurality of panoramic images, at least a portion of a geographic area depicted in the first panoramic image is also depicted in the second panoramic image, and first time information associated with the first panoramic image and second time information associated with the second panoramic image indicate that the first panoramic image and the second panoramic image were captured on different days;

determining whether the second 1D snap location is within a threshold range of the first 1D snap location;

providing the first panoramic image for display with an indication that the second panoramic image is available when the second 1D snap location is within the threshold range of the first 1D snap location;

determining a zoom level for the second panoramic image based on first 3D geometry data and first multi-dimensional location information of the first panoramic image, and second 3D geometry data and second multi-dimensional location information of the second panoramic image, such that the zoom level for the second panoramic image is consistent with a zoom level of the first panoramic image; and

providing the second panoramic image for display with the zoom level for the second panoramic image.

16. The medium of claim 15, wherein the method further comprises:

receiving view port information identifying a viewing angle of a display of the first panoramic image and a zoom level of the display of the first panoramic image, and

determining the zoom level for the second panoramic image further includes;

identifying a first distance to a three dimensional point on a surface of an object in the viewing angle of the display of the first panoramic image based on the first 3D geometry data, the first multi-dimensional location information, and the zoom level of the display of the first panoramic image;

identifying a second distance to the three dimensional point on the surface of the object in the second panoramic image based on the first location information and the second 3D geometry data; and

determining the zoom level for the second panoramic image using at least the first distance and the second distance.

17. The medium of claim 16, wherein determining the zoom level for the second panoramic image is further based on a ratio of the first distance to the second distance.

18. The medium of claim 16, wherein the second distance is a linear distance between the three dimensional point and the second location information.

19. The medium of claim 16, wherein when the zoom level of the display of the first panoramic image indicates that the first panoramic is not zoomed in, the first distance is a linear distance between the first location of the first panoramic image and the three dimensional point on the surface of the object in the viewing angle of the display of the first panoramic image.

20. The medium of claim 16, wherein the view port information further identifies a viewing angle of a display of the first panoramic image, and the method further comprises:

determining a viewing angle of the second panoramic image based on the viewing angle; and

the viewing angle for the second panoramic image is provided with the second panoramic image.