SELECTING TIME-DISTRIBUTED PANORAMIC IMAGES FOR DISPLAY

• US 20150302633A1
• Filed: 04/22/2014
• Published: 10/22/2015
• Est. Priority Date: 04/22/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Hutchison, Allen, Li, Wenfeng
• Granted Patent: US 9,972,121 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 15/10 Geometric effects
• CPC Class Codes: G06T 3/04 Context-preserving transformations, e.g. by using an importance map panospheric to cylindrical image transformations G06T3/12
• CPC Class Codes: G09G 2340/045 Zooming at least part of an image, i.e. enlarging it or shrinking it
• CPC Class Codes: G09G 2360/122 Tiling
• CPC Class Codes: G09G 2380/10 Automotive applications
• CPC Class Codes: G09G 5/14 Display of multiple viewports

Abstract

Images may be automatically annotated with place identifiers based on textual metadata associated with the images. The quality of these place identifiers may be scored based on a number of different factors. For example, these annotations may also include confidence and topicality values indicative of the relationships between the place identifier and the text as well as topics of the text of the textual metadata. Images that also have additional location information that includes GPS coordinates or a location specified by a user, may be used to evaluate the quality of a given place identifier, combination of confidence and topicality values, and the overall annotation system. These valuations may, in turn, be used to score the quality of such automatic annotations of other images that do not have such additional location information.

Claims

1. A computer implemented method for selecting time-distributed panoramas for display, the method comprising:

accessing, by one or more computing devices, a plurality of panoramic images, each given panoramic image of the plurality of panoramic images being associated with multi-dimensional location information defined in at least two dimensions, uni-dimensional (1D) location information defined in only one dimension, three dimensional (3D) geometry data corresponding to a surface of an object depicted in the given panoramic image, and time information identifying when the given panoramic image was captured;

for a first one of the plurality of panoramic images, the first panoramic image being associated with first multi-dimensional location information, first 1D location information, and first time information, selecting, by the one or more computing devices, a second panoramic image of the plurality of panoramic images, the second panoramic image panoramic image being associated with second multi-dimensional location information, second 1D location information, and second time information, the second panoramic image being selected such that the second 1D location information is closest to the first 1D location information and the first time information and second time information indicate that the first panoramic image and the second panoramic image were captured on different days;

providing, by the one or more computing devices, the first panoramic image for display with an indication that the second panoramic image is available;

determining, by the one or more computing devices, a zoom level for the second panoramic image based on the first 3D geometry data, the first multi-dimensional location information, the second 3D geometry data, and the second multi-dimensional location information; and

providing, by the one or more computing devices, the second panoramic image for display with the zoom level for the second panoramic image.

2. The method of claim 1, further comprising:

receiving view port information identifying a viewing angle of a display of the first panoramic image and a zoom level of the display of the first panoramic image,

wherein determining the zoom level for the second panoramic image further includes;

identifying a first distance to a three dimensional point on a surface of an object in the viewing angle of the display of the first panoramic image based on the first 3D geometry data, the first multi-dimensional location information, and the zoom level of the display of the first panoramic image;

identifying a second distance to the three dimensional point on the surface of the object in the second panoramic image based on the first location information and the second 3D geometry data; and

determining the zoom level for the second panoramic image using at least the first distance and the second distance.

3. The method of claim 2, wherein determining the zoom level for the second panoramic image is further based on a ratio of the first distance to the second distance.

4. The method of claim 2, wherein the second distance is a linear distance between the three dimensional point and the second location information.

5. The method of claim 2, wherein when the zoom level of the display of the first panoramic image indicates that the first panoramic is not zoomed in, the first distance is a linear distance between the first location of the first panoramic image and the three dimensional point on the surface of the object in the viewing angle of the display of the first panoramic image.

6. The method of claim 2, wherein the view port information further identifies a viewing angle of a display of the first panoramic image, and the method further comprises:

determining a viewing angle of the second panoramic image based on the viewing angle; and

the viewing angle for the second panoramic image is provided with the second panoramic image.

7. The method of claim 1, wherein the 1D location information associated with each particular panoramic image of the plurality of panoramic images is defined relative to a point on a line corresponding to a portion of road.

14. The system of claim 2, wherein the 1D location information associated with each particular panoramic image of the plurality of panoramic images is defined relative to a point on a line corresponding to a portion of road.

8. A system comprising one or more computing devices, the one or more computing devices being configured to:

access a plurality of panoramic images, each given panoramic image of the plurality of panoramic images being associated with multi-dimensional location information defined in at least two dimensions, uni-dimensional (1D) location information defined in only one dimension, three dimensional (3D) geometry data corresponding to a surface of an object depicted in the given panoramic image, and time information identifying when the given panoramic image was captured;

for a first one of the plurality of panoramic images, the first panoramic image being associated with first multi-dimensional location information, first 1D location information, and first time information, select a second panoramic image of the plurality of panoramic images, the second panoramic image panoramic image being associated with second multi-dimensional location information, second 1D location information, and second time information, the second panoramic image being selected such that the second 1D location information is closest to the first 1D location information and the first time information and second time information indicate that the first panoramic image and the second panoramic image were captured on different days;

provide the first panoramic image for display with an indication that the second panoramic image is available;

determine a zoom level for the second panoramic image based on the first 3D geometry data, the first multi-dimensional location information, the second 3D geometry data, and the second multi-dimensional location information; and

provide the second panoramic image for display with the zoom level for the second panoramic image.

9. The system of claim 8, wherein the one or more computing devices are further configured to:

receive view port information identifying a viewing angle of a display of the first panoramic image and a zoom level of the display of the first panoramic image, and

determine the zoom level for the second panoramic image further by;

identifying a first distance to a three dimensional point on a surface of an object in the viewing angle of the display of the first panoramic image based on the first 3D geometry data, the first multi-dimensional location information, and the zoom level of the display of the first panoramic image;

identifying a second distance to the three dimensional point on the surface of the object in the second panoramic image based on the first location information and the second 3D geometry data; and

determining the zoom level for the second panoramic image using at least the first distance and the second distance.

10. The system of claim 9, wherein determining the zoom level for the second panoramic image is further based on a ratio of the first distance to the second distance.

11. The system of claim 9, wherein the second distance is a linear distance between the three dimensional point and the second location information.

12. The system of claim 9, wherein when the zoom level of the display of the first panoramic image indicates that the first panoramic is not zoomed in, the first distance is a linear distance between the first location of the first panoramic image and the three dimensional point on the surface of the object in the viewing angle of the display of the first panoramic image.

13. The system of claim 9, wherein the view port information further identifies a viewing angle of a display of the first panoramic image, and the one or more computing devices are further configured to:

determine a viewing angle of the second panoramic image based on the viewing angle; and

wherein the viewing angle for the second panoramic image is provided with the second panoramic image.

15. A non-transitory computer-readable storage medium on which computer readable instructions of a program are stored, the instructions, when executed by one or more processors, cause the one or more processors to perform a method, the method comprising:

accessing a plurality of panoramic images, each given panoramic image of the plurality of panoramic images being associated with multi-dimensional location information defined in at least two dimensions, uni-dimensional (1D) location information defined in only one dimension, three dimensional (3D) geometry data corresponding to a surface of an object depicted in the given panoramic image, and time information identifying when the given panoramic image was captured;

for a first one of the plurality of panoramic images, the first panoramic image being associated with first multi-dimensional location information, first 1D location information, and first time information, selecting a second panoramic image of the plurality of panoramic images, the second panoramic image panoramic image being associated with second multi-dimensional location information, second 1D location information, and second time information, the second panoramic image being selected such that the second 1D location information is closest to the first 1D location information and the first time information and second time information indicate that the first panoramic image and the second panoramic image were captured on different days;

providing the first panoramic image for display with an indication that the second panoramic image is available;

determining a zoom level for the second panoramic image based on the first 3D geometry data, the first multi-dimensional location information, the second 3D geometry data, and the second multi-dimensional location information; and

providing the second panoramic image for display with the zoom level for the second panoramic image.

16. The medium of claim 15, wherein the method further comprises:

receiving view port information identifying a viewing angle of a display of the first panoramic image and a zoom level of the display of the first panoramic image, and

determining the zoom level for the second panoramic image further includes;

identifying a first distance to a three dimensional point on a surface of an object in the viewing angle of the display of the first panoramic image based on the first 3D geometry data, the first multi-dimensional location information, and the zoom level of the display of the first panoramic image;

identifying a second distance to the three dimensional point on the surface of the object in the second panoramic image based on the first location information and the second 3D geometry data; and

determining the zoom level for the second panoramic image using at least the first distance and the second distance.

17. The medium of claim 16, wherein determining the zoom level for the second panoramic image is further based on a ratio of the first distance to the second distance.

18. The medium of claim 16, wherein the second distance is a linear distance between the three dimensional point and the second location information.

19. The medium of claim 16, wherein when the zoom level of the display of the first panoramic image indicates that the first panoramic is not zoomed in, the first distance is a linear distance between the first location of the first panoramic image and the three dimensional point on the surface of the object in the viewing angle of the display of the first panoramic image.

20. The medium of claim 16, wherein the view port information further identifies a viewing angle of a display of the first panoramic image, and the method further comprises:

determining a viewing angle of the second panoramic image based on the viewing angle; and

the viewing angle for the second panoramic image is provided with the second panoramic image.