Street-level imagery acquisition and selection

• US 9,036,000 B1
• Filed: 09/27/2011
• Issued: 05/19/2015
• Est. Priority Date: 09/27/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ogale, Abhijit S., Carceroni, Rodrigo L., Dulong, Carole, Vincent, Luc
• Primary Examiner(s): Patel, Jay
• Assistant Examiner(s): Huang, Frank
• Application Number:
  US13/246,456
• Time in Patent Office: 1,330 Days
• Field of Search: 348/36, 348/39, 715/757
• US Class Current: 348/36
• CPC Class Codes: G01C 21/3667 Display of a road map G01C21/3614 takes precedence; guidance using 3D or perspective road maps G01C21/3635
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

A street-level imagery acquisition and selection process identifies which images are published in a street field view. An imagery database includes panoramas each corresponding to a set of images acquired from a single viewpoint. The panoramas are attached to corresponding positions on a road network graph. The graph is divided into a set of selection paths, each of which includes a topologically linear sequence of road segments. Each selection path is evaluated to select a set of panoramas to be published in the path. Panoramas of interior road segments are selected before panoramas at intersections. Selected panorama identifiers for each interior road segment of the selection paths and each intersection correspond to a position along the road network graph. The selected panorama identifiers are then published in the street field view.

Claims

1. A computer-implemented method comprising:

associating, using one or more processors, a plurality of panoramas to corresponding positions on a road network graph, wherein each panorama comprises at least one image captured at a geographic location of the corresponding position;

dividing, using the one or more processors, the road network graph into a plurality of selection paths, wherein each selection path comprises one or more road segments, each road segment being associated with a subset of multiple candidate panoramas of the plurality of panoramas;

determining, using the one or more processors, characteristics of the multiple candidate panoramas, the characteristics including visual properties and the characteristics being prioritized such that a first type of characteristic is valued differently than a second type of characteristic;

selecting, using the one or more processors, one of the multiple candidate panoramas associated with each position for interior portions of each road segment in each selection path, wherein the selecting is based at least in part on a highest priority characteristic;

selecting, using the one or more processors, one of the plurality of panoramas for each intersection in each selection path; and

publishing, using the one or more processors, identifiers for each panorama selected for each selection path or a selected intersection in the road network graph.

2. The method of claim 1, further comprising:

selecting one or more run sequences to cover each selection path before selecting the one or more of the plurality of panoramas, wherein each run sequence comprises a continuous sequence of panoramas captured consecutively in time.

3. The method of claim 1, further comprising:

selecting, as a path seed, one panorama of each selection path; and

constructing the selection path beginning from the panorama selected as the path seed.

4. The method of claim 1, wherein selecting panoramas for interior portions of each road segment in each selection path and selecting the one of the plurality of panoramas for each intersection in each selection path satisfies one or more predetermined conditions for panorama selection.

5. The method of claim 4, wherein the one or more predetermined conditions comprise requirements based on at least one of a group consisting of:

an acquisition-continuity requirement that selects spatially adjacent panoramas that comprise images captured under the same conditions;

a spacing and coverage requirement that maintains spacing between consecutively selected panoramas; and

an intrinsic constraint requirement that corresponds to independent panorama properties.

6. The method of claim 5, wherein the intrinsic constraint requirement is satisfied before the acquisition-continuity requirement and the spacing and coverage requirement.

7. A system comprising:

a processor configured to manage a plurality of modules, the modules comprising;

a panorama snapper module configured to associate a plurality of panoramas to corresponding positions on a road network graph, wherein each of the plurality of panoramas comprises at least one image captured at a geographic location of the corresponding position;

a selection path creator module configured to divide the road network graph into a plurality of selection paths, wherein each selection path comprises one or more road segments, each road segment being associated with a subset of multiple candidate panoramas of the plurality of panoramas;

a selection path configured to determine characteristics of the multiple candidate panoramas, the characteristics including visual properties and the characteristics being prioritized such that a first type of characteristic is valued differently than a second type of characteristic;

a selection path optimizer module configured to select one of the multiple candidate panoramas associated with each position for interior portions of each road segment in each selection path, wherein the selecting is based at least in part on a highest priority characteristic;

a selection intersection creator module configured to receive the plurality of selection paths from the selection path creator and identify at least one intersection in each selection path, wherein each intersection is associated with the corresponding panoramas at positions associated therewith; and

a selection intersection optimizer module configured to select one of the plurality of panoramas for each intersection in the road network graph.

8. The system of claim 7, wherein the selection path creator module is further configured to select one or more run sequences to cover each selection path, wherein each run sequence comprises a continuous sequence of panoramas captured consecutively in time.

9. The system of claim 7, wherein the selection path creator module is further configured to:

select, as a path seed, one panorama of each selection path; and

construct the selection path beginning from the panorama selected as the path seed.

10. The system of claim 7, wherein the selection path optimizer is configured to select the one or more of the plurality of the panoramas for interior portions of each road segment in each selection path such that one or more predetermined conditions for panorama selection are satisfied.

11. The system of claim 10, wherein the one or more predetermined conditions comprise requirements based on at least one of a group consisting of:

an acquisition-continuity requirement that selects spatially adjacent panoramas that comprise images captured under the same conditions;

a spacing and coverage requirement that maintains spacing between consecutively selected panoramas; and

an intrinsic constraint requirement that corresponds to independent panorama properties.

12. The system of claim 11, wherein the intrinsic constraint requirement is satisfied before the acquisition-continuity requirement and the spacing and coverage requirement.

13. The system of claim 7, wherein the selection intersection optimizer is configured to select the one of the panoramas for each intersection in each selection path such that one or more predetermined conditions for panorama selection are satisfied.

14. The system of claim 13, wherein the one or more predetermined conditions comprise requirements based on at least one of a group consisting of:

an acquisition-continuity requirement that selects spatially adjacent panoramas that comprise images captured under the same conditions;

a spacing and coverage requirement that maintains spacing between consecutively selected panoramas; and

an intrinsic constraint requirement that corresponds to independent panorama properties.

15. The system of claim 14, wherein the intrinsic constraint requirement is satisfied before the acquisition-continuity requirement and the spacing and coverage requirement.

16. The system of claim 7, further comprising:

a database for storing the plurality of panoramas, wherein the panorama snapper module receives the plurality of panoramas as input from the database.

17. The system of claim 7, wherein the selection path optimizer module and the selection intersection optimizer module are further configured to publish identifiers for each panorama selected for each selection path in the road network graph.

18. A computer-implemented method for selecting imagery, the method comprising:

associating a plurality of panoramas to corresponding positions on a road network graph, wherein each panorama comprises at least one image captured at a geographic location of the corresponding position;

dividing the road network graph into a plurality of selection paths, wherein each selection path comprises one or more road segments, each road segment being associated with a subset of multiple candidate panoramas of the plurality of panoramas;

determining, using the one or more processors, characteristics of the multiple candidate panoramas, the characteristics including visual properties and the characteristics being prioritized such that a first type of characteristic is valued differently than a second type of characteristic;

selecting one of the multiple candidate panoramas associated with each position for interior portions of each road segment in each selection path, wherein the selecting is based at least in part on a highest priority characteristic; and

publishing identifiers for each panorama selected for each selection path in the road network graph.

19. The method of claim 18, further comprising,

selecting one or more run sequences to cover each selection path before selecting the one or more of the plurality of panoramas, wherein each run sequence comprises a continuous sequence of panoramas captured consecutively in time.

20. The method of claim 18, wherein each selection path comprises at least one intersection associated with the corresponding panoramas at positions associated therewith.

21. The method of claim 20, further comprising:

selecting one of the plurality of panoramas for each intersection in each selection path after selecting the one or more of the plurality of panoramas for the interior portions of each road segment in each selection path.

22. The method of claim 21 wherein selecting the one or more of the plurality of panoramas for interior portions of each road segment in each selection path and selecting the one of the plurality of panoramas for each intersection in each selection path satisfies one or more predetermined conditions for panorama selection.