PLANAR PANORAMA IMAGERY GENERATION

• US 20150249786A1
• Filed: 05/18/2015
• Published: 09/03/2015
• Est. Priority Date: 11/01/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Williams, Samuel Thomas, Rosenberg, Jonathan Paul, Kopf, Johannes, Kim, Don Dongwoo, Kanumuri, Sandeep
• Granted Patent: US 10,038,842 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

One or more techniques and/or systems are disclosed for generating improved planar panorama imagery, which may be used to view street-level, human-scale locations along a desired path, for example. One or more desired objects can be detected in source imagery that may be used to produce the resulting planar panorama imagery. Respective object regions comprising a desired object can be identified in one or more frames of the source imagery. One or more portions of the source imagery that do not comprise the object region for the desired object can be joined with at least some of the one or more frames that do comprise the object region for the desired object. The resulting planar panorama imagery can thus comprise the desired object. Also, the planar panorama imagery can be generated so as to lack one or more non-desired objects and/or adverse effects that may result therefrom.

Claims

1. A method for generating planar panorama imagery, comprising:

inputting a first frame of planar panorama source imagery into a plurality of object detectors, comprising;

comparing, by a first object detector of the plurality of object detectors, characteristics of objects identified within the first frame to characteristics of a first type of object to identify a first object within the first frame corresponding to the first type of object; and

comparing, by a second object detector of the plurality of object detectors, characteristics of objects identified within the first frame to characteristics of a second type of object to identify a second object within the first frame corresponding to the second type of object;

inputting a second frame of the planar panorama source imagery into the plurality of object detectors, comprising;

comparing, by the first object detector of the plurality of object detectors, characteristics of objects identified within the second frame to characteristics of the first type of object to identify the first object within the second frame corresponding to the first type of object;

determining that the second object overlaps the first object in the first frame and that the second object does not overlap the first object in the second frame; and

selecting the second frame, but not the first frame, for use in the planar panorama imagery responsive to the determining when a first prioritization associated with the first object is higher than a second prioritization associated with the second object.

2. The method of claim 1, the inputting a second frame comprising:

comparing, by the second object detector of the plurality of object detectors, characteristics of objects identified within the second frame to characteristics of the second type of object to identify the second object within the second frame corresponding to the second type of object.

3. The method of claim 1, the inputting a second frame comprising:

comparing, by the second object detector of the plurality of object detectors, characteristics of objects identified within the second frame to characteristics of the second type of object; and

determining that the second object is not present in the second frame.

4. The method of claim 1, the first object located within a first object region of the first frame and the second object located with a second object region of the first frame and the determining that the second object overlaps the first object in the first frame comprising:

determining that a pixel of the first frame is situated within the first object region and the second object region.

5. The method of claim 1, comprising:

assigning the first prioritization to the first object; and

assigning the second prioritization to the second object.

6. The method of claim 1, comprising at least one of:

assigning the first prioritization to the first object based upon the first object being of the first type of object;

or

assigning the second prioritization to the second object based upon the second object being of the second type of object.

7. The method of claim 1, comprising at least one of:

assigning the first prioritization to the first object based upon user input;

or assigning the second prioritization to the second object based upon user input.

8. The method of claim 1, characteristics of objects corresponding to at least one of an object shape, an object color, or an object edge.

9. The method of claim 1, the second frame consecutive to the first frame in the planar panorama source imagery.

10. A system for generating planar panorama imagery, comprising:

one or more processing units; and

memory comprising instructions that when executed by at least one of the one or more processing units perform operations, comprising;

inputting a first frame of planar panorama source imagery into a plurality of object detectors, comprising;

comparing, by a first object detector of the plurality of object detectors, characteristics of objects identified within the first frame to characteristics of a first type of object to identify a first object within the first frame corresponding to the first type of object; and

comparing, by a second object detector of the plurality of object detectors, characteristics of objects identified within the first frame to characteristics of a second type of object to identify a second object within the first frame corresponding to the second type of object;

inputting a second frame of the planar panorama source imagery into the plurality of object detectors, comprising;

comparing, by the first object detector of the plurality of object detectors, characteristics of objects identified within the second frame to characteristics of the first type of object to identify the first object within the second frame corresponding to the first type of object;

determining that the second object overlaps the first object in the first frame and that the second object does not overlap the first object in the second frame; and

selecting the second frame, but not the first frame, for use in the planar panorama imagery responsive to the determining when a first prioritization associated with the first object is higher than a second prioritization associated with the second object.

11. The system of claim 10, the inputting a second frame comprising:

comparing, by the second object detector of the plurality of object detectors, characteristics of objects identified within the second frame to characteristics of the second type of object to identify the second object within the second frame corresponding to the second type of object.

12. The system of claim 10, the inputting a second frame comprising:

comparing, by the second object detector of the plurality of object detectors, characteristics of objects identified within the second frame to characteristics of the second type of object; and

determining that the second object is not present in the second frame.

13. The system of claim 10, the first object located within a first object region of the first frame and the second object located with a second object region of the first frame and the determining that the second object overlaps the first object in the first frame comprising:

determining that a pixel of the first frame is situated within the first object region and the second object region.

14. The system of claim 10, the operations comprising:

assigning the first prioritization to the first object; and

assigning the second prioritization to the second object.

15. The system of claim 10, the operations comprising at least one of:

assigning the first prioritization to the first object based upon the first object being of the first type of object;

or

assigning the second prioritization to the second object based upon the second object being of the second type of object.

16. The system of claim 10, the operations comprising at least one of:

assigning the first prioritization to the first object based upon user input;

or

assigning the second prioritization to the second object based upon user input.

17. The system of claim 10, characteristics of objects corresponding to at least one of an object shape, an object color, or an object edge.

18. The system of claim 10, the second frame consecutive to the first frame in the planar panorama source imagery.

19. A computer readable storage medium comprising instructions that when executed perform a method for generating planar panorama imagery, comprising:

inputting a first frame of planar panorama source imagery into a plurality of object detectors, comprising;

comparing, by a first object detector of the plurality of object detectors, characteristics of objects identified within the first frame to characteristics of a first type of object to identify a first object within the first frame corresponding to the first type of object; and

comparing, by a second object detector of the plurality of object detectors, characteristics of objects identified within the first frame to characteristics of a second type of object to identify a second object within the first frame corresponding to the second type of object;

inputting a second frame of the planar panorama source imagery into the plurality of object detectors, comprising;

comparing, by the first object detector of the plurality of object detectors, characteristics of objects identified within the second frame to characteristics of the first type of object to identify the first object within the second frame corresponding to the first type of object;

determining that the second object overlaps the first object in the first frame and that the second object does not overlap the first object in the second frame; and

selecting the second frame, but not the first frame, for use in the planar panorama imagery responsive to the determining when a first prioritization associated with the first object is higher than a second prioritization associated with the second object.

20. The computer readable storage medium of claim 19, the inputting a second frame comprising:

comparing, by the second object detector of the plurality of object detectors, characteristics of objects identified within the second frame to characteristics of the second type of object; and

at least one of;

identifying the second object within the second frame as corresponding to the second type of object;

or

determining that the second object is not present in the second frame.