Planar panorama imagery generation

• US 10,038,842 B2
• Filed: 05/18/2015
• Issued: 07/31/2018
• Est. Priority Date: 11/01/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Williams, Samuel Thomas, Rosenberg, Jonathan Paul, Kopf, Johannes, Kim, Don Dongwoo, Kanumuri, Sandeep
• Primary Examiner(s): Torrente, Richard
• Application Number:
  US14/714,768
• Publication Number:
  US 20150249786A1
• Time in Patent Office: 1,170 Days
• Field of Search: None
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

One or more techniques and/or systems are disclosed for generating improved planar panorama imagery, which may be used to view street-level, human-scale locations along a desired path, for example. One or more desired objects can be detected in source imagery that may be used to produce the resulting planar panorama imagery. Respective object regions comprising a desired object can be identified in one or more frames of the source imagery. One or more portions of the source imagery that do not comprise the object region for the desired object can be joined with at least some of the one or more frames that do comprise the object region for the desired object. The resulting planar panorama imagery can thus comprise the desired object. Also, the planar panorama imagery can be generated so as to lack one or more non-desired objects and/or adverse effects that may result therefrom.

Claims

1. A method for generating planar panorama imagery, comprising:

analyzing a first frame of planar panorama source imagery to identify objects within the first frame, the analysis comprising;

comparing characteristics of the objects identified within the first frame to characteristics of a first type of object to classify a first object within the first frame corresponding to the first type of object; and

comparing characteristics of the objects identified within the first frame to characteristics of a second type of object to identify a second object within the first frame corresponding to the second type of object;

analyzing a second frame of the planar panorama source imagery to identify objects within the second frame, the analysis comprising;

comparing characteristics of the objects identified within the second frame to characteristics of the first type of object to identify the first object within the second frame corresponding to the first type of object;

receiving an indication that the first object type has a higher prioritization than the second object type;

determining that the second object overlaps the first object in the first frame and that the second object does not overlap the first object in the second frame;

selecting the second frame, but not the first frame, for use in the planar panorama imagery based on the prioritization and the determination that the second object overlaps the first object in the first frame; and

generating the planar panorama imagery with at least the second frame, but not the first frame.

2. The method of claim 1, the analyzing the second frame comprising:

comparing characteristics of objects identified within the second frame to characteristics of the second type of object to identify the second object within the second frame corresponding to the second type of object.

3. The method of claim 1, the analyzing the second frame comprising:

comparing characteristics of objects identified within the second frame to characteristics of the second type of object; and

determining that the second object is not present in the second frame.

4. The method of claim 1, the first object located within a first object region of the first frame and the second object located with a second object region of the first frame and the determining that the second object overlaps the first object in the first frame comprising:

determining that a pixel of the first frame is situated within the first object region and the second object region.

5. The method of claim 1, comprising:

assigning a first prioritization to the first object; and

assigning a second prioritization to the second object.

6. The method of claim 5, comprising at least one of:

assigning the first prioritization to the first object based upon the first object being of the first type of object;

or

assigning the second prioritization to the second object based upon the second object being of the second type of object.

7. The method of claim 5, comprising at least one of:

assigning the first prioritization to the first object based upon user input;

or

assigning the second prioritization to the second object based upon user input.

8. The method of claim 1, characteristics of objects corresponding to at least one of an object shape, an object color, or an object edge.

9. The method of claim 1, the second frame consecutive to the first frame in the planar panorama source imagery.

10. A system for generating planar panorama imagery, comprising:

one or more processing units; and

memory comprising instructions that when executed by at least one of the one or more processing units perform operations, comprising;

analyzing a first frame of planar panorama source imagery to identify objects within the first frame, the analysis comprising;

comparing characteristics of the objects identified within the first frame to characteristics of a first type of object to classify a first object within the first frame corresponding to the first type of object; and

comparing characteristics of objects identified within the first frame to characteristics of a second type of object to classify a second object within the first frame corresponding to the second type of object;

analyzing a second frame of the planar panorama source imagery, the analysis comprising;

comparing characteristics of objects identified within the second frame to characteristics of the first type of object to classify the first object within the second frame corresponding to the first type of object;

receiving an indication that the first object type has a higher prioritization than the second object type;

determining that the second object overlaps the first object in the first frame and that the second object does not overlap the first object in the second frame;

selecting the second frame, but not the first frame, for use in the planar panorama imagery based on the prioritization and the determination that the second object overlaps the first object in the first frame; and

generating the planar panorama imagery with at least the second frame, but not the first frame.

11. The system of claim 10, the analyzing the second frame comprising:

comparing characteristics of objects identified within the second frame to characteristics of the second type of object to identify the second object within the second frame corresponding to the second type of object.

12. The system of claim 10, the analyzing the second frame comprising:

comparing characteristics of objects identified within the second frame to characteristics of the second type of object; and

determining that the second object is not present in the second frame.

13. The system of claim 10, the first object located within a first object region of the first frame and the second object located with a second object region of the first frame and the determining that the second object overlaps the first object in the first frame comprising:

determining that a pixel of the first frame is situated within the first object region and the second object region.

14. The system of claim 10, the operations comprising:

assigning a first prioritization to the first object; and

assigning a second prioritization to the second object.

15. The system of claim 14, the operations comprising at least one of:

assigning the first prioritization to the first object based upon the first object being of the first type of object;

or

assigning the second prioritization to the second object based upon the second object being of the second type of object.

16. The system of claim 14, the operations comprising at least one of:

assigning the first prioritization to the first object based upon user input;

or

assigning the second prioritization to the second object based upon user input.

17. The system of claim 10, characteristics of objects corresponding to at least one of an object shape, an object color, or an object edge.

18. The system of claim 10, the second frame consecutive to the first frame in the planar panorama source imagery.

19. A computer readable storage device comprising instructions that when executed perform a method for generating planar panorama imagery, the method comprising:

accessing source imagery including at least a first frame and a second frame;

receiving an indication that a first object type in source imagery has a higher prioritization than a second object type in the source imagery;

comparing characteristics of objects within the first frame to characteristics of a first type of object to identify a first object within the first frame corresponding to the first type of object;

comparing characteristics of objects within the first frame to characteristics of a second type of object to identify a second object within the first frame corresponding to the second type of object;

comparing characteristics of objects identified within the second frame to characteristics of the first type of object to identify the first object within the second frame corresponding to the first type of object;

determining that the second object overlaps the first object in the first frame and that the second object does not overlap the first object in the second frame;

selecting the second frame, but not the first frame, for use in the planar panorama imagery based on the prioritization and the determination that the second object overlaps the first object in the first frame; and

generating the planar panorama imagery with at least the second frame, but not the first frame.

20. The computer readable storage device of claim 19, wherein the first object is located within a first object region of the first frame and the second object located with a second object region of the first frame and the determining that the second object overlaps the first object in the first frame comprises determining that a pixel of the first frame is situated within the first object region and the second object region.