Image three-dimensional (3D) modeling

• US 9,324,184 B2
• Filed: 12/14/2011
• Issued: 04/26/2016
• Est. Priority Date: 12/14/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Williams, Samuel, Jones, Jett Evan, Sibley, Peter G., Kim, Don, Kanumuri, Sandeep, Rosenberg, Jonathan
• Primary Examiner(s): McDowell, Jr., Maurice L
• Assistant Examiner(s): Isanians, Raffi
• Application Number:
  US13/325,466
• Publication Number:
  US 20130155047A1
• Time in Patent Office: 1,595 Days
• Field of Search: 345/582, 345423-425, 345/430, 345/435, 345/437, 345/442, 382/268, 382/271, 382/272, 382/275, 382/285
• US Class Current: 1/1
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration

Abstract

Among other things, one or more techniques and/or systems are disclosed for creating a three-dimensional model of an image. Image data, comprise a series of images of a location, such as along a route, can be received. The image data can comprise a first image element location for a first image element, where the first image element location may correspond to a three-dimensional point in the image, such as a location in the image with a depth from a point of observation. The first image element can be segmented into a first façade plane using the first image element location. The first façade plane can be merged with a second façade plane, resulting in a three-dimensional model of the image. The second façade plane can comprise a second image element, where the image data comprises a second image element location for the second image element.

Claims

1. A computer-based method, comprising:

receiving a two-dimensional image comprising a first image element and a second image element, the first image element representing a first object in the two-dimensional image and the second image element representing a second object in the two-dimensional image;

determining a first depth for the first object relative to an observation point;

determining a second depth for the second object relative to the observation point;

segmenting, by a processing unit, the first image element into a first faç

ade plane representing objects having depths within a first depth range relative to the observation point, the first depth within the first depth range;

segmenting the second image element into a second faç

ade plane representing objects having depths within a second depth range relative to the observation point, the second depth within the second depth range, the second faç

ade plane parallel to the first faç

ade plane; and

merging the first faç

ade plane with the second faç

ade plane resulting in a three-dimensional model of the two-dimensional image, the three-dimensional model providing for movement of the first faç

ade plane at a first speed relative to the observation point and the second faç

ade plane at a second speed relative to the observation point, the first speed different than the second speed.

2. The method of claim 1, wherein the first image element corresponds to a first pixel of the two-dimensional image.

3. The method of claim 2, wherein the second image element corresponds to a second pixel of the two-dimensional image.

4. The method of claim 1, wherein the determining a first depth comprising determining the first depth based at least upon stereoscopic estimation using at least two images respectively comprising different views.

5. The method of claim 1, wherein the determining a first depth comprising determining the first depth based at least upon surface modeling of a scene depicted by the two-dimensional image.

6. The method of claim 1, wherein the determining a first depth comprising determining the first depth based at least upon light detection and ranging (LIDAR).

7. The method of claim 1, wherein the determining the first depth comprising determining the first depth based at least upon radio wave detection and ranging (RADAR).

8. The method of claim 1, wherein the first object is different than the second object.

9. The method of claim 1, further comprising merging the first faç

ade plane and the second faç

ade plane with a third faç

ade plane representing objects having depths within a third depth range relative to the observation point.

10. The method of claim 9, further comprising rendering a planar panorama using the first faç

ade plane, the second faç

ade plane, and the third faç

ade plane after the merging.

11. The method of claim 1, wherein the first image element is different than the second image element.

12. The method of claim 1, further comprising rendering a planar panorama using the three-dimensional model of the two-dimensional image.

13. A system, implemented at least in part via a processing unit, comprising:

at least one processor; and

a memory, operatively connected with the at least one processor, storing computer-executable instructions that, when executed, cause the at least one processor to;

receive a two-dimensional image comprising a first image element and a second image element, the first image element representing a first object in the two-dimensional image and the second image element representing a second object in the two-dimensional image, the second object different than the first object;

determine a first depth for the first object relative to an observation point;

determine a second depth for the second object relative to the observation point;

a segmentation component configured to;

segment the first image element into a first faç

ade plane representing objects having depths within a first depth range relative to the observation point, the first depth within the first depth range; and

segment the second image element into a second faç

ade plane representing objects having depths within a second depth range relative to the observation point, the second depth within the second depth range; and

merge the first faç

ade plane with the second faç

ade plane resulting in a three-dimensional model of the two-dimensional image.

14. The system of claim 13, wherein the first image element corresponds to a first pixel of the two-dimensional image and the second image element corresponds to a second pixel of the two-dimensional image.

15. The system of claim 13, wherein the three-dimensional model providing for movement of the first faç

ade plane at a first speed relative to the observation point and the second faç

ade plane at a second speed relative to the observation point, and wherein the first speed is different than the second speed.

16. The system of claim 13, wherein the computer-executable instructions, when executed, cause the at least one processor to determine the first depth based at least upon stereoscopic estimation using at least two images respectively comprising different views.

17. The system of claim 13, wherein the computer-executable instructions, when executed, cause the at least one processor to determine the first depth based at least upon light detection and ranging (LIDAR).

18. The system of claim 13, wherein the computer-executable instructions, when executed, cause the at least one processor to determine the first depth based at least upon radio wave detection and ranging (RADAR).

19. The system of claim 13, wherein the first image element is different than the second image element.

20. A computer readable device comprising computer executable instructions that when executed perform a method, comprising:

receiving an image comprising a first image element and a second image element, the first image element representing a first object in the image and the second image element representing a second object in the image;

determining a first depth for the first object relative to an observation point;

determining a second depth for the second object relative to the observation point;

segmenting the first image element into a first faç

ade plane representing objects having depths within a first depth range relative to the observation point, the first depth within the first depth range;

segmenting the second image element into a second faç

ade plane representing objects having depths within a second depth range relative to the observation point, the second depth within the second depth range; and

merging the first faç

ade plane with the second faç

ade plane resulting in a three-dimensional model of the image.