Procedural authoring

• US 9,659,406 B2
• Filed: 05/23/2014
• Issued: 05/23/2017
• Est. Priority Date: 05/07/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Brewer, Brett D., Flake, Gary W., Lindsay, Donald J., Drucker, Steven M., Arcas, Blaise Aguera Y., Farouki, Karim T, Lawler, Stephen, Sheppard, Adam P
• Primary Examiner(s): DULANEY, KATHLEEN YUAN
• Application Number:
  US14/286,264
• Publication Number:
  US 20140254921A1
• Time in Patent Office: 1,096 Days
• Field of Search: 382154
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 17/10 Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/20212 Image combination
• CPC Class Codes: G06T 2210/36 Level of detail
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85

Abstract

A three dimensional (3D) model of an object or environment may be created from images and other information of the object or environment. The 3D model may be created by aligning the images. The 3D model may include surfaces that are based on surfaces of the object or environment. The 3D model may be displayed, extorted, modified and so on.

Claims

1. A method comprising:

receiving, by a computing device and from a scanning device, a plurality of color 2D images;

analyzing the plurality of color 2D images for similarities to identify 2D images representing points-of-view of an object in a physical environment;

obtaining a point cloud that describes a third dimension of the object;

generating, by the computing device, a 3D mesh model of the object based at least in part on the identified 2D images and the point cloud that describes the third dimension of the object, the 3D mesh model including surfaces that are scaled based at least in part on surfaces of the object;

utilizing a value of information (VOI) analysis to identify one or more dimensional reduction settings;

analyzing the 3D mesh model based at least in part on the one or more dimensional reduction settings to identify another object that is represented in the 3D mesh model; and

exporting the 3D mesh model.

2. The method of claim 1, further comprising:

enabling a user to view the 3D mesh model and to modify the 3D mesh model.

3. The method of claim 1, wherein the 3D mesh model is exported to a virtual environment.

4. The method of claim 1, further comprising:

performing dimensionality reduction on the 3D mesh model to reduce a number of features of the 3D mesh model.

5. The method of claim 1, wherein the generating includes aligning the identified 2D images onto one or more surfaces of the 3D mesh model.

6. One or more computer-readable hardware storage devices storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform the method of claim 1.

7. One or more computer-readable hardware storage devices storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising:

collecting, from a first source, a plurality of 2D color images of an object in a physical environment;

collecting, from a second source, data of the object in the physical environment, the data being of a different data type than the plurality of 2D color images and the second source being different than the first source;

creating a 3D mesh model of the object based at least in part on the plurality of 2D color images and the data, the 3D mesh model being sized based at least in part on a size of the object, the creating including aligning the plurality of 2D color images;

utilizing a value of information (VOI) analysis to identify one or more dimensional reduction settings;

analyzing the 3D mesh model based at least in part on the one or more dimensional reduction settings to identify another object that is represented in the 3D mesh model; and

exporting the 3D mesh model.

8. The one or more computer-readable hardware storage devices of claim 7, wherein the acts further comprise:

enabling a user to view the 3D mesh model and to modify the 3D mesh model.

9. The one or more computer-readable hardware storage devices of claim 7, wherein the 3D mesh model is exported to a virtual environment.

10. The one or more computer-readable hardware storage devices of claim 7, wherein the acts further comprise:

performing dimensionality reduction on the 3D mesh model to reduce a number of features of the 3D mesh model.

11. The one or more computer-readable hardware storage devices of claim 7, wherein the plurality of 2D color images are aligned by overlaying the plurality of 2D color images onto one or more surfaces of the 3D mesh model.

12. The one or more computer-readable hardware storage devices of claim 7, wherein the acts further comprise scaling a surface of the 3D model based at least in part on a corresponding surface of the object.

13. The one or more computer-readable hardware storage devices of claim 7, wherein the 3D mesh model is exported into a virtual gaming environment.

14. A system comprising:

one or more processors;

memory communicatively coupled to the one or more processors;

a content aggregator stored in memory and executable by the one or more processors to receive, from a first source, a plurality of images representing points-of-view of an object in a physical environment and receive, from a second source, scan data of the object in the physical environment, the scan data being of a different data type than the plurality of images and the second source being different than the first source;

a model component stored in the memory and executable by the one or more processors to;

generate a 3D mesh model of the object based at least in part on the plurality of images and the scan data, the 3D mesh model including surfaces that are scaled based at least in part on surfaces of the object, the generating including aligning the plurality of images; and

utilize a value of information (VOI) analysis to identify one or more dimensional reduction settings and analyze the 3D mesh model based at least in part on the one or more dimensional reduction settings to identify another object that is represented in the 3D mesh model; and

an interface communicatively coupled to the one or more processors and configured to export the 3D mesh model.

15. The system of claim 14, wherein the model component is further executable by the one or more processors to perform dimensionality reduction on the 3D mesh model to reduce a number of features of the 3D mesh model.

16. The system of claim 14, wherein the interface is configured to export the 3D mesh model to a virtual environment.

17. The system of claim 14, wherein the model component is configured to align the plurality of images by overlaying the plurality of images onto one or more surfaces of the 3D mesh model.

18. The system of claim 14, further comprising:

an output device communicatively coupled to the one or more processors, the output device configured to display the 3D mesh model.

19. The system of claim 14, further comprising:

an input device communicatively coupled to the one or more processors, the input device configured to receive an input to modify the 3D mesh model.

20. The system of claim 14, wherein the interface is configured to export the 3D mesh model to a virtual gaming environment.