Modeling system using surface patterns and geometric relationships

• US 6,226,004 B1
• Filed: 09/12/1997
• Issued: 05/01/2001
• Est. Priority Date: 09/12/1997
• Status: Expired due to Term

• Current Assignee: Autodesk, Inc.
• Original Assignee: Autodesk, Inc.
• Inventors: Nishihara, H. Keith
• Primary Examiner(s): Luu, Matthew
• Assistant Examiner(s): HARRISON, CHANTE E
• Application Number:
  US08/928,200
• Time in Patent Office: 1,327 Days
• Field of Search: 345/419, 345/420, 345/429
• US Class Current: 345/420
• CPC Class Codes: G06T 11/40 Filling a planar surface by adding surface attributes, e.g. colour or texture
• CPC Class Codes: G06T 17/10 Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation

Abstract

Method for creating a three dimensional model of a scene based on a two dimensional image of the scene. The method includes identifying a plurality of surfaces in the image, identifying a plurality of constraints between the surfaces in the image, identifying a pattern on a first surface in the image, identifying a first constraint between the pattern and another feature of the image, and generating a three dimensional model of the scene based on the surfaces, the plurality of constraints, and the first constraint. The pattern may include a grout line in a tile surface or an identifying a mark on a wall paper.

Claims

1. A method of creating a three dimensional model of a scene based on a two dimensional image of the scene, the method comprising:

identifying a plurality of surfaces in the image;

identifying a plurality of constraints between the surfaces in the image;

identifying a pattern on a first surface in the image;

identifying a first constraint between the pattern and another feature of the image; and

generating a three dimensional model of the scene based on the surfaces, the plurality of constraints, and the first constraint.

2. The method of claim 1, wherein identifying a pattern includes identifying a grout line in a tile surface.

3. The method of claim 1, wherein identifying a pattern includes identifying a mark on a wall paper.

4. The method of claim 1, wherein the another feature comprises a surface.

5. The method of claim 1, wherein the another feature comprises a second pattern.

6. The method of claim 1, identifying a plurality of surfaces in the image comprising:

drawing lines on an image, the lines corresponding to edges of a surface.

7. The method of claim 1, the plurality of constraints comprising a perpendicular relationship between a second surface in the image and a third surface in the image.

8. The method of claim 1, the plurality of constraints comprising a parallel relationship between a second surface in the image and a third surface in the image.

9. The method of claim 1, the plurality of constraints comprising a distance relationship between a second surface in the image and a third surface in the image.

10. The method of claim 9, the plurality of constraints comprising a parallel relationship between a second surface in the image and a third surface in the image.

11. The method of claim 1, the plurality of constraints comprising an angular relationship between a second surface in the image and a third surface in the image.

12. The method of claim 1, the first constraint comprising a perpendicular relationship between the pattern and the another feature of the image.

13. The method of claim 1, the first constraint comprising a parallel relationship between the pattern and the another feature of the image.

14. The method of claim 1, the first constraint comprising a distance relationship between the pattern and the another feature of the image.

15. The method of claim 1, the first constraint comprising an angular relationship between the pattern and the another feature of the image.

16. The method of claim 1, wherein generating the three dimensional model comprises:

identifying a second surface in the image;

generating a first three dimensional representation of the second surface;

generating a second three dimensional representation of a third surface based on the first three dimensional representation and a constraint between the second surface and the third surface; and

repeatedly generating three dimensional representations of surfaces based on generated representations of surfaces and constraints between surfaces.

17. The method of claim 1, wherein identifying a pattern on a first surface in the image comprises:

receiving a user input corresponding to a line drawn by a user, the line corresponding to the pattern; and

the user identifying a coplanar relationship between the pattern and a first surface.

18. The method of claim 1, wherein identifying a pattern on a first surface in the image comprises:

automatically identifying a direction corresponding to the pattern; and

automatically identifying a coplanar relationship between the pattern and a first surface.

19. The method of claim 1, comprising generating a second image based on the three dimensional model.

20. A method of creating a three dimensional model of a scene based on a two dimensional image of the scene, the method comprising:

generating a camera model based on vanishing points determined from the image;

identifying a plurality of constraints between surfaces in the image;

generating a first three dimensional model based on the constraints and the camera model;

identifying a first constraint between the pattern and a first surface in the image;

generating a second three dimensional model of the scene based on the surfaces, the plurality of constraints, and the first constraint.

21. The method of claim 20, generating a first three dimensional model comprising:

identifying a second surface in the image;

generating a first three dimensional representation of the second surface;

generating a second three dimensional representation of a third surface based on the first three dimensional representation and a constraint between the second surface and the third surface; and

repeatedly generating three dimensional representations of surfaces based on generated representations of surfaces and constraints between surfaces.

22. The method of claim 21, identifying a first surface in the image comprising identifying a surface corresponding to a floor in the scene.

23. The method of claim 21, the three dimensional representation of the first surface comprising a three dimensional vector.

24. The method of claim 20, comprising:

determining an error in the three dimensional model based on constraints in the plurality of constraints; and

adjusting the model to reduce the error.

25. The method of claim 20, wherein generating the three dimensional model comprises:

weighing the first constraint between the pattern and a first surface in the image more heavily than at least one other constraint in the plurality of constraints in order to cause a second image generated from the three dimensional model to tend to display a second pattern correctly.

26. The method of claim 20, wherein generating the three dimensional model comprises:

using a subset of the constraints in order to increase processing speed of the generating.

27. A method of creating a three dimensional model of a scene based on an image of the scene, the method comprising:

identifying at least a first surface and a second surface in the scene;

identifying a first geometric relationship between the first surface and the second surface;

identifying a pattern in the first surface;

identifying a second geometric relationship between the pattern and another feature of the image; and

generating a camera model based on the image;

generating a three dimensional model of the scene based on the model of the camera, the first geometric relationship, and the second geometric relationship.

28. The method of claim 27, generating a camera model comprising:

identifying mutually orthogonal sets of lines that appear parallel in the scene represented by the image;

determining vanishing points based on the mutually orthogonal sets of lines;

generating a model based on the vanishing points.

29. The method of claim 28, wherein generating a camera model comprises:

determining a focal length; and

determining an image center.

30. The method of claim 28, wherein generating a camera model comprises:

determining a distance between a third surface and a camera by which the image may have been generated; and

determining a three dimensional model orientation.

31. The method of claim 28, wherein generating a camera model comprises:

determining a focal length;

determining an image center;

determining a distance between the camera by which the image was generated; and

determining a three dimensional model orientation.

32. The method of claim 27, wherein generating a camera model comprises:

receiving a user input from a menu of cubes, the cubes corresponding to possible scene orientations.

33. The method of claim 27, comprising:

generating a second image based on the three dimensional model.

34. The method of claim 33, comprising:

displaying a second pattern in the first surface on the second image based on the three dimensional model.

35. The method of claim 27, comprising:

providing an indication to a user that sufficient constraints have been identified to create a three dimensional model of the scene.

36. The method of claim 27, comprising:

generating a three dimensional model;

determining an error in the three dimensional model based on constraints; and

providing the indication to the user based on the error.

37. The method of claim 27, wherein providing an indication comprises:

coloring a surface in the image.

38. A computer program product, comprising:

a computer usable medium having a computer readable program code embodied therein for causing a computer to create a three dimensional model of a scene based on a two dimensional image of the scene, the computer readable program code comprising;

computer readable program code that identifies a plurality of surfaces in the image;

computer readable program code that identifies a plurality of constraints between the surfaces in the image;

computer readable program code that identifies a pattern on a first surface in the image;

computer readable program code that identifies a first constraint between the pattern and a feature of the image; and

computer readable program code that generates a three dimensional model of the scene based on the surfaces, the plurality of constraints, and the first constraint.

39. The computer program product of claim 38, wherein code that generates the three dimensional model comprises:

code that identifies a second surface in the image;

code that generates a first three dimensional representation of the second surface;

code that generates a second three dimensional representation of a third surface based on the first three dimensional representation and a constraint between the second surface and the third surface; and

code that repeatedly generates three dimensional representations of surfaces based on generated representations of surfaces and constraints between surfaces.

40. The computer program product of claim 38, wherein code that identifies a pattern on a first surface in the image comprises:

code that automatically identifies a direction corresponding the pattern; and

code that automatically identifies a coplanar relationship between the pattern and the first surface.