Method and system for creating realistic smooth three-dimensional depth contours from two-dimensional images

• US 6,515,659 B1
• Filed: 12/19/2001
• Issued: 02/04/2003
• Est. Priority Date: 05/27/1998
• Status: Expired due to Term

• Current Assignee: Reald DDMG Acquisition LLC (RealD Incorporated)
• Original Assignee: In-Three Incorporated (Digital Domain Media Group Incorporated)
• Inventors: Kaye, Michael C., Best, Charles J. L., Haynes, Robby R.
• Primary Examiner(s): Nguyen, Phu K.
• Application Number:
  US10/029,625
• Time in Patent Office: 412 Days
• Field of Search: 345/419,420,426,619,620
• US Class Current: 345/419
• CPC Class Codes: G06T 11/203 Drawing of straight lines or curves
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2219/2021 Shape modification

Abstract

An image processing method and system for converting two-dimensional images into realistic reproductions, or recreations of three-dimensional images are embodied in a variety of interactive image processing tools that allow a user to apply any number or combination of image pixel repositioning depth contouring effects, algorithms or the like to create three-dimensional images with smooth realistic appearing surface contours.

Claims

1. An image processing system for converting two-dimensional images into three-dimensional images, comprising:

an interactive user interface providing an object drawing tool configured to allow a user of the system to apply one or more straight or curved path lines within a defined object of a two-dimensional image, the interactive user interface being configured to generate from the two-dimensional image a three-dimensional image in which one or more of the path lines appears to the user as a ridge protruding toward or away from the user, the ridge contouring from all points on the path line towards a boundary perimeter of the defined object.

2. The image processing system for converting two-dimensional images into three-dimensional images of claim 1, wherein the interactive interface is configured such that the object drawing tool allows the user to freehand draw the one or more straight or curved path lines within the defined object.

41. An article of data storage media upon which is stored images, information or data created employing any of the methods or systems of claims 1-40.

42. A method for providing a three-dimensional image, comprising the steps of:

receiving or accessing data created employing any of the methods or systems of claims 1-40; and

employing the data to reproduce a three-dimensional image.

3. An image processing method for converting two-dimensional images into three-dimensional images, comprising the steps of:

freehand drawing a straight or curved path line within a defined object of a two-dimensional image;

associating the path line as points of a crest or trough; and

displacing a plurality of image pixels of the two-dimensional image to provide a three-dimensional image in which the crest or trough appears within the three-dimensional image to transition in depth along the path line, the crest or trough contouring from all points on the path line towards a boundary perimeter of the defined object.

4. An image processing method for converting two-dimensional images into three-dimensional images, comprising the steps of:

applying one or more straight or curved path lines within a defined object of a two-dimensional image;

associating surface contours of a three-dimensional shape with the path lines; and

generating a graphical representation of the surface contours along the path lines.

5. The image processing method for converting two-dimensional images into three-dimensional images of claim 4, wherein the step of generating a graphical representation comprises:

rotating the graphical representation to correspond to a different perspective view of the surface contours.

6. An image processing system for converting two-dimensional images into three-dimensional images, comprising:

an interactive user interface configured to allow a user of the system to view a perimeter boundary of a first defined object of a two-dimensional image, designate a portion of the perimeter boundary, and employ the portion to create a second defined object which facilitates an identification of pixels to be horizontally displaced to create an image for a three-dimensional image.

7. The image processing system for converting two-dimensional images into three-dimensional images of claim 6, wherein the interactive user interface includes a drawing tool configured to allow the user to designate a portion of the perimeter boundary by drawing around it.

8. An image processing system for converting two-dimensional images into three-dimensional images, comprising:

an interactive user interface configured to allow a user of the system to apply one or more points or straight or curved path lines within a defined object of a two-dimensional image, and to generate a three-dimensional image that shows surface contours between a boundary perimeter of the defined object and the one or more points or straight or curved path lines;

wherein the interactive user interface is configured to allow the user to adjust a shape of the surface contours about one or more of the points or path lines while viewing changes made to the surface contours.

9. The image processing system for converting two-dimensional images into three-dimensional images of claim 8, wherein the interactive user interface includes an object drawing tool configured to allow the user to freehand draw the one or more points or straight or curved path lines within a defined object.

10. The image processing system for converting two-dimensional images into three-dimensional images of claim 8, wherein the interactive user interface is configured to allow the user to designate a point or path line within the defined object as an upper or lower-most elevation within the three-dimensional image.

11. The image processing system for converting two-dimensional images into three-dimensional images of claim 8, wherein the interactive user interface is configured to allow the user to select one or more of the points or path lines and to apply a plurality of three-dimensional shapes about the points or path lines to generate the surface contours.

12. The image processing system for converting two-dimensional images into three-dimensional images of claim 11, wherein the interactive user interface is configured to allow the user to apply a plurality of shaping algorithms corresponding to the three-dimensional shapes in any combination and in any order.

13. The image processing system for converting two-dimensional images into three-dimensional images of claim 8, wherein the interactive user interface is configured to allow the user to employ a function to adjust a curved shape of the surface contours.

14. The image processing system for converting two-dimensional images into three-dimensional images of claim 8, wherein the interactive user interface is configured to allow the user to control variations in perceived depth between any two of the points or path lines.

15. The image processing system for converting two-dimensional images into three-dimensional images of claim 8, wherein the interactive user interface is configured to allow the user to change a shape of the surface contours by modifying or repositioning one or more of the points or path lines.

16. The image processing system for converting two-dimensional images into three-dimensional images of claim 8, wherein the interactive user interface is configured to allow the user to apply amplitude adjustments to the surface contours.

17. The image processing system for converting two-dimensional images into three-dimensional images of claim 8, wherein the interactive user interface is configured to allow the user to apply shape adjustments to the surface contours.

18. The image processing system for converting two-dimensional images into three-dimensional images of claim 8, wherein the interactive user interface is configured to employ information already associated with a defined object to adjust a shape of the surface contours.

19. The image processing system for converting two-dimensional images into three-dimensional images of claim 18, wherein the information comprises depth information is employed by the system to automatically determine surface contour changes along a path line applied to the defined object.

20. The image processing system for converting two-dimensional images into three-dimensional images of claim 18, wherein the information comprises skew or stretch information that is employed by the system to determine variations in the shape of the surface contours from foreground to background.

21. The image processing system for converting two-dimensional images into three-dimensional images of claim 8, wherein:

the defined object excludes one or more areas within a perimeter boundary of the defined object; and

the interactive user interface is configured to generate the three-dimensional image with the surface contours not being applied to the one or more areas such that the one or more areas appear as holes in the defined object allowing any object with its own depth properties to be visible.

22. The image processing system for converting two-dimensional images into three-dimensional images of claim 8, wherein the interactive user interface is configured to allow the user of the system to capture any portion or the entirety of a defined object of the two-dimensional image and to apply the surface contours to the captured portion of the defined object.

23. An image processing system for converting two-dimensional images into three-dimensional images, comprising:

an interactive user interface configured to allow a user of the system to apply a path identifying an area within an object of a two-dimensional image, the two-dimensional image to become one of the images of a three-dimensional image, and to adjust depth information associated with an area defined by the path that results in changes in a perceived depth of the area and of its adjoining surfaces relative to a perimeter boundary of the defined object in the three-dimensional image.

24. An image processing system for converting two-dimensional images into three-dimensional images, comprising:

an interactive user interface configured to allow a user of the system to complete a full shape of a partial visible portion of a shape of a defined object in a two-dimensional image, the full shape of the partial object portion being a virtual object, as if the full object portion was visible and not hidden, having associated with it information pertaining to visible surface contours within a perimeter boundary of the defined object and hidden surface contours outside the perimeter boundary, the interactive user interface being configured to generate a three-dimensional image that shows the hidden surface contours when image objects in the two-dimensional image are repositioned such that a previously hidden portion of the image object is revealed.

25. An image processing method for converting two-dimensional images into three-dimensional images, comprising the steps of:

processing data associated with a defined object and one or more ridge points or path lines within a two-dimensional image to generate a mesh pattern of triangles that overlays the two-dimensional image; and

associating pixel depth values with the mesh pattern of triangles to create three-dimensional surface contours for rendering the two-dimensional image into a three-dimensional image.

26. The image processing method for converting two-dimensional images into three-dimensional images of claim 25, wherein the data associated with the defined object comprises locations of points that are positioned along a boundary perimeter of the defined object.

27. The image processing method for converting two-dimensional images into three-dimensional images of claim 25, wherein the data associated with the path lines comprises locations of points that are positioned along the path lines.

28. The image processing method for converting two-dimensional images into three-dimensional images of claim 25, wherein the data processing step comprises:

defining triangles of the mesh pattern with vertices taken from points along a boundary perimeter of the defined object and from the one or more ridge points.

29. The image processing method for converting two-dimensional images into three-dimensional images of claim 25, wherein the data processing step comprises:

defining triangles of the mesh pattern with vertices taken from points along an outline of the defined object and from points along the one or more path lines.

30. The image processing method for converting two-dimensional images into three-dimensional images of claim 25, wherein the data processing step comprises:

defining triangles of the mesh pattern such that the triangles obey one or more rules pertaining to distance minimization between vertices of the triangles.

31. The image processing method for converting two-dimensional images into three-dimensional images of claim 30, wherein the triangles are defined employing a recursive processing technique.

32. The image processing method for converting two-dimensional images into three-dimensional images of claim 30, wherein the triangles are defined employing an edge swapping technique.

33. The image processing method for converting two-dimensional images into three-dimensional images of claim 25, wherein the data processing step comprises:

defining triangles of the mesh pattern such that the triangles obey one or more rules pertaining to vertices of the triangles crossing contours.

34. The image processing method for converting two-dimensional images into three-dimensional images of claim 25, further comprising the step of:

processing the data associated with the mesh pattern employing an edge smoothing processing technique to further modify the triangles of the mesh pattern prior to the step of associating pixel depth values.

35. The image processing method for converting two-dimensional images into three-dimensional images of claim 34, wherein the edge smoothing processing technique further modifies the triangles such that two vertices of each triangle lie at the same depth level and the third vertex lies at an adjacent depth level.

36. The image processing method for converting two-dimensional images into three-dimensional images of claim 25, further comprising the step of:

processing the data associated with the mesh pattern employing a triangle refining processing technique to further modify the mesh pattern to accommodate one or more additional surface contours prior to the step of associating pixel depth values.

37. The image processing method for converting two-dimensional images into three-dimensional images of claim 36, wherein the triangle refining processing technique determines the one or more additional surface contours in response to a desired level of contour resolution.

38. The image processing method for converting two-dimensional images into three-dimensional images of claim 25, wherein the pixel depth values correspond to a surface shape that is applied to the mesh pattern.

39. The image processing method for converting two-dimensional images into three-dimensional images of claim 38, wherein the surface shape is three-dimensional.

40. The image processing method for converting two-dimensional images into three-dimensional images of claim 25, wherein the pixel depth values correspond to a desired amplitude of the mesh pattern.