Generation of texture maps for use in 3D computer graphics

• US 7,528,831 B2
• Filed: 09/17/2004
• Issued: 05/05/2009
• Est. Priority Date: 09/18/2003
• Status: Active Grant

• Current Assignee: Canon Europa N.V. (Canon Inc.)
• Original Assignee: Canon Europa N.V. (Canon Inc.)
• Inventors: Lyons, Alexander Ralph, Hong, Qi He, Baumberg, Adam Michael
• Primary Examiner(s): Nguyen; Kimbinh T
• Application Number:
  US10/943,103
• Publication Number:
  US 20050151751A1
• Time in Patent Office: 1,691 Days
• Field of Search: 345/581, 345/582, 345/427, 345/629, 345/420, 345/426, 345/589
• US Class Current: 345/420
• CPC Class Codes: G06T 15/04 Texture mapping

Abstract

A plurality of texture maps for a 3D computer model of an object are generated by processing images of the object, data defining the computer model, and data defining the relative positions and orientations of the images and computer model. A respective texture map is generated for the viewing position and direction of each image. Each texture map stores texture data for all parts of the computer model which are visible from the associated viewing position and direction and also texture data for all invisible parts of the computer model. Each texture map therefore stores texture data that can be used for rendering accurate images of the 3D computer model over a wide range of viewing positions and directions.

Claims

1. A method of processing data defining a three-dimensional computer model of the surface of an object, image data defining a plurality of different images of the object, and data defining the position and direction of each image and the three-dimensional computer model, to generate texture data from the images for use in rendering an image of the three-dimensional computer model, the method comprising:

generating a plurality of texture maps, each texture map being associated with a respective view of the three-dimensional computer model and each texture map defining texture data for parts of the three-dimensional computer model that are visible in the corresponding view and at least some parts of the three-dimensional computer model that are not visible in the corresponding view, wherein the steps are performed on a computer.

2. A method according to claim 1, wherein a respective texture map is generated for each image, each texture map defining texture data for parts of the three-dimensional computer model that are visible in the corresponding image and at least some parts of the three-dimensional computer model that are not visible in the corresponding image.

3. A method according to claim 1, wherein each texture map is generated by combining image data from a plurality of the images.

4. A method according to claim 3, wherein each respective texture map is generated by:

calculating a respective measure of the visibility of each part of the three-dimensional computer model in each image; and

generating texture data from the images for the parts of the three-dimensional computer model in dependence upon the calculated visibility measures.

5. A method according to claim 4, wherein the visibility measures are calculated taking into account occlusions of parts of the three-dimensional computer model by other parts.

6. A method according to claim 3, wherein each texture map is generated by combining image data from a plurality of images such that the contribution of image data from the image corresponding to the texture map is greater than the contribution of image data from any other respective image for at least some parts of the three-dimensional computer model.

7. A method according to claim 6, wherein the texture data for each respective texture map is generated such that the contribution of image data from the image corresponding to the texture map is greater than the contribution of image data from any other respective image for each part of the three-dimensional computer model that is visible in the corresponding image.

8. A method according to claim 1, wherein the texture data for each respective texture map is generated such that the texture data for parts of the three-dimensional computer model that are visible in the corresponding image is determined solely from the image data in the corresponding image.

9. A method according to claim 6, wherein each texture map is generated by:

calculating a respective measure of similarity of the view of the three-dimensional computer model available in the image corresponding to the texture map with the view of the three-dimensional computer model available in each other image; and

generating texture data for the texture map from the images in dependence upon the calculated measures.

10. A method according to claim 4, wherein texture data is generated for each respective texture map by combining image data from the images in proportions dependent upon the calculated measures.

11. A method according to claim 1, wherein the three-dimensional computer model comprises a mesh of polygons, each polygon defining a part of the three-dimensional computer model, and wherein each generated texture map defines a two-dimensional array of texels together with colour component values for the texels and positions within the array of the vertices of the polygons comprising the parts of the three-dimensional computer model for which the texture map stores texture data.

12. A method according to claim 1, wherein the three-dimensional computer model comprises a plurality of unconnected points in three-dimensional space, each point defining a part of the three-dimensional computer model, and wherein each generated texture map defines colour component values for each point comprising a part of the three-dimensional computer model for which the texture map stores texture data.

13. A method according to claim 1, further comprising generating a signal carrying data defining the generated texture maps.

14. A method according to claim 1, further comprising making a direct or indirect recording of data defining the generated texture maps.

15. A method of generating texture data for use in rendering an image of a three-dimensional computer model, the method comprising:

storing data defining a three-dimensional computer model of the surface of an object, image data defining a plurality of different images of the object, and data defining the position and direction of each image and the three-dimensional computer model; and

processing the data to generate a plurality of texture maps, each texture map being associated with a respective view of the three-dimensional computer model and each texture map defining texture data for parts of the three-dimensional computer model that are visible in the corresponding view and at least some parts of the three-dimensional computer model that are not visible in the corresponding view, wherein

the steps are performed on a computer.

16. A method of rendering an image of a three-dimensional computer model, the method comprising:

receiving data defining a viewing position and direction from which the image is to be rendered;

selecting a texture map for rendering the image from among a plurality of texture maps, wherein each texture map is associated with a camera viewing position and direction and defines texture data for parts of the three-dimensional computer model that are visible from the associated viewing position and direction and at least some parts of the three-dimensional computer model that are not visible from the associated viewing position and direction, and wherein the selection of the texture map for rendering is made in dependence upon the received viewing direction and the viewing directions associated with the texture maps; and

rendering an image of the three-dimensional computer model from the received viewing position and direction using texture data from the selected texture map to generate pixel data, wherein

the steps are performed on a computer.

17. A method according to claim 16, further comprising generating a signal carrying the generated pixel data.

18. A method according to claim 16, further comprising making a direct or indirect recording of the generated pixel data.

19. Apparatus for processing data defining a three-dimensional computer model of the surface of an object, image data defining a plurality of different images of the object, and data defining the position and direction of each image and the three-dimensional computer model, to generate texture data from the images for use in rendering an image of the three-dimensional computer model, the apparatus comprising:

a texture map generator operable to generate a plurality of texture maps, each texture map being associated with a respective view of the three-dimensional computer model and each texture map defining texture data for parts of the three-dimensional computer model that are visible in the corresponding view and at least some parts of the three-dimensional computer model that are not visible in the corresponding view.

20. Apparatus according to claim 19, wherein the texture map generator is operable to generate a respective texture map for each image, each texture map defining texture data for parts of the three-dimensional computer model that are visible in the corresponding image and at least some parts of the three-dimensional computer model that are not visible in the corresponding image.

21. Apparatus according to claim 19, wherein the texture map generator is operable to generate each respective texture map by combining image data from a plurality of the images.

22. Apparatus according to claim 21, wherein the texture map generator is operable to generate each respective texture map by:

calculating a respective measure of the visibility of each part of the three-dimensional computer model in each image; and

generating texture data from the images for the parts of the three-dimensional computer model in dependence upon the calculated visibility measures.

23. Apparatus according to claim 22, wherein the texture map generator is operable to calculate the visibility measures taking into account occlusions of parts of the three-dimensional computer model by other parts.

24. Apparatus according to claim 21, wherein the texture map generator is operable to generate each texture map by combining image data from a plurality of images such that the contribution of image data from the image corresponding to the texture map is greater than the contribution of image data from any other respective image for at least some parts of the three-dimensional computer model.

25. Apparatus according to claim 24, wherein the texture map generator is operable to generate the texture data for each respective texture map such that the contribution of image data from the image corresponding to the texture map is greater than the contribution of image data from any other respective image for each part of the three-dimensional computer model that is visible in the corresponding image.

26. Apparatus according to claim 19, wherein the texture map generator is operable to generate the texture data for each respective texture map such that the texture data for parts of the three-dimensional computer model that are visible in the corresponding image is determined solely from the image data in the corresponding image.

27. Apparatus according to claim 24, wherein the texture map generator is operable to generate each texture map by;

calculating a respective measure of similarity of the view of the three-dimensional computer model available in the image corresponding to the texture map with the view of the three-dimensional computer model available in each other image; and

generating texture data for the texture map from the images in dependence upon the calculated measures.

28. Apparatus according to claim 22, wherein the texture map generator is operable to generate texture data for each respective texture map by combining image data from the images in proportions dependent upon the calculated measures.

29. Apparatus according to claim 19, wherein the three-dimensional computer model comprises a mesh of polygons, each polygon defining a part of the three-dimensional computer models and wherein the texture map generator is operable to generate each texture map as data defining a two-dimensional array of texels together with colour component values for the texels and positions within the array of the vertices of the polygons comprising the parts of the three-dimensional computer model for which the texture map stores texture data.

30. Apparatus according to claim 19, wherein the three-dimensional computer model comprises a plurality of unconnected points in three-dimensional space, each point defining a part of the three-dimensional computer model, and wherein the texture map generator is operable to generate each texture map as data defining colour component values for each point comprising a part of the three-dimensional computer model for which the texture map stores texture data.

31. Apparatus for generating texture data for use in rendering an image of a three-dimensional computer model, the apparatus comprising:

a data store configured to store data defining a three-dimensional computer model of the surface of an object, image data defining a plurality of different images of the object, and data defining the position and direction of each image and the three-dimensional computer model; and

a texture map generator operable to process the data to generate a plurality of texture maps, each texture map being associated with a respective view of the three-dimensional computer model and each texture map defining texture data for parts of the three-dimensional computer model that are visible in the corresponding view and at least some parts of the three-dimensional computer model that are not visible in the corresponding view.

32. Apparatus for rendering an image of a three-dimensional computer model comprising:

a data receiver operable to receive data defining a viewing position and direction from which an image is to be rendered;

a texture map selector operable to select a texture map for rendering the image from among a plurality of texture maps, wherein each texture map is associated with a camera viewing position and direction and defines texture data for parts of the three-dimensional computer model that are visible from the associated viewing position and direction and at least some parts of the three-dimensional computer model that are not visible from the associated viewing position and direction, and wherein the texture map selector is operable to select the texture map for rendering in dependence upon the received viewing direction and the viewing directions associated with the texture maps; and

an image renderer operable to render an image of the three-dimensional computer model from the received viewing position and direction using texture data from the selected texture map.

33. Apparatus for processing data defining a three-dimensional computer model of the surface of an object, image data defining a plurality of different images of the object, and data defining the position and direction of each image and the three-dimensional computer model, to generate texture data from the images for use in rendering an image of the three-dimensional computer model, the apparatus comprising:

means for generating a plurality of texture maps, each texture map being associated with a respective view of the three-dimensional computer model and each texture map defining texture data for parts of the three-dimensional computer model that are visible in the corresponding view and at least some parts of the three-dimensional computer model that are not visible in the corresponding view.

34. Apparatus for generating texture data for use in rendering an image of a three-dimensional computer model, the apparatus comprising:

means for storing data defining a three-dimensional computer model of the surface of an object, image data defining a plurality of different images of the object, and data defining the position and direction of each image and the three-dimensional computer model; and

means for processing the data to generate a plurality of texture maps, each texture map being associated with a respective view of the three-dimensional computer model and each texture map defining texture data for parts of the three-dimensional computer model that are visible in the corresponding view and at least some parts of the three-dimensional computer model that are not visible in the corresponding view.

35. Apparatus for rendering an image of a three-dimensional computer model comprising:

means for receiving data defining a viewing position and direction from which an image is to be rendered;

selecting means for selecting a texture map for rendering the image from among a plurality of texture maps, wherein each texture map is associated with a camera viewing position and direction and defines texture data for parts of the three-dimensional computer model that are visible from the associated viewing position and direction and at least some parts of the three-dimensional computer model that are not visible from the associated viewing position and direction, and wherein the selecting means is operable to select the texture map for rendering in dependence upon the received viewing direction and the viewing directions associated with the texture maps; and

means for rendering an image of the three-dimensional computer model from the received viewing position and direction using texture data from the selected texture map.