Method and apparatus for scanning three-dimensional objects

• US 7,098,435 B2
• Filed: 08/09/2002
• Issued: 08/29/2006
• Est. Priority Date: 10/25/1996
• Status: Expired due to Fees

• Current Assignee: Waveworks, Inc.
• Original Assignee: Frederick E. Mueller
• Inventors: Miramonti, John L., Mueller, Frederick E.
• Primary Examiner(s): LE, QUE TAN
• Application Number:
  US10/216,088
• Publication Number:
  US 20030066949A1
• Time in Patent Office: 1,481 Days
• Field of Search: 250/208.1, 250/559.4, 250/559.46, 250/223.B, 356/601, 356/430, 356/434, 356/237.1, 356/239.1, 356/239.7, 382/142, 382/154, 382/203
• US Class Current: 250/208.1
• CPC Class Codes: G01B 11/00 Measuring arrangements characterised by the use of optical techniques
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

Apparatus and method for creating 3D imagery of an object using calibration means to transfer data to a reference frame and visibility analysis to determine and resolve occlusion.

Claims

1. A method of forming a digital representation of a real object from a plurality of multiple images depicting a plurality of surface portions acquired at multiple viewing angles, the method comprising:

a) computing a first representation of the object, the first representation comprising a plurality sub representations, each sub-representation corresponding to a local surface region of the object and comprising;

i) a 3-D surface data component,

ii) multiple image data components including color,

iii) a multiple viewing angle data component,

b) computing a second representation of the object, the second representation comprising a plurality of view dependent sub-representations of local surface regions of the object, each sub-representation comprising;

i) a 3-D surface data component,

ii) multiple view dependent image data components,

iii) multiple local viewing angle data components,

c) said step of computing the second representation of the object comprising compressing image data components.

2. The method of claim 1 wherein said step of computing the second representation further comprises independently processing each sub-representation by removing redundant data from the first plurality of sub-representations and selecting non-redundant data to be included in the second plurality of sub-representation.

3. The method of claim 1 wherein said step of computing the second representation further comprises developing representations for surface attributes in addition to color.

4. The method of claim 3 wherein at least one representation includes at least one viewing angle parameter.

5. The method of claims 2 wherein the representation includes at least one parameter for representing the change in surface appearance for a plurality of points of the surface of the object, the change corresponding to a change in lighting conditions.

6. The method of claim 2 wherein the redundant data is image data.

7. The method of claim 2 wherein the redundant data is three dimensional data.

8. A method for scanning a three dimensional object, the method comprising:

a) developing a plurality of images of a three dimensional object to be scanned, said plurality of images being taken from a plurality of relative angles with respect to said object, said plurality of images depicting a plurality of surface portions of said object to be scanned; and

b) computing a representation of said plurality of surface portions of said object from said plurality of images, the representation comprising view dependent sub-representations of local surface regions of the object, each sub-representation comprising;

i) a 3-D surface data component,

ii) multiple image data components,

iii) multiple local viewing angle data components,

c) wherein said step of computing the representation further comprises compressing the image data.

9. A method for scanning a three-dimensional object according to claim 8 wherein the multiple image data components of each sub-representation are independently compressed.

10. The method of claim 8 further comprising developing representations for surface attributes in addition to color.

11. The method of claim 8 wherein at least one sub-representation includes interpolated data.

12. The method of claims 8 wherein the representation includes at least one parameter for representing the change in surface appearance for a plurality of points of the surface of the object, the change corresponding to a change in lighting conditions.

13. The method of claims 8 wherein a combination of compression techniques are applied to the same data.

14. The method of claims 8 wherein at least one compression technique is applied to three-dimensional data.

15. A method for scanning a three dimensional object, the method comprising:

a) developing a plurality of images of a three dimensional object to be scanned, said plurality of images being taken from a plurality of relative angles with respect to said object, said plurality of images depicting a plurality of surface portions of said object to be scanned; and

b) computing a first representation of said plurality of surface portions of said object from said plurality of images, the representation comprising

i) 3-D surface data,

ii) multiple image data including color data,

iii) multiple viewing angle data,

c) developing at least local representation of a local surface region of said object, the local representation comprising surface attributes in addition to color, from a sub portion of the first representation utilizing the

i) local 3-D surface data,

ii) local multiple image data, and

iii) local viewing angle data corresponding to the local surface region.

16. The method of claim 15 further comprising a plurality of local sub representations wherein the step of developing local representations further comprises independently processing a sub portion of the first representation.

18. The method of claim 15 wherein said developing further comprises compressing image data.

19. The method of claim 15 wherein the surface attributes includes specularity.

20. The method of claim 15 wherein at least one representation includes at least one viewing angle parameter.

21. The method of claim 15 wherein the representation includes at least one parameter for representing the change in surface appearance that corresponds to a change in lighting conditions.

22. The method of claim 15 wherein the surface attributes includes absorption.

17. The method 16 wherein said developing comprises removing redundant multiple image data from the first representation and selecting non-redundant multiple image data to be included in the sub representation.

23. A computer-readable medium having stored thereon a data structure representing an object, the data structure comprising:

a) a plurality of sub representations from multiple images, each sub-representation to correspond to a local surface region of the object and further comprising;

i) a first data for field for containing 3d surface data,

ii) a second data for field for containing compressed multiple image data, the compressed image data having been compressed by a combination of compression technique,

iii) a third data field for containing viewing angle data,

iv) wherein the first, second and third data fields of the sub-representation are combinable to form a view dependent representation of the object.

24. A computer-readable medium according to claim 23 wherein at least a portion of the image data to be contained in each sub-representation is to be independently compressable with respect to portions of image data to be stored in the other sub-representations.

25. A computer-readable medium according to claim 23 wherein at least one representation includes at least one viewing angle parameter.

26. A computer-readable medium according to claim 23 wherein at least one subrepresentation is to includes at least one parameter for representing the change in surface appearance that corresponds to a change in lighting conditions.

27. A computer-readable medium according to claim 23 wherein each of said plurality of sub-representation is to be comprised of:

a) 3d surface data for a plurality of local 3d surface points, the plurality of local surface points being part of an array of regularly spaced 3d surface points representing the surface of the object, and

b) a plurality of viewing angle sets comprising multiple relative viewing angles, each relative viewing angle set corresponding to a local 3d surface point in the array.

32. The computer-readable medium according to claim 23 wherein the digital representation is further comprised of a plurality of sub representations, each sub representation corresponding to a local surface region, each of the plurality sub representations comprising a plurality of sub components, the plurality sub components being combinable to describe the 3d surface of the local surface region and the view dependent image of a plurality of surface points of the local surface region, the image of the plurality of points changing from point to point and viewing angle to viewing angle.

28. A computer-readable medium having stored thereon a digital representation of a real object developed at least in part from color images of a plurality of surface portions of the real object, the representation comprising:

a) a first component containing 3d surface data,

b) a second component containing image data, the image data being comprised of selected non redundant multiple image data, the second component linked to the first component, the multiple image data describing the view dependent change in image a plurality of surface points

c) a third component containing viewing angle data, the third component being linked to the both the second component and to the first component,

wherein the components are integrated to form a representation describing the 3d surface of the plurality of surface portions of the object and the view dependent image of a plurality of surface points of the object, the image of the plurality of points changing from point to point and viewing angle to viewing angle.

29. The computer-readable medium according to claim 28 wherein the image data of the second component is further comprised of color data.

30. The computer-readable medium according to claim 28 wherein the image data is compressed by a plurality of compression techniques.

31. The computer-readable medium according to claim 28 further comprising a fourth component comprising surface non view dependent color data, the color values remaining constant from point to point and viewing angle to viewing angle of the local surface region

i) wherein the components are integrated to form a representation describing,

ii) the 3d surface of the plurality of surface portions the object,

iii) the view dependent image of at least a portion of the surface of the object and of a plurality of surface points of the object the image of the plurality of points changing from point to point and viewing angle to viewing angle, and

iv) the non view dependent color image of the plurality of surface portions of the object.

33. A computer-readable medium having stored thereon a digital representation of at least a portion of a real object developed at least in part from color images of the object the representation comprising:

3-D surface data and image information linked to the 3d surface data, at least a portion of the image information comprising color information and at least a portion of the image information comprising selected non-redundant image data describing the change in image for each of at least a portion of the plurality of points, the image changing from viewing angle to viewing angle over a plurality of angles.

34. The computer-readable medium according to claim 33 wherein at least one compression technique has been independently applied to at least a portion of the image data corresponding to at least one local surface region.

35. The computer-readable medium according to claim 33 wherein at least a portion of the image data corresponding at least one local surface region has been compressed by a combination of compression techniques.

36. The computer-readable medium according to claim 33 wherein at least a portion of the color data does not change from viewing angle to viewing angle.

37. The computer-readable medium according to claim 33 wherein at least a portion of the color data not compressed.

38. A computer-readable medium having stored thereon a digital representation of at least a portion of a real object developed at least in part from images of the object, the representation comprised of:

a plurality of sub-representations each sub representation corresponding to a local surface, each sub representation comprised of;

3-D surface data and image information linked to the 3d surface data developed at least in part from images of a real object, at least a portion the image information being stored in a compressed format, at least a portion the image information being color information.

39. The computer-readable medium according to claim 38 wherein at least one compression techniques has been independently applied to the image data corresponding each of at least a portion of the local surface regions.

40. The computer-readable medium according to claim 38 wherein each of at least a plurality of the sub representations each describe the change in image of a plurality surface points of the local surface region, the image changing viewing angle to viewing angle over a plurality of viewing angles.

41. The computer-readable medium according to claim 38 wherein at least a portion of the compressed multiple image information is comprised of selected non redundant image information, and at least a portion of the image information being comprised of color information.

42. The computer-readable medium according to claim 38 wherein at least a portion of the compressed multiple image has been compressed by a combination of compression techniques.

43. The computer-readable medium according to claim 38 wherein each of at least a plurality of the sub representations describe the change in image of a plurality surface points of the local surface region, the image changing from and from viewing angle to viewing angle over a plurality of viewing angles.

44. The computer-readable medium of claim 38 wherein the image information comprises:

a) non view dependent color image information, and

b) multiple image information describing at least the change in image corresponding to a change in viewing angle for a plurality of points of the surface of the object.

57. The method of claim 56 further comprising storing the digital representation on the computer readable medium of claim 38.

63. The method of claim 60 further comprising storing the digital representation on the computer readable medium of claim 38.

45. A computer-readable medium having stored thereon a digital representation of at least a portion of a real object developed at least in part from color images of the object, the representation comprising:

3-D surface data and image information linked to the 3d surface data at least a portion of the image information being stored in a compressed format the representation further comprising;

a) non view dependent color image information, and

b) multiple image information describing at least the change in image corresponding to a change in viewing angle for a plurality of points of the surface of the object.

46. A computer-readable medium having stored thereon a digital representation of at least a portion of a real object developed at least in part from color images of the object, the representation comprising:

3-D surface data and color image information linked to the 3d surface data,

a) the representation further comprising data describing surface attributes in addition to color, the surface attributes being stored in connection with the 3d surface data, the surface attributes including at least absorption.

47. A computer-readable medium having stored thereon a digital representation of at least a portion of an object, the representation comprising:

3-D surface data and image information linked to the 3d surface data, at least a portion of the image information comprising color image information and at least a portion of the image information being comprised of multiple image information describing the specular appearance of a plurality of surface points of the object, the multiple image information being stored in a compressed format.

48. The computer-readable medium according to claim 47 wherein the multiple image information describing a local surface region of the object has been compressed independently of the image data corresponding to other regions of the object.

50. The computer-readable medium according to claim 47 wherein the multiple image information describing a local surface region of the object has been compressed independently of the image data corresponding to other regions of the object.

49. A computer-readable medium having stored thereon a digital representation of at least a portion of an object, the representation comprising:

3-D surface data and image information linked to the 3d surface data at least a portion of the image information comprising color image information and at least a portion of the image information being comprised of multiple image information describing the absorption of light of a plurality of surface points of the object, the multiple image information being stored in a compressed format.

51. A computer-readable medium having stored thereon a digital representation of at least a portion of an object, the representation comprising:

3-D surface data and image information linked to the 3d surface data at least a portion of the image information comprising color image information and at least a portion of the image information being comprised of multiple image information describing the emissivity of a plurality of surface points of the object, the multiple image information being stored in a compressed format.

52. The computer-readable medium according to claim 51 wherein the multiple image information describing a local surface region of the object has been compressed independently of the image data corresponding to other regions of the object.

53. A method for scanning a three dimensional object comprising:

a) developing a plurality of images and corresponding viewing angle data describing a three dimensional object to be scanned, said plurality of images being taken from a plurality of relative angles with respect to said object, said plurality of images depicting a plurality of points on the surface of at least a portion of the object to be scanned; and

b) computing from said plurality of images and corresponding viewing angle data, a digital representation describing at least a portion of the object, the computing further comprising;

i) deriving 3d surface data,

ii) processing multiple image information to develop approximate non view dependent color image data describing the plurality of points,

iii) forming, from the multiple image information, image data describing the approximate change in image corresponding to the change in viewing angle for each of one or more of the plurality of points, for each point the change in image being relative to the approximate non view dependent color data describing the point, the forming comprising compressing the multiple image information,

iv) associating the viewing angle data, surface data, multiple image data and non view dependent color data to describe both the 3d shape and surface image of the at least a portion of the object from a plurality of viewing angles.

54. The method of claim 53 further comprising developing the non view dependent color image data of one or more points from a weighted average of the color of the point in different images, wherein the weighting factor varies with the viewing angle of the image.

55. The method of claim 53 wherein said step of compressing comprises selecting, from the multiple image information, non redundant image data, the non-redundant image data describing only the change in image of the one or more of the plurality of points corresponding to the change in viewing angle.

56. The method of claim 53 wherein said step of computing further comprises:

a) developing a representation comprising;

i) a first component containing 3-D surface data,

ii) a second component containing an approximate non-view dependent color data describing the plurality of points, and

iii) multiple image data describing the change in image corresponding to the change in viewing angle of at least a portion of the plurality of points on the surface of the object, the change in image being relative to the approximate non-view dependent color data describing the plurality of points, and the multiple image data comprised of compressed image data,

b) linking the multiple viewing angle data, 3d surface data and image data for storage on a computer readable medium.

58. The method of claim 56 wherein said step of developing further comprises forming a representation comprised of sub representations, each sub representation corresponding to a local surface region of the object.

59. The method of claim 58 wherein each of the sub representations are comprised of a plurality of components that are combinable to describe both the 3d shape and the apparent surface image of the at least a portion of the object from a plurality of viewing angles.

60. A method of forming a digital representation of an real object from a plurality of multiple images depicting a plurality of surface portions acquired at multiple viewing angles, the method comprising:

a) developing a plurality of images of a three dimensional object to be scanned, said plurality of images being taken from a plurality of relative angles with respect to said object said plurality of images depicting a plurality of points on the surface of at least a portion of the object to be scanned;

b) computing from the at least the plurality of images a digital representations, the representation comprising 3-D surface data and image information linked to the 3d surface data describing the at least a portion of the object, at least a portion of the image information being color information and at least a portion of the image information being compressed image information, and

c) said step of computing comprises transferring multiple images over a network and using processors that are distributed over a network.

61. The method of claim 60 wherein the network is the internet.

62. The method of claim 60 wherein further comprising;

a) developing viewing angle information.

b) developing multiple image information, the multiple image information representing the images of the plurality of points from multiple viewing angles,

c) linking the multiple image information and viewing angle information to create a view dependent representation describing the color and change in image for at least a portion of the plurality of points, the image changing from point to point and from viewing angle to viewing angle.

64. The method of claim 60 wherein the change in image includes a change in intensity.

65. The method of claim 60 wherein the change in image includes a change in color.

66. The method of claim 60 wherein developing includes automatically controlling at least a portion of the acquisition process.

67. The method of claim 60 further comprising providing a communications interface and using the communications interface for computing the representation.

68. The method of claim 60 wherein the acquisition process is monitored over the network.

69. The method of claim 60 wherein the step of computing is monitored over the network.

70. The method of claim 60 wherein the multiple image data is comprised of selected non redundant multiple image data.

71. The method of claim 60 wherein image data is comprised image data having been compressed by a plurality of compression techniques.

72. A method of forming a digital representation of an real object from a plurality of multiple images depicting a plurality of surface portions acquired at multiple viewing angles, the method comprising:

a) developing a plurality of color images of a three dimensional object to be scanned, said plurality of images being taken from a plurality of relative angles with respect to said object, said plurality of images depicting a plurality of points of the surface of at least a portion of the object;

b) creating a representation comprised of a plurality of sub representations each sub representation describing a local surface region of the object, each sub representation corresponding to a local surface region of the object, each sub representation comprising 3d surface data and image information linked to the 3d surface data, at least a portion of the image information comprising compressed image information, at least a portion of the image information comprising color information, wherein said step of creating includes compressing at least a portion of the image information using a combination of compression techniques.

73. The method of claim 72 wherein at least one of the combination of compression techniques is independently applied the image data corresponding a local surface region of the object.

74. The method of claim 72 wherein the step of creating includes developing viewing angle information and linking the multiple image information, viewing angle information and 3d surface data to describe the change in image of a plurality of points of the local surface corresponding to each of at least a portion of the sub representations.

75. The method of claim 72 wherein the step of creating includes deriving surface attributes in addition to color and storing the surface attributes in connection with the 3d surface data.

76. The method of claim 74 wherein the data describing in image is comprised of data describing the change in intensity for a plurality of surface points.

77. The method of claim 74 wherein the data describing in image is comprised of data describing the change in color for a plurality of surface points.

78. The method of claim 74 wherein at least portion of the creating is performed using a network.

79. The method of claim 74 wherein the at least portion of the developing is automatically performed.

80. The method of claim 72 wherein at least portion of the developing is performed using a network.

81. The method of claim 80 wherein the network is the internet.

82. A method of forming a digital representation of an real object from a plurality of multiple images depicting a plurality of surface portions acquired at multiple viewing angles, the method comprising:

a) developing a plurality of images of a three dimensional object to be scanned, said plurality of images being taken from a plurality of relative angles with respect to said object said plurality of images depicting a plurality of points on the surface of the object to be scanned;

b) creating a digital representation from at least the plurality of images, the representation comprised of 3-D surface data and image information linked to the 3d surface data, at least a portion of the representation comprising compressed image information, wherein said step of creating includes compressing the image information using a combination of compression techniques.

83. The method of claim 82 wherein the step of creating includes developing viewing angle information and linking the multiple image information and viewing angle information to create a view dependent representation describing the color and specular appearance for at least a portion of the plurality of points.

84. The method of claim 82 wherein the step of creating includes deriving surface attributes in addition to color and storing the surface attributes in connection with the 3d surface data.

85. The method of claim 82 wherein said step of creating includes processing the images using processors that are distributed over a network.

86. The method of claim 82 wherein the network is the internet.

87. A Computer readable medium having stored thereon instructions for accessing and processing data to be included within a digital representation of a real object,

a) wherein the digital representation of a real object is to comprise;

i) a first component comprising compressed color image data, the color image data having been developed at least in part from portions of color images of a plurality of surface portions of a real object,

ii) a second component comprising data describing at least the change in image corresponding to a change in viewing angle for a plurality of points of the surface of the object,

iii) a third component comprising 3d surface data having been developed at from one or more portions of images of the object,

iv) wherein the representation to be accessed will be further comprised of 3d surface data linked with the image data and wherein the instructions are operative to form a view dependent description of the view dependent image and of a plurality of surface points on at least portions of the object,

b) wherein the instructions are further operative to decompress at least a portion of the image data to be contained within the digital representation.

88. The computer readable medium according to claim 87 wherein the second component is for storing information to be developed from images of the object and is to be comprised of at least one of data describing specularity, compressed multiple image information, selected non-redundant image information and multiple image information compressed by a plurality of compression techniques.

89. The computer readable medium according to claim 87 wherein the representation is to be further comprised of a plurality of sub-representations, each sub-representation corresponding to a local surface, each sub representation to have sub-portions of each of the first, second and third components, wherein the sub-portions are to be combinable, to describe the change in image corresponding to a change in viewing angle for a plurality of points of the local surface wherein at least a portion of the image data corresponding to each subrepresentation is to be independently decompressable with respect to the image data corresponding to the other subrepresentations.

90. A method for accessing and decompressing data included within a digital representation of a real object, the method comprising the steps of:

a) accessing a digital representation of a real object that comprises;

i) a first component comprising compressed color image data, the color image data having been developed at least in part from one or more portions of images of the real object,

ii) a second component comprising information describing at least the change in image corresponding to a change in viewing angle for a plurality of points of the surface of the object,

iii) a third component comprising 3d surface data having been developed at least in part from at least portion of the images taken of a real object,

iv) wherein the first, second and third components are combinable for forming a view-dependent representation comprised of 3d surface data and linked image information describing the view dependent image of a plurality of surface points of portions the object,

b) decompressing at least a portion of the image data to be stored within the structure.

91. The method according to claim 90 further comprising the steps of transmitting a request for at least a portion of the representation over the a network and receiving at least a portion of the representation.

92. The method according to claim 90 wherein the network is the internet.

93. The method according to claim 90 further wherein the second component includes at least one of data describing specularity, compressed multiple image information, selected non-redundant image information and multiple image information compressed by a plurality of compression techniques.

94. The method according to claim 90 wherein the representation is further comprised of a plurality of sub-representations corresponding to a local surface, each sub representation having a first, second and third data component describe the change in image corresponding to a change in viewing angle for a plurality of points of the local surface wherein at least a portion of the image data corresponding to each subrepresentation is independently expandable with respect to the image data corresponding to other local surfaces.

95. A method for accessing and processing image based data included within a digital representation of a real object, the method comprising the steps of:

a) accessing a digital representation of a real object, the representation comprising;

i) a first component comprising compressed color image data developed in part from at least portions of images of the real object,

ii) a second component comprising 3d surface data developed at least in part from the images of the object,

iii) wherein the 3d shape and color image data of first and second components are linked to be combinable for forming a representation describing of the object,

b) decompressing at least a portion of the image data stored within the structure.

96. A method according to claim 95 further comprising transmitting a request over the internet for at least a portion of a digital representation of the real object.

97. A method according to claim 95 wherein the network is the internet.

99. The method according to claim 95 wherein the representation is further comprised of a plurality of sub-representations corresponding to a local surface, each sub representation having a first, second and third data component to describe the change in image corresponding to a change in viewing angle for a plurality of points of the local surface and wherein at least a portion of the image data corresponding to each subrepresentation is independently expandable with respect to the image data corresponding to other local surface.

98. A computer-readable medium having stored thereon a data structure for storing a digital representation of a real object, the data structure to comprise:

a) a first component for storing compressed color image data, the data to have been developed in part from at least portions of images of the real object,

b) a second component for storing 3d surface data, the data to have been developed at least in part from the images of the object,

c) a third component for storing data to describe at least the change in image corresponding to a change in viewing angle for a plurality of points of the surface of the object the data further to be comprised of at least one of data describing specularity, compressed multiple image information, selected non-redundant image information and multiple image information compressed by a plurality of compression techniques,

d) wherein color image data and the 3d surface data of first and second components are to be linked.

100. A method for accessing and expanding image based data included within a digital representation of a real object, the method comprising the steps of:

transmitting a request over the internet for at least a portion of a digital representation of the real object, the representation is organized in a data structure that comprises;

i) a first component for storing compressed color image data developed from color images of the object,

ii) a second component for storing 3d surface data developed at least in part from images of the object,

iii) wherein the first and second components are linked so as to be combinable to represent the real object,

b) decompressing at least a portion of the image data to be stored within the structure.

101. The method according to claim 100 further comprising the steps of transmitting a request for at least a portion of the representation over a network and receiving at least a portion of the representation from the network.

102. The method according to claim 101 wherein the the data structure includes at least one more component for storing information developed from images of the object comprised of at least one data describing specularity, compressed multiple image information, selected non-redundant image information and multiple image information compressed by a plurality of compression techniques.

103. The method according to claim 102 wherein the representation is further comprised of:

a plurality of sub-representations corresponding to a local surface, each sub representation having a first, second and third data component, that are combinable for describing the change in image corresponding to a change in viewing angle for a plurality of points of the local surface wherein at least a portion of the image data to be stored in each subrepresentation is independently expandable with respect to the image data corresponding to other local surfaces.

116. The method according to claim 100 wherein the representation is to be further comprised of:

a plurality of sub-representations, each sub-representation corresponding to a local surface and for storing at least in part local color image data to have been selected using viewing angle data corresponding to each portion of the color image.

117. The method of claim 116 wherein the color image data to be stored is to have been selected from a weighted average of the color of at least a portion of the images, wherein the weighting factor varies with the viewing angle data.

118. The method according to claim 100 wherein the data structure includes at least one more component is for storing information describing at least one of the group consisting of absorption, emissivity and specularity.

119. The method of claim 100 wherein at least a portion the three-dimensional data is to be stored in a compressed format.

104. A computer readable medium having stored thereon a data structure for forming a digital representation of an object, the data structure comprising:

a) a plurality of sub-representations, each sub-representation to correspond to a local surface region of the object and to describe the 3d shape and color image for a plurality of points of the surface of the local surface, and further comprising;

i) a first component for storing compressed color image information,

ii) a second component for storing 3d surface data,

iii) a third component for storing information describing at least the change in surface image corresponding to a change in viewing angle for a plurality of points of the surface of the object, wherein at least a portion of the image data to be contained in each sub-representation is to be independently decompressable with respect to portions of image data to be stored in the other sub-representations.

105. A computer-readable medium according to claim 104 wherein the digital representation includes information describing at least the change in image corresponding to a change in viewing angle for a plurality of points of the surface of the object, in which one or more of the data components comprises at least one of compressed multiple image information, selected non-redundant multiple image information and multiple image information compressed by a plurality of compression techniques.

106. A computer-readable medium according to claim 104 wherein the representation is further comprised of a component for storing data describing a change in image corresponding to a change in lighting conditions.

107. A computer-readable medium according to claim 104 further comprising instructions to access and decompress at least a portion of the multiple image information.

108. A computer-readable medium according to claim 104 further comprising instructions to compress at least a portion of the multiple image information.

120. A computer readable medium according to claim 104 further comprising transmitting a request for a sub-representation over a network and receiving the sub-representation wherein the data structure contains data developed at least in part from images.

121. A computer-readable medium according to claim 104 wherein for at least a portion of the data to be stored in a plurality of the components the compression format is different from component to component.

122. A computer-readable medium according to claim 104 wherein at least a portion of data to be stored is compressed 3d data.

123. A computer-readable medium according to claim 122 wherein the data structure is comprised of a fourth component for describing emissivity.

124. A computer-readable medium according to claim 104 wherein the data to be stored in the third component is to be for describing specularity.

125. A computer-readable medium according to claim 104 wherein the data to be stored in the third component is for describing at least one of emissivity and absorption.

126. A computer-readable medium according to claim 104 wherein the data structure is comprised of a fourth component for describing emissivity.

127. A computer-readable medium according to claim 104 wherein the data structure is comprised of a fourth component for describing absorption.

128. A computer-readable medium according to claim 104 wherein the data structure is comprised of a component for storing compressed data describing at least one of specularity, absorption and emissivity.

129. A computer-readable medium according to claim 122 wherein at least a portion of data is to be compressed by at least a second compression technique after having been compressed by a first compression technique.

130. The computer-readable medium according to claim 104 wherein at least a portion of the 3d data to be stored is to be comprised of a plurality of 3d surface points, the surface points being part of an array of spatially regularized 3d surface points.

133. A computer-readable medium according to claim 104 wherein the data structure is for storing data describing at least one of emissivity, absorption and specularity.

134. A computer-readable medium according to claim 133 wherein at least a portion of data to be stored is compressed 3d data.

135. A computer-readable medium according to claim 134 wherein for at least a portion of the data to be stored in a plurality of the components the compression format is to differ from component to component.

136. A computer-readable medium according to claim 124 wherein for at least a portion of the data to be stored in a plurality of the components the compression format is to differ from component to component.

137. A computer-readable medium according to claim 125 wherein at least a portion of data to be stored is compressed 3d data.

138. A computer-readable medium according to claim 124 wherein at least a portion of data is to be compressed by at least a second compression technique after having been compressed by a first compression technique.

139. A computer-readable medium according to claim 124 wherein at least a portion of data to be stored is compressed 3d data.

140. The computer-readable medium of claim 124 further comprising instructions to perform one of compressing and decompressing at least a portion of the data to be stored.

141. The computer-readable medium according to claim 139 wherein at least a portion of the 3d data to be stored is to be comprised of a plurality of 3d surface points, the surface points being part of an array of spatially regularized 3d surface points.

142. The computer-readable medium according to claim 104 wherein at least a portion of the 3d data to be stored is to be comprised of a plurality of 3d surface points, the surface points being part of an array of spatially regularized 3d surface points.

144. The computer-readable medium according to claim 142 wherein a component of the data structure is for storing a plurality of viewing angle sets that comprise multiple relative viewing angles, each viewing angle set describing a plurality of viewing angles with respect to a local 3d surface point in the array.

109. A computer readable medium having stored thereon a data structure for forming a digital representation of an object, the data structure comprising:

a) a first component for storing color image information,

b) a second component for storing 3d surface data,

c) a third component for storing compressed multi image information describing at least the change in image corresponding to a change in viewing angle for a plurality of points of the surface of the object,

d) wherein the image information is to be linked to the 3d surface data and wherein the first, second and third components are combinable for forming a view-dependent representation describing at least a view dependent image and of the plurality of surface points.

110. The computer-readable medium according to claim 109 wherein the representation is comprised of a plurality of sub-representations, each of the sub-representations corresponding to a local surface region and having a first, second and third component.

111. The computer-readable medium according to claim 110 wherein one or more of said sub-representations have:

a) a data field for containing a plurality of local 3d surface points, the plurality of local surface points being part of an may of spatially regularized 3d surface points representing the surface of the object, and

b) a data field for containing a plurality of viewing angle sets comprising multiple relative viewing angles, each relative viewing angle set corresponding to a describing a plurality of viewing angles with respect to a local 3d surface point in the array.

112. A computer readable medium according to claim 110 wherein at least a portion of the image data for each plurality of sub-representations is to be independently compressible with respect to the image data representing other surface regions of the object.

113. A computer-readable medium according to claim 109 wherein the representation is further comprised of a component for storing data describing a change in image corresponding to a change in lighting conditions.

114. A computer-readable medium according to claim 109 further comprising instructions to access and decompress at least a portion of the multiple image information.

115. A computer-readable medium according to claim 109 further comprising instructions to compress at least a portion of the multiple image information.

131. The computer-readable medium according to claim 109 wherein at least a portion of the 3d data to be stored is to be comprised of a plurality of 3d surface points, the surface points being part of an array of spatially regularized 3d surface points.

132. The computer-readable medium according to claim 131 wherein the data structure is for storing for storing a plurality of viewing angle sets comprising multiple relative viewing angles, each viewing angle set describing a plurality of viewing angles with respect to a local 3d surface point in the array.

143. The computer-readable medium according to claim 109 wherein at least a portion of the 3d data to be stored is to be comprised of a plurality of 3d surface points, the surface points being part of an array of spatially regularized 3d surface points.

145. A method of requesting and receiving a digital representation of an object, the method comprising;

a) transmitting a request for at least a portion of a digital representation over a network, wherein the representation is stored in a data structure that comprises;

i) a plurality of sub-representations, each sub-representation corresponding to a local surface region and describing the 3d shape and color image for a plurality of points of the local surface, and further comprising;

(1) a first component storing compressed color image information,

(2) a second component storing 3d surface data,

(3) a third component storing information describing at least the change in image corresponding to a change in viewing angle for the plurality of points of the surface of the object, wherein the image data of the first, second and third components of each sub-representation are independently decompressable with respect to portions of image data to be stored in other sub-representations.

b) receiving the representation from the network.

146. The method of requesting and receiving a digital representation of an object according to claim 145 wherein the data structure is comprised of a fourth component for containing data describing at least one of emissivity and absorption.

147. The method of requesting and receiving a digital representation of an object according to claim 145 further comprising the step of decompressing the data received from the network as part of the representation.

148. The method of requesting and receiving a digital representation of an object according to claim 145 wherein the network is the internet.

149. The method of requesting and receiving a digital representation of an object according to claim 148 wherein the data to be stored in the third component is to be for describing specularity.

150. The method of requesting and receiving a digital representation of an object according to claim 148 wherein the data to be stored in the third component is for describing at least one of emissivity and absorption.

151. The method of requesting and receiving a digital representation of an object according to claim 148 wherein at least a portion of the data is for describing at least one of emissivity, absorption and specularity.

152. A computer-readable medium according to claim 145 wherein for at least a portion of the data to be stored in at least one of the first, second and third the components the compression format is to differ from component to component.

153. The method of requesting and receiving a digital representation of an object according to claim 148 wherein at least a portion of data to be stored is compressed 3d data.

154. The method of requesting and receiving a digital representation of an object according to claim 153 wherein at least a portion of data is to be compressed by at least a second compression technique after having been compressed by a first compression technique.