Method and apparatus for controlling distortion of a projected image

• US 5,329,310 A
• Filed: 06/30/1992
• Issued: 07/12/1994
• Est. Priority Date: 06/30/1992
• Status: Expired due to Term

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: The Walt Disney Company
• Inventors: Redmann, William G., Watson, Scott F., Liljegren, Gordon E.
• Primary Examiner(s): Groody, James J.
• Assistant Examiner(s): Burgess, Glenton B.
• Application Number:
  US07/907,108
• Time in Patent Office: 742 Days
• Field of Search: 358/108, 358/104, 358/87, 358/209, 358/93, 382/44, 359/718, 359/728, 359/364, 359/363, 395/137-139, 434/43, 434/44, 434/40
• US Class Current: 348/147
• CPC Class Codes: G09B 9/326 the image being transformed by optical means
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 17/04 for receivers
• CPC Class Codes: H04N 9/3185 Geometric adjustment, e.g. keystone or convergence optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14
• CPC Class Codes: H04N 9/3194 including sensor feedback

Abstract

A method and apparatus for displaying an image that is subject to multiple distortions substantially without distortion is provided. The image is recorded from a first viewpoint and subject to a first distortion and played back from a second viewpoint and subject to a second distortion. The display surface may also be distorted (e.g., curved). By mapping back through the projector to the screen and through the camera (at the viewpoint), the image is transformed so that it can be viewed from the viewpoint with little or no distortion.

Claims

1. A method of displaying an image of a subject on a display surface having a display surface shape and a display surface position, for observation from a viewpoint having a viewpoint position at which said display surface subtends a first field of view, said displayed image having a controlled amount of distortion, said method comprising the steps of:

recording source image visual data for said image by transforming source visual data of said subject through a first distorting transformation process with respect to a recording point at which said subject subtends a second field of view which is at least partially contained within said first field of view at said recording point;

transforming said recorded source image visual data through an intermediate transformation process to produce target image visual data; and

playing back said target image visual data from a playback point having a playback point position at which said display surface subtends a third field of view and such that said subject image subtends at said viewpoint a fourth field of view at least partially contained within said first field of view of said viewpoint, through a second distorting transformation process;

wherein;

said first and second distorting transformation processes, and said intermediate transformation process are selected so that said second and fourth fields of view are substantially identical, thereby causing said controlled amount of distortion.

2. The method of claim 1 wherein said recording step comprises generating digital image visual data.

3. The method of claim 1 wherein said recording step comprises capturing live image visual data.

4. The method of claim 3 wherein said capturing step comprises capturing said live image visual data on a photographic medium.

5. The method of claim 3 wherein said capturing step comprises capturing said live image visual data in electronic form.

6. The method of claim 5 wherein said capturing step comprises capturing said live image visual data in a video format.

7. The method of claim 3 wherein said capturing step comprises capturing said image as a number of captured image segments, each of said captured image segments being captured from said recording point, said captured image segments together subtending said second field of view at said recording point.

8. The method of claim 1 wherein said playing back step comprises playing said recorded image visual data back on a nonplanar display surface, said nonplanar display surface contributing also to said controlled amount of distortion.

9. The method of claim 1 wherein said controlled amount of distortion is the absence of distortion.

10. The method of claim 1 wherein:

said intermediate transformation process is a digital transformation process;

said method further comprising the step of;

digitizing said source image visual data for use in said digital transformation process.

11. The method of claim 10 wherein said digital transformation process comprises, for each target pixel of said target image visual data:

determining first from said second distorting transformation process, said playback point position, said display surface position and said display surface shape, a location on said display surface onto which said target pixel would be projected;

determining second from said first distorting transformation process, said viewpoint position, said display surface position and said display surface shape, a source pixel in said source image visual data which corresponds to said location on said display surface; and

filling said target pixel with image data from said source pixel.

12. The method of claim 11 wherein:

at least one of said first and second determining steps is performed for each target pixel in said target image visual data before said filling step is performed for any target pixel;

results of at least one of said first and second determining steps for each target pixel are used to produce a table of correspondence between source and target pixels; and

said filling step comprises, for each target pixel, looking up a corresponding source pixel in said table.

13. The method of claim 1 wherein said playing back step comprises playing back said target image visual data in electronic form.

14. The method of claim 13 wherein said playing back step comprises projecting said target image visual data onto said display surface in a video format.

15. The method of claim 14 wherein said viewpoint is on a first side of said display surface;

and

said projecting step comprising projecting said target image visual data onto said first side of said display surface.

16. The method of claim 14 wherein:

said viewpoint is on a first side of said display surface; and

said projecting step comprises projecting said target image visual data onto a second side of said display surface opposite said first side.

17. The method of claim 1 wherein said playing back step comprises playing back said target image visual data photographically.

18. The method of claim 17 wherein said playing back step comprises projecting said target image visual data onto said display surface photographically.

19. The method of claim 18 wherein:

said viewpoint is on a first side of said display surface; and

said projecting step comprises projecting said target image visual data onto said first side of said display surface.

20. The method of claim 18 wherein:

said viewpoint is on a first side of said display surface; and

said projecting step comprises projecting said target image visual data onto a second side of said display surface opposite said first side.

21. The method of claim 1 wherein said image is displayed on a number of display surfaces which together subtend said first field of view at said viewpoint.

22. The method of claim 21 wherein said playing back step comprises playing back said target image visual data as a number of playback image segments which together subtend said third field of view at said playback point.

23. The method of claim 22 wherein said number of display surfaces is equal to said number of playback image segments.

24. The method of claim 22 wherein said number of display surfaces is different from said number of playback image segments.

25. The method of claim 1 wherein said playing back step comprises playing back said target image visual data as a number of playback image segments which together subtend said third field of view at said playback point.

26. The method of claim 25 wherein said recording step comprises capturing said image as a number of captured image segments, each of said captured image segments being captured from said recording point, said captured image segments together subtending said second field of view at said recording point.

27. The method of claim 26 wherein said number of captured image segments is equal to said number of playback image segments.

28. The method of claim 26 wherein said number of captured image segments is different from said number of playback image segments.

29. The method of claim 1 wherein said recording step comprises capturing said image as a number of captured image segments, each of said captured image segments being captured from said recording point, said captured image segments together subtending said second field of view at said recording point.

30. The method of claim 29 wherein said image is displayed on a number of display surfaces which together subtend said first field of view at said viewpoint.

31. The method of claim 30 wherein said number of captured image segments is equal to said number of display surfaces.

32. The method of claim 30 wherein said number of captured image segments is different from said number of display surfaces.

33. The method of claim 30 wherein said playing back step comprises playing back said target image visual data as a number of playback image segments which together subtend said third field of view at said playback point.

34. The method of claim 33 wherein said number of captured image segments, said number of playback image segments, and said number of display surfaces are all equal.

35. The method of claim 33 wherein said number of captured image segments, said number of playback image segments, and said number of display surfaces are unequal.

36. The method of claim 35 wherein any two of said number of captured image segments, said number of playback image segments, and said number of display surfaces are equal.

37. Apparatus for displaying an image of a subject on a display surface having a display surface shape and a display surface position, for observation from a viewpoint having a viewpoint position at which said display surface subtends a first field of view, said displayed image having a controlled amount of distortion, said apparatus comprising:

means for recording source image visual data for said image including means for transforming source visual data of said subject through a first distorting transformation process with respect to a recording point at which said subject subtends a second field of view which is at least partially contained within said first field of view at said recording point;

intermediate transforming means for transforming said recorded source image visual data through an intermediate transformation process to produce target image visual data; and

means for playing back said target image visual data from a playback point having a playback point position at which said display surface subtends a third field of view and such that said subject image subtends at said viewpoint a fourth field of view at least partially contained within said first field of view at said viewpoint, through a second distorting transformation process;

wherein;

said first and second distorting transformation processes, and said intermediate transformation process are selected so that said second and fourth fields of view are substantially identical, thereby causing said controlled amount of distortion.

38. The apparatus of claim 37 wherein said recording means comprises means for generating digital image visual data.

39. The apparatus of claim 37 wherein said recording means comprises means for capturing live image visual data.

40. The apparatus of claim 39 wherein said capturing means captures said live image visual data on a photographic medium.

41. The apparatus of claim 39 wherein said capturing means captures said live image visual data in electronic form.

42. The apparatus of claim 41 wherein said capturing means captures said live image visual data in a video format.

43. The apparatus of claim 39 wherein said capturing means captures said image as a number of captured image segments, each of said captured image segments being captured from said recording point, said captured image segments together subtending said second field of view at said recording point.

44. The apparatus of claim 37 wherein said playing back means plays said recorded image visual data back on a nonplanar display surface, said nonplanar display surface contributing also to said controlled amount of distortion.

45. The apparatus of claim 37 wherein said controlled amount of distortion is the absence of distortion.

46. The apparatus of claim 37 wherein:

said intermediate transforming means comprises digital transforming means;

said apparatus further comprising;

means for digitizing said source image visual data for use in said digital transforming means.

47. The apparatus of claim 46 wherein said digital transforming means comprises, for each target pixel of said target image visual data:

first determining means for determining from said second distorting transformation process, said playback point position, said display surface position and said display surface shape, a location on said display surface onto which said target pixel would be projected;

second determining means for determining from said first distorting transformation process, said viewpoint position, said display surface position and said display surface shape, a source pixel in said source image visual data which corresponds to said location on said display surface; and

means for filling said target pixel with image data from said source pixel.

48. The apparatus of claim 47 wherein:

at least one of said first and second determining means operates on each target pixel in said target image visual data before said filling means operates for any target pixel;

output of at least one of said first and second determining means for each target pixel is used to produce a table of correspondence between source and target pixels; and

for each target pixel, said filling means look up a corresponding source pixel in said table.

49. The apparatus of claim 37 wherein said playing back means plays back said target image visual data in electronic form.

50. The apparatus of claim 49 wherein said playing back means projects said target image visual data onto said display surface in a video format.

51. The apparatus of claim 50 wherein said viewpoint is on a first side of said display surface;

and

said projecting means projects said target image visual data onto said first side of said display surface.

52. The apparatus of claim 50 wherein said viewpoint is on a first side of said display surface;

and

said projecting means projects said target image visual data onto a second side of said display surface opposite said first side.

53. The apparatus of claim 37 wherein said playing back means plays back said target image visual data photographically.

54. The apparatus of claim 53 wherein said playing back means projects said target image visual data onto said display surface photographically.

55. The apparatus of claim 54 wherein said viewpoint is on a first side of said display surface;

and

said projecting means projects said target image visual data onto said first side of said display surface.

56. The apparatus of claim 54 wherein said viewpoint is on a first side of said display surface;

and

said projecting means projects said target image visual data onto a second side of said display surface opposite said first side.

57. The apparatus of claim 37 wherein said image is displayed on a number of display surfaces which together subtend said first field of view at said viewpoint.

58. The apparatus of claim 57 wherein said playing back means plays back said target image visual data as a number of playback image segments which together subtend said third field of view at said playback point.

59. The apparatus of claim 58 wherein said number of display surfaces is equal to said number of playback image segments.

60. The apparatus of claim 58 wherein said number of display surfaces is different from said number of playback image segments.

61. The apparatus of claim 37 wherein said playing back means plays back said target image visual data as a number of playback image segments which together subtend said third field of view at said playback point.

62. The apparatus of claim 61 wherein said recording means captures said image as a number of captured image segments, each of said captured image segments being captured from said recording point, said captured image segments together subtending said second field of view at said recording point.

63. The apparatus of claim 62 wherein said number of captured image segments is equal to said number of playback image segments.

64. The apparatus of claim 62 wherein said number of captured image segments is different from said number of playback image segments.

65. The apparatus of claim 37 wherein said recording means captures said image as a number of captured image segments, each of said captured image segments being captured from said recording point, said captured image segments together subtending said second field of view at said recording point.

66. The apparatus of claim 65 wherein said image is displayed on a number of display surfaces which together subtend said first field of view at said viewpoint.

67. The apparatus of claim 66 wherein said number of captured image segments is equal to said number of display surfaces.

68. The apparatus of claim 66 wherein said number of captured image segments is different from said number of display surfaces.

69. The apparatus of claim 66 wherein said playing back means plays back said target image visual data as a number of playback image segments which together subtend said third field of view at said playback point.

70. The apparatus of claim 69 wherein said number of captured image segments, said number of playback image segments, and said number of display surfaces are all equal.

71. The apparatus of claim 69 wherein said number of captured image segments, said number of playback image segments, and said number of display surfaces are unequal.

72. The apparatus of claim 71 wherein any two of said number of captured image segments, said number of playback image segments, and said number of display surfaces are equal.

73. A method of serially displaying a plurality of images of a subject on a display surface having for each image a respective display surface shape and a respective display surface position, for observation from a corresponding plurality of viewpoints, one for each of said plurality of images, each of said viewpoints having a respective viewpoint position at which said display surface subtends a respective first field of view for its respective image, each of said displayed images having a controlled amount of distortion, said method comprising the steps of:

serially recording source image visual data for each of said plurality of images with respect to a respective recording point at which said subject subtends at said respective viewpoint a respective second field of view which is at least partially contained within said respective first field of view at said respective recording point;

for each of said plurality of images, transforming said respective recorded image visual data through a respective intermediate transformation process to produce respective target image visual data; and

serially playing back and respective target image visual data from respective playback points each having a respective playback point position at which said display surface subtends a respective third field of view and such that said subject image subtends at said respective viewpoint a respective fourth field of view at least partially contained within said respective first field of view at said respective viewpoint, through a respective playback distorting transformation process;

wherein;

for each of said plurality of images, said respective intermediate transformation process, said respective playback distorting transformation process, and said respective first, second, third and fourth fields of view combine to contribute to said controlled amount of distortion.

74. The method of claim 73 wherein said series of respective playback points in said serial playing back step comprises a single playback point used for all said plurality of images.

75. The method of claim 73 wherein said respective display surface shapes and positions comprise a single display surface shape and a single display surface position used for all of said plurality of images.

76. The method of claim 73 wherein said controlled amount of distortion is the absence of distortion.

77. The method of claim 73 wherein said playing back step comprises playing said target image visual data back on a nonplanar display surface, said nonplanar display surface contributing also to said controlled amount of distortion.

78. The method of claim 73 wherein said recording step comprises generating digital image visual data.

79. The method of claim 73 wherein said recording step comprises capturing live image visual data.

80. The method of claim 79 wherein said capturing step comprises capturing said live image visual data on a photographic medium.

81. The method of claim 79 wherein said capturing step comprises capturing said live image visual data in electronic form.

82. The method of claim 81 wherein said capturing step comprises capturing said live image visual data in a video format.

83. The method of claim 79 wherein said capturing step comprises capturing said respective image as a number of captured image segments, each of said captured image segments being captured from said respective recording point, said captured image segments together subtending said respective second field of view at said respective recording point.

84. The method of claim 73 wherein:

said respective intermediate transformation process is a respective digital transformation process;

said method further comprising the step of;

digitizing said source image visual data for use in said respective digital transformation process.

85. The method of claim 73 wherein said serial recording step comprises transforming, for each respective one of said images, respective source visual data of said subject through a respective recording distorting transformation process, said respective recording distorting transformation process contributing also to said controlled amount of distortion.

86. The method of claim 85 wherein:

said respective intermediate transformation process is a respective digital transformation process;

said method further comprising the step of;

digitizing said source image visual data for use in said respective digital transformation process.

87. The method of claim 86 wherein said respective digital transformation process comprises, for each target pixel of said respective target image visual data:

determining first from said respective playback distorting transformation process, said respective playback point position, said respective display surface position and said respective display surface shape, a location on said display surface onto which said target pixel would be projected;

determining second from said respective recording distorting transformation process, said respective viewpoint position, said respective display surface position and said respective display surface shape, a source pixel in said respective source image visual data which corresponds to said location on said display surface; and

filling said respective target pixel with image data from said source pixel.

88. The method of claim 87 wherein:

at least one of said first and second determining steps is performed for each respective target pixel in said respective target image visual data before said filling step is performed for any respective target pixel;

results of at least one of said first and second determining steps for each respective target pixel are used to produce a table of correspondence between source and target pixels; and

said filling step comprises, for each target pixel, looking up a respective corresponding source pixel in said table.

89. The method of claim 73 wherein said playing back step comprises playing back said respective target image visual data in electronic form.

90. The method of claim 89 wherein said playing back step comprises projecting said respective target image visual data onto said display surface in a video format.

91. The method of claim 90 wherein:

each said respective viewpoint is on a first side of said display surface; and

said projecting step comprises projecting said respective target image visual data onto said first side of said display surface.

92. The method of claim 90 wherein:

each said respective viewpoint is on a first side of said display surface; and

said projecting step comprises projecting said respective target image visual data onto a second side of said display surface opposite said first side.

93. The method of claim 73 wherein said playing back step comprises playing back said respective target image visual data photographically.

94. The method of claim 93 wherein said playing back step comprises projecting said respective target image visual data onto said display surface photographically.

95. The method of claim 94 wherein:

each said respective viewpoint is on a first side of said display surface; and

said projecting step comprises projecting said respective target image visual data onto said first side of said display surface.

96. The method of claim 94 wherein:

each said respective viewpoint is on a first side of said display surface; and

said projecting step comprises projecting said respective target image visual data onto a second side of said display surface opposite said first side.

97. The method of claim 73 wherein each said respective image is displayed on a number of display surfaces which together subtend said respective first field of view at said respective viewpoint.

98. The method of claim 97 wherein said playing back step comprises playing back said respective target image visual data as a number of playback image segments which together subtend said respective third field of view at said respective playback point.

99. The method of claim 98 wherein said number of display surfaces is equal to said number of playback image segments.

100. The method of claim 98 wherein said number of display surfaces is different from said number of playback image segments.

101. The method of claim 73 wherein said playing back step comprises playing back said respective target image visual data as a number of playback image segments which together subtend said respective third field of view at said respective playback point.

102. The method of claim 101 wherein said recording step comprises capturing said respective image as a number of captured image segments, each of said captured image segments being captured from said respective recording point, said captured image segments together subtending said respective second field of view at said respective recording point.

103. The method of claim 102 wherein said number of captured image segments is equal to said number of playback image segments.

104. The method of claim 102 wherein said number of captured image segments is different from said number of playback image segments.

105. The method of claim 73 wherein said recording step comprises capturing said respective image as a number of captured image segments, each of said captured image segments being captured from said respective recording point, said captured image segments together subtending said respective second field of view at said respective recording point.

106. The method of claim 105 wherein said respective image is displayed on a number of display surfaces which together subtend said respective first field of view at said respective viewpoint.

107. The method of claim 106 wherein said number of captured image segments is equal to said number of display surfaces.

108. The method of claim 106 wherein said number of captured image segments is different from said number of display surfaces.

109. The method of claim 106 wherein said playing back step comprises playing back said respective target image visual data as a number of playback image segments which together subtend said respective third field of view at said respective playback point.

110. The method of claim 109 wherein said number of captured image segments, said number of playback image segments, and said number of display surfaces are all equal.

111. The method of claim 109 wherein said number of captured image segments, said number of playback image segments, and said number of display surfaces are unequal.

112. The method of claim 111 wherein any two of said number of captured image segments, said number of playback image segments, and said number of display surfaces are equal.

113. Apparatus for serially displaying a plurality of images of a subject on a display surface having for each image a respective display surface shape and a respective display surface position, for observation from a corresponding plurality of viewpoints, one for each of said plurality of images, each of said viewpoints having a respective viewpoint position at which said display surface subtends a respective first field of view for its respective image, each of said displayed images having a controlled amount of distortion, said apparatus comprising:

means for serially recording source image visual data for each of said plurality of images with respect to a respective recording point at which said subject subtends a respective second field of view which is at least partially contained within said respective first field of view at said respective recording point;

intermediate transforming means for transforming said respective recorded image visual data for each of said plurality of images through a respective intermediate transformation process to produce respective target image visual data; and

means for serially playing back said respective target image visual data from respective playback points each having respective playback point position at which said display surface subtends a respective third field of view and such that said subject image subtends at said respective viewpoint a respective fourth field of view at least partially contained within said respective first field of view at said respective viewpoint, through a respective playback distorting transformation process;

wherein;

for each of said plurality of images, said respective intermediate transformation process, said respective playback distorting transformation process, and said respective first, second, third and fourth fields of view combine to contribute to said controlled amount of distortion.

114. The apparatus of claim 113 wherein said serial playing back means plays back all of said plurality of images from a single playback point.

115. The apparatus of claim 113 wherein said respective display surface shapes and positions comprise a single display surface shape and single display surface position used for all of said plurality of images.

116. The apparatus of claim 113 wherein said controlled amount of distortion is the absence of distortion.

117. The apparatus of claim 113 wherein said playing back means plays said target image visual data back on a nonplanar display surface, said nonplanar display surface contributing also to said controlled amount of distortion.

118. The apparatus of claim 113 wherein said recording means comprises means for generating digital image visual data.

119. The apparatus of claim 113 wherein said recording means comprises means for capturing live image visual data.

120. The apparatus of claim 119 wherein said capturing means captures said live image visual data on a photographic medium.

121. The apparatus of claim 119 wherein said capturing means captures said live image visual data on electronic form.

122. The apparatus of claim 121 wherein said capturing means captures said live image visual data in a video format.

123. The apparatus of claim 119 wherein said capturing means captures said respective image as a number of captured image segments, each of said respective captured image segments being captured from said respective recording point, said captured image segments together subtending said respective second field of view at said respective recording point.

124. The apparatus of claim 113 wherein:

said intermediate transforming means comprises digital transforming means;

said apparatus further comprising;

means for digitizing said source image visual data for use in said respective digital transforming means.

125. The apparatus of claim 113 wherein said serial recording means transforms, for each respective one of said images, respective source visual data of said subject through a respective recording distorting transformation process, said respective recording distorting transformation contributing also to said controlled amount of distortion.

126. The apparatus of claim 125 wherein:

said intermediate transforming means comprises digital transforming means;

said apparatus further comprising;

means for digitizing said source image visual data for use in said respective digital transforming means.

127. The apparatus of claim 126 wherein said digital transforming means comprises, for each target pixel of said respective target image visual data:

first determining means for determining from said respective playback distorting transformation process, said respective playback point position, said respective display surface position and said respective display surface shape, a location on said display surface onto which said target pixel would be projected;

second determining means for determining from said respective recording distorting transformation process, said respective viewpoint position, said respective display surface position and said respective display surface shape, a source pixel in said respective source image visual data which corresponds to said location on said display surface; and

means for filling said respective target pixel with image data from said source pixel.

128. The apparatus of claim 127 wherein:

at least one of said first and second determining means operates on each respective target pixel in said respective target image visual data before said filling means operates on any respective target pixel;

output of at least one of said first and second determining means for each respective target pixel is used to produce a table of correspondence between source and target pixels; and

for each respective target pixel, said filling means looks up a respective corresponding source pixel in said table.

129. The apparatus of claim 113 wherein said playing back means plays back said respective target image visual data in electronic form.

130. The apparatus of claim 129 wherein said playing back means projects said respective target image visual data onto said display surface in a video format.

131. The apparatus of claim 130 wherein:

each said respective viewpoint is on a first side of said display surface; and

said projecting means projects said respective target image visual data onto said first side of said display surface.

132. The apparatus of claim 130 wherein:

each said respective viewpoint is on a first side of said display surface; and

said projecting means projects said respective target image visual data onto a second side of said display surface opposite said first side.

133. The apparatus of claim 113 wherein said playing back means plays back said respective target image visual data photographically.

134. The apparatus of claim 133 wherein said playing back means projects said respective target image visual data onto said display surface photographically.

135. The apparatus of claim 134 wherein:

each said respective viewpoint is on a first side of said display surface; and

said projecting means projects said respective target image visual data onto said first side of said display surface.

136. The apparatus of claim 134 wherein:

each said respective viewpoint is on a first side of said display surface; and

said projecting means projects said respective target image visual data onto a second side of said display surface opposite said first side.

137. The apparatus of claim 113 wherein each said respective image is displayed on a number of display surfaces which together subtend said respective first field of view at said respective viewpoint.

138. The apparatus of claim 137 wherein said playing back means plays back said respective target image visual data as a number of playback image segments which together subtend said respective third field of view at said respective playback point.

139. The apparatus of claim 138 wherein said number of display surfaces is equal to said number of playback image segments.

140. The apparatus of claim 138 wherein said number of display surfaces is different from said number of playback image segments.

141. The apparatus of claim 113 wherein said playing back means plays back said respective target image visual data as a number of playback image segments which together subtend said respective third field of view at said respective playback point.

142. The apparatus of claim 141 wherein said recording means captures said respective image as a number of captured image segments, each of said captured image segments being captured from said respective recording point, said captured image segments together subtending said respective second field of view at said respective recording point.

143. The apparatus of claim 142 wherein said number of captured image segments is equal to said number of playback image segments.

144. The apparatus of claim 142 wherein said number of captured image segments is different from said number of playback image segments.

145. The apparatus of claim 113 wherein said recording means captures said respective image as a number of captured image segments, each of said captured image segments being captured from said respective recording point, said captured image segments together subtending said respective second field of view at said respective recording point.

146. The apparatus of claim 145 wherein said respective image is displayed on a number of display surfaces which together subtend said respective first field of view at said respective viewpoint.

147. The apparatus of claim 146 wherein said number of captured image segments is equal to said number of display surfaces.

148. The apparatus of claim 146 wherein said number of captured image segments is different from said number of display surfaces.

149. The apparatus of claim 146 wherein said playing back means plays back said respective target image visual data as a number of playback image segments which together subtend said respective third field of view at said respective playback point.

150. The apparatus of claim 149 wherein said number of captured image segments, said number of playback image segments, and said number of display surfaces are all equal.

151. The apparatus of claim 149 wherein said number of captured image segments, said number of playback image segments, and said number of display surfaces are unequal.

152. The apparatus of claim 151 wherein any two of said number of captured image segments, said number of playback image segments, and said number of display surfaces are equal.