Method And Device For Representing A Digital Image On A Surface Which Is Non-Trivial In Terms Of Its Geometry And Photometry

• US 20080095468A1
• Filed: 08/02/2005
• Published: 04/24/2008
• Est. Priority Date: 08/30/2004
• Status: Active Grant

• Current Assignee: Vioso Gmbh
• Original Assignee: Bauhaus-Universitaet Weimar
• Inventors: Emmerling, Andreas, Bimber, Oliver, Klemmer, Thomas
• Granted Patent: US 8,195,006 B2
• Time in Patent Office: Days
• US Class Current: 382/285
• CPC Class Codes: H04N 9/3147 Multi-projection systems displays in general H04N9/12; video walls G06F3/1446, G09G2300/026
• CPC Class Codes: H04N 9/3194 including sensor feedback

Abstract

The invention relates to a method and a device for representing a two-dimensional digital image on a projection surface, whereby at least one projector projects the content of an image buffer onto the projection surface. The aim of the invention is to provide a method and a device which allow to project a digital image onto a background having any surface structure and/or color in such a manner that any influences on the image caused by the background are compensated to the last pixel for at least one special observer perspective. In a special embodiment, definition of the representation is optimized to the last pixel even on an uneven background. For this purpose, the digital image is processed during a rendering step by geometrically distorting it using a two-dimensional pixel-offset field which contains information on the projection surface, and by manipulating the color of the image by means of a two-dimensional surface texture of the projection surface. A least a part of the processed image is rendered in an image buffer and a rectangle is written into the image buffer for initiating the rendering step, thereby compensating scattered light influences. The invention especially relates to a system comprising a plurality of projectors. Definition of representation can be optimized to the last pixel by means of definition values and representation is furthermore possible subject to the observer'"'"'s position.

Claims

1-41. -41. (canceled)

42. A method of representing a digital image on a projection surface, comprising:

processing said digital image to create a processed image, said processing including geometric distortion of said digital image using a two-dimensional pixel displacement map that contains information about said projection surface and/or color manipulation using a two-dimensional surface texture of said projection surface; and

rendering at least a part of the processed image into an image buffer; and

projecting a content of the image buffer onto said projection surface by at least one projector.

43. A method according to claim 42, wherein the geometric distortion and/or color manipulation are performed using at least one pixel shader.

44. A method according to claim 42, further comprising:

performing automatic calibration which includes projecting at least one projected calibration image onto said projection surface by said projector; and

at least partially recording and digitizing said at least one projected calibration image by use of a camera.

45. A method according to claim 44, wherein said camera is arranged at an observer position for the automatic calibration.

46. A method according to claim 44, further comprising:

identifying an associated recorded pixel of a digitized recorded calibration image for at least one projected pixel of said at least one projected calibration image and a two-dimensional geometric displacement between said at least one projected pixel;

determining said recorded pixel; and

storing said recorded pixel as an element of said two-dimensional pixel displacement map.

47. A method according to claim 46, further comprising:

at least partially projecting and recording multiple ones of said at least one projected calibration image successively, said projected calibration images containing temporally multiplexed stripe patterns; and

determining recorded pixels associated with pixels to be projected by using said stripe patterns.

48. A method according to claim 46, further comprising:

determining a mean value of two-dimensional geometric displacements belonging to a recorded pixel when there are more than one digitized recorded calibration image; and

storing the mean value as an element of said two-dimensional pixel displacement map.

49. A method according to claim 44, further comprising:

projecting at least one test image by use of said projector;

repeatedly recording and digitizing images by use of said camera until said test image is identified in said digitized recorded images; and

measuring a latency time between projection of said test image and identification of said test image.

50. A method according to claim 44, wherein:

in the case of a single projector, a rectangle with very high intensity white color is projected as said projected calibration image by means of said projector, said projection surface not being illuminated with environmental illumination; and

a recorded calibration image digitized thereby being used as said two-dimensional surface texture of said projection surface.

51. A method according to claim 44, wherein:

in the case of more than one said projector, a rectangle with very high intensity white color is projected sequentially as said projected calibration image by means of each said projector, said projection surface not being illuminated with environmental illumination; and

a recorded calibration image digitized thereby being used as said two-dimensional surface texture of said projection surface for said respective one of said projector.

52. A method according to claim 44, wherein a black calibration image is projected by said projector and is recorded and digitized by said camera, said digitized recorded calibration image being used as an environmental illumination texture for said color manipulation.

53. A method according to claim 44, wherein a digitized recorded calibration image used for a surface texture or for an environmental illumination texture is initially distorted into a perspective of said projector using said two-dimensional pixel displacement map and/or an associated pixel displacement texture.

54. A method according to claim 44, wherein a lens distortion of said camera is compensated after the digitizing and recording therein by a computer.

55. A method according to claim 44, wherein recordings of said camera are provided with a mask.

56. A method according to claim 44, wherein said calibration is performed repeatedly, said camera being arranged in a different position for each calibration process, and said pixel displacement map determined for each position is stored as a pixel displacement auxiliary map and said surface texture determined is stored as a surface auxiliary texture allocated to an associated position.

57. A method according to claim 56, wherein a position of said observer is determined dynamically prior to said geometric distortion and color manipulation and said pixel displacement map and said surface texture are determined using said pixel displacement auxiliary maps determined for said different camera positions and said position determined for said surface auxiliary textures.

58. A method according to claim 57, wherein said pixel displacement map is determined in that there is weighted interpolation between said pixel displacement auxiliary maps determined for said different camera positions, and in that said surface texture is determined in that there is weighted interpolation between said surface auxiliary textures determined for said different camera positions.

59. A method according to claim 42, wherein:

said at least one projector includes at least two projectors;

a plurality of digital images are represented on said projection surface by said projectors that are set for different image planes;

contents of appropriate image buffers that contain intensities of pixels to be projected are projected in areas of said projection surface; and

for one image point of said projection surface onto which point a plurality of pixels are simultaneously projected from different ones of said projectors, said intensities to be projected for said pixels using respective ones of said projectors are manipulated using definition values for individual respective pixels.

60. A method according to claim 59, wherein said pixels are manipulated using said definition values by use of at least one pixel shader.

61. A method according to claim 59, wherein said pixels which are manipulated using said definition values occurs in that of a plurality of pixels that can be projected from different projectors onto a same image point of said projection surface, the intensity of those with the highest of said definition values remains unchanged, while the rest of said pixels are darkened.

62. A method according to claim 59, wherein said manipulating using said definition values occurs in that the intensities of a plurality of pixels that can be projected from different projectors onto a same image point of said projection surface are scaled in a weighted manner using ones of said definition values associated therewith.

63. A method according to claim 59, wherein said definition values of said pixels are determined in a calibration process, for which purpose a calibration pattern is projected successively by means of each said projector and the projections reflected by said projection surface are recorded in appropriate calibration images by said camera.

64. A method according to claim 60, wherein in one of said recorded calibration images an intensity of a pixel is normalized a first time in that it is placed in relation to the intensity of the corresponding pixel in said projected calibration pattern.

65. A method according to claim 61, wherein a plurality of intensities that have been normalized for a first time and that are from a calibration image are normalized with one another a second time.

66. A method according to claim 61, wherein a definition value is determined from normalized intensities of one of said at least one calibration image by means of image decomposition using a moment retaining principle.

67. A method according to claim 66, wherein said image decomposition is performed within a two-dimensional scanning segment of the associated calibration image.

68. A method according to claim 60, wherein said calibration pattern for one said projector is displaced in multiple steps and an associated calibration image is recorded after each of said steps.

69. A method according to claim 60, wherein a matrix of circular points is used for said calibration pattern.

70. A method according to claim 60, further comprising correcting said calibration pattern in terms of geometry and/or color with respect to the properties of said projection surface individually for each said projector before the projection.

71. A method according to claim 42, wherein during color manipulation, mutual illumination of different projection surface segments is compensated by form factors corresponding to various projection surface segments.

72. A method according to claim 71, wherein said compensation is performed iteratively, an intensity to be projected onto a projection surface segment being manipulated using a reflected scatter intensity.

73. An apparatus for representing a digital image on a projection surface, comprising:

a control unit;

an image buffer;

at least one projector; and

a camera, said projector and said camera being coupled to the control unit that processes said digital image during a rendering process, said control unit configured to conduct geometric distortion using a two-dimensional pixel displacement map that contains information about said projection surface and/or manipulates color using a two-dimensional surface texture of said projection surface, at least a part of the processed image being rendered into the image buffer.

74. An apparatus according to claim 73, wherein said control unit is connected to an image-producing source that delivers to said control unit digital images that are to be represented on said projection surface after a calibration process.

75. An apparatus according to claim 73, wherein during color manipulation, said control unit compensates mutual illumination of various projection surface segments using form factors corresponding to said projection surface segments.

76. An apparatus according to claim 73, wherein said at least one projector includes at least two projectors arranged to project from different directions, illuminated areas of said projectors overlapping or each illuminating a portion of said projection surface independent of one another.

77. An apparatus according to claim 73, wherein said control unit includes an image buffer that is connected to at least one programmable pixel shader that geometrically distorts said digital image when rendering into said image buffer.

78. An apparatus according to claim 73, wherein said at least one projector includes at least two projectors which are oriented to respective projection fields thereof at an angle, said respective projection fields overlapping in a multifocal projection, and a first of said projectors being set having a fixed focus on a first image plane and second of said projectors being set having a fixed focus on a second image plane.

79. An apparatus according to claim 73, wherein said at least one projector includes four projectors which are set to different image planes, respective projection fields of every two of said four projectors overlap one another at least partially.

80. An apparatus according to claim 73, wherein said at least one projector includes four projectors which are arranged in a single equipment unit, four respective projection fields of the four projectors being oriented offset in parallel and overlapping one another.

81. An apparatus according to claim 80, wherein said camera is arranged in or on said equipment unit.

82. An apparatus according to claim 73, wherein said camera is detachably attached to other elements of said apparatus.