Data processing for three-dimensional displays

• US 20050230641A1
• Filed: 03/16/2005
• Published: 10/20/2005
• Est. Priority Date: 04/05/2004
• Status: Active Grant

• Current Assignee: Gula Consulting Limited Liability Company (Intellectual Ventures LLC)
• Original Assignee: Actuality Systems, Inc. (Ellis Amalgamated LLC)
• Inventors: Chun, Won, Cossairt, Oliver Strider
• Granted Patent: US 7,525,541 B2
• Time in Patent Office: Days
• US Class Current: 250/484.400
• CPC Class Codes: G03H 2210/454 into planes
• CPC Class Codes: H04N 13/10 Processing, recording or transmission of stereoscopic or multi-view image signals
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/139 Format conversion, e.g. of frame-rate or size
• CPC Class Codes: H04N 13/207 using a single 2D image sensor
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 13/282 for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
• CPC Class Codes: H04N 13/302 for viewing without the aid of special glasses, i.e. using autostereoscopic displays
• CPC Class Codes: H04N 13/351 for displaying simultaneously
• CPC Class Codes: H04N 13/363 using image projection screens volumetric display H04N13/388
• CPC Class Codes: H04N 13/365 using digital micromirror devices [DMD]

Abstract

Generating a three-dimensional image of a three-dimensional scene by generating projection data and projecting light rays based on the projection data to generate the three-dimensional image. Sampling of the data is based on at least one physical parameter associated with the projection of light rays by a projection system used to project the light rays.

Claims

1. A method of processing image data, the method comprising:

generating projection data by sampling data representing a three-dimensional scene, the sampling based on at least one physical parameter associated with projection of light rays by a projection system; and

causing the projection system to project light rays based on the projection data to generate a three-dimensional image representative of the scene.

2. The method of claim 1, further comprising defining an image surface onto which the light rays are projected.

3. The method of claim 2, in which sampling data comprises sampling based on at least one of (a) a size of the image surface, (b) a shape of the image surface, and (c) a position of the image surface relative to intended observation positions for observing the three-dimensional image.

4. The method of claim 2, further comprising defining positions from which the light rays appear to be projected.

5. The method of claim 4, in which sampling data comprises sampling based on at least one of (a) a position of the image surface relative to positions from which the light rays appear to be projected, and (b) an extent of a region from which the light rays appear to be projected.

6. The method of claim 4, in which projecting the light rays comprises focusing light rays that are modulated by a spatial light modulator, in which the light rays are focused on the positions from which the light rays appear to be projected.

7. The method of claim 6, in which the spatial light modulator modulates the light rays based on the projection data.

8. The method of claim 2, in which defining an image surface comprises defining an image surface that diffuses the light rays only along a specified direction.

9. The method of claim 1, in which the three-dimensional scene comprises at least one of a collection of points, a collection of lines, a collection of surfaces, and a collection of volumes.

10. The method of claim 1, in which generating projection data comprises generating data that specify at least one of color and brightness of light rays that propagate along specified paths relative to the image surface.

11. The method of claim 1, in which the data representing the three-dimensional scene comprises information about observations of the three-dimensional scene from different positions.

12. The method of claim 11, in which the data representing the three-dimensional scene comprises images of the three-dimensional scene captured by at least one camera from different positions.

13. The method of claim 12, in which the camera has a pixel resolution that is different from a pixel resolution of a spatial light modulator used to modulate the projected light rays.

14. The method of claim 12, in which sampling data comprises sampling data to compensate for distortions in the images.

15. The method of claim 11, in which the data representing the three-dimensional scene comprises synthesized images of the three-dimensional scene as viewed from different positions.

16. The method of claim 15, in which the synthesized images each has a pixel resolution that is different from a pixel resolution of a spatial light modulator used to modulate the projected light rays.

17. The method of claim 1, in which projecting light rays comprises projecting a first set of modulated light rays from a first position during a first interval, and projecting a second set of modulated light rays from a second position during a second interval.

18. The method of claim 1, in which the three-dimensional image comprises an image of at least one of an accurate copy, an approximation, a closer view, and a farther view of the three-dimensional scene.

19. The method of claim 1, in which the data representing the three-dimensional scene comprises information about at least one of color and brightness of light rays emanating from the three-dimensional scene.

20. A method of processing image data, the method comprising:

generating sets of projection data by sampling data representing a three-dimensional scene, the sampling based on a physical parameter associated with a projection system, the sets of projection data having information representing light rays that, when projected, generate a three-dimensional image representative of the three-dimensional scene.

21. A method of processing image data, the method comprising:

sampling a view volume, which comprises data representing a three-dimensional scene, to generate projection data, the sampling based at least on a parameter associated with an image surface and with projection positions that define locations from which light rays appear to be projected; and

projecting light rays from the projection positions toward the image surface based on the projection data, thereby generating a three-dimensional image that is representative of the three-dimensional scene.

22. A method of processing image data, the method comprising:

generating a view volume that comprises data representing observations of a three-dimensional scene from different positions;

sampling the view volume to generate projection data, the sampling based on at least a position of an image surface relative to a position of a presentation surface;

modulating light rays based on the projection data to generate presentation images; and

projecting the presentation images from the presentation surface toward the image surface to reproduce a three-dimensional image that is representative of the three-dimensional scene.

23. A method of processing image data, the method comprising:

dithering a series of images using dither matrices to generate a series of dithered images, in which each two consecutive images are dithered using two different dither matrices; and

projecting the dithered images from a plurality of locations to generate a three-dimensional image of a three-dimensional scene.

24. The method of claim 23, in which two consecutive images are dithered by a first dither matrix and a second dither matrix generated by rotating the first dither matrix by 90 degrees.

25. The method of claim 24, in which each dither matrix comprises N rows and N columns of cells, and a cell (i, j) in the second matrix has a same value as a cell (j, N−

i+1) in the first matrix.

26. The method of claim 23, in which four consecutive images are dithered by a first dither matrix, a second dither matrix, a third dither matrix, and a fourth dither matrix, in which the second, the third, and the fourth dither matrices are rotations of the first dither matrix by 90, 180, and 270 degrees, respectively.

27. The method of claim 23, in which the dither matrix comprises a dispersed dot dither matrix.

28. A method of processing image data, the method comprising:

receiving an image having pixels, each of which defines a pixel value; and

for each group of n pixels of the image, storing the most significant bit of each of the n pixel values in an n-bit binary number by successively adding each of the n pixel values to a shift register and shifting the value in the shift register by one bit after the addition, thereby generating compressed image data.

29. The method of claim 28, further comprising driving a digital micromirror device based on the compressed image data.

30. A method of processing image data, the method comprising:

sending at least one physical parameter of a projection system to a server having data representing a three-dimensional scene;

at the projection system, receiving projection data from the server, the projection data derived by sampling the data representing the three-dimensional scene based on the at least one physical parameter; and

at the projection system, projecting light rays based on the projection data to generate a three-dimensional image of the three-dimensional scene.

31. The method of claim 30, further comprising:

sending at least one physical parameter of the projection system to a second server having data representing a second three-dimensional scene;

at the projection system, receiving a second set of projection data from the second server, the second set of projection data derived by sampling the data representing the second three-dimensional scene based on the at least one physical parameter; and

at the projection system, projecting light rays based on the projection data to generate a three-dimensional image of the second three-dimensional scene.

32. A method of processing image data, the method comprising:

at a server having data representing a three-dimensional scene, receiving at least one physical parameter of a projection system;

at the server, generating projection data by sampling the data representing the three-dimensional scene based on the at least one physical parameter; and

transmitting the projection data from the server to the projection system to enable the projection system to project light rays based on the projection data to generate a three-dimensional image of the three-dimensional scene.

33. The method of claim 32, further comprising:

at the server, receiving at least one physical parameter of a second projection system;

at the server, generating a second set of projection data by sampling the data representing the three-dimensional scene based on the at least one physical parameter of the second projection system; and

transmitting the second set of projection data from the server to the second projection system to enable the second projection system to project light rays based on the projection data to generate a three-dimensional image of the three-dimensional scene.

34. A method of processing image data, the method comprising:

generating a stream of data representing observations of a three-dimensional scene from a set of positions;

at a given time, storing a subset of the stream of data, the subset representing observations of the three-dimensional scene from a subset of the positions;

generating a stream of projection data by sampling the stored subset of the stream of data; and

projecting light rays based on the stream of projection data to generate a three-dimensional image of scene.

35. The method of claim 34, further comprising defining an image surface onto which the light rays are projected.

36. The method of claim 35, in which sampling the stored subset of the stream of data comprises sampling based on at least one of:

(a) a position of the image surface relative to positions from which the light rays appear to be projected,

(b) a position of the image surface relative to intended observation positions for observing the three-dimensional image,

(c) a size of the image surface,

(d) an extent of positions from which the light rays appear to be projected, and

(e) a shape of the image surface.

37. The method of claim 34, in which the stream of data representing observations of the scene ends after data representing observations of the scene from all of the set of positions have been generated.

38. The method of claim 34, in which the stream of data representing observations of the scene repeats itself after data representing observations of the scene from all of the set of positions have been generated.

39. An image data processing apparatus, the apparatus comprising:

a data processor to generate projection data by sampling data representing a three-dimensional scene; and

a projection device to project light rays based on the projection data to generate a three-dimensional image that is representative of the three-dimensional scene, in which the sampling of data by the data processor is based on a physical parameter associated with projection of the light rays.

40. The apparatus of claim 39, in which the projection device defines an image surface onto which the light rays are projected.

41. The apparatus of claim 40, in which the data processor samples the data based on at least one of (a) a size of the image surface, (b) a shape of the image surface, and (c) a position of the image surface relative to intended observation positions for observing the three-dimensional image.

42. The apparatus of claim 40, in which the projection device comprises a spatial light modulator that modulates light rays according to the projection data.

43. The apparatus of claim 42, in which the projection device defines projection positions from which light rays modulated by the spatial light modulator appear to be projected.

44. The apparatus of claim 43, in which the data processor samples the data based on at least one of (a) a position of the image surface relative to the projection positions, and (b) a farthest distance between any two projection positions.

45. The apparatus of claim 43, in which the spatial light modulator modulates light rays to generate different presentation images at different times according to the projection data, and the projection device projects the presentation images from different projection positions in a time-division-multiplexed manner.

46. A machine-accessible medium, which when accessed results in a machine performing operations comprising:

generating projection data by sampling data representing a three-dimensional scene, the sampling based on at least one physical parameter associated with projection of light rays by a projection system; and

projecting light rays based on the projection data to generate a three-dimensional image that is representative of the scene.

47. The machine-accessible medium of claim 46, which when accessed results in the machine performing operations comprising controlling a spatial light modulator to modulate projected light rays based on the projection data.

48. The machine-accessible medium of claim 46, which when accessed results in the machine performing operations comprising projecting light rays modulated by the spatial light modulator from different positions in a time-division-multiplexed manner.

49. An apparatus for processing image data, the apparatus comprising:

a projection system for projecting light rays; and

means for generating projection data by sampling data representing a three-dimensional scene based on at least one physical parameter associated with projection of light rays by the projection system, in which light rays projected by the projection system based on the projection data generate a three-dimensional image that is representative of the scene.