Variable Fixation Viewing Distance Scanned Light Displays

• US 20080117289A1
• Filed: 08/05/2005
• Published: 05/22/2008
• Est. Priority Date: 08/06/2004
• Status: Active Grant

• Current Assignee: Brian T. Schowengerdt, Erie J. Seibel
• Original Assignee: Brian T. Schowengerdt, Erie J. Seibel
• Inventors: Seibel, Erie J., Schowengerdt, Brian T.
• Granted Patent: US 8,248,458 B2
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: G02B 26/005 based on electrowetting
• CPC Class Codes: H04N 13/322 using varifocal lenses or mirrors
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays

Abstract

In addition to varying the luminance and chrominance levels of each voxel in an image, a viewing distance for objects within the image is varied to display a multi-focal, three-dimensional image. The focal distance of the voxels is determined by a depth map that indicates a fixation viewing depth of each object. When the image is viewed, there is a natural correspondence between the accommodation and vergence of the viewer'"'"'s eye. The focal distance of the voxels displayed in the image are modulated using, for example, one or more dynamically variable focus mirrors or lenses, a plurality of fixed focus mirrors or lenses having different focal distances, a plurality of different length optical paths with different focal distance ranges, or at least one spatial light modulator. Light used to produce the image is scanned in two orthogonal directions either before or after the light is focus modulated.

Claims

The. The invention in which an exclusive right is claimed is defined by the following:

1. A method for producing a multi-focal three-dimensional visually perceptible image that includes objects at different fixation viewing distances and in which a fixation viewing distance of each object in the image substantially corresponds to its vergence when the image is viewed, comprising the steps of:

(a) providing both two-dimensional image data and a depth map for voxels of the multi-focal three-dimensional visually perceptible image to be displayed, the depth map indicating a fixation viewing distance of objects in the image;

(b) in response to the two-dimensional image data, producing light for displaying voxels of the image;

(c) varying a wavefront divergence of the light for the voxels of the image in response to the fixation viewing distance indicated by the depth map; and

(d) displaying the image, so that as a function of a viewing distance in the image of objects represented by the voxels, the voxels of the image are displayed with a corresponding divergence, wherein the viewing distance of objects in the image can vary within a maximum range that extends from less than 7 cm to infinity.

2. The method of claim 1, wherein the step of producing the light for displaying voxels of the image comprises the steps of:

(a) for each voxel of the image, varying an intensity of the light used to produce the image, as a function of the two-dimensional image data for the voxel, so that an intensity of the light at the voxel in the image corresponds to the two-dimensional data for the voxel; and

(b) directing the light to produce the image so that it is visually perceptible.

3. The method of claim 2, wherein the image is multi-color, and wherein for each voxel, the step of producing the light for displaying voxels of the image comprises the step varying an intensity of the light from a plurality of different wavelength light sources, before the step of directing the light to produce the image.

4. The method of claim 1, further comprising the step of scanning the light to produce the image, wherein the step of varying the wavefront divergence comprises the step of modulating a focal distance of the light for the voxels, so that the voxels comprising the image are at different focal distances generally corresponding to different viewing distances of the objects in the image, as indicated by the depth map.

5. The method of claim 4, wherein the step of varying the wavefront divergence comprises the step of employing at least one optical element to produce a plurality of different focal distances of the light for the voxels of the image, wherein the at least one optical element has a focus characteristic that is one of fixed and dynamically variable.

6. The method of claim 5, further comprising the steps of:

(a) providing a plurality of light sources, so that each of the plurality of different focal distances has at least one separate light source;

(b) separately luminance modulating the plurality of light sources over time to produce an individual voxel stream for each separate light source;

(c) optically combining and superimposing images formed of the individual voxel streams that are at different focal distances; and

(d) scanning the images that were superimposed, to produce the multi-focal three-dimensional visually perceptible image as a composite of images formed with the light from the plurality of light sources at the plurality of different focal distances.

7. The method of claim 6, wherein the composite of images enables objects at a greater focal distance to be seen through an image formed at a nearer focal distance that is semi-transparent.

8. The method of claim 6, wherein the composite of images enables reflections from objects at different focal distances to be visible in the image.

9. The method of claim 6, further comprising the step of providing different path lengths along which the light from the plurality of light sources travels, to produce the different focal distances.

10. The method of claim 5, wherein the at least one optical element modifies the focal distance of voxels after at least one scanning of the light has occurred.

11. The method of claim 4, wherein the step of modulating the focal distance of the light comprises the step of dynamically varying a focus of a deformable membrane mirror before the light is scanned.

12. The method of claim 4, wherein the step of modulating the focal distance of the light comprises the step of dynamically varying the focal distance of the light using one of:

(a) a fluid lens;

(b) a nematic liquid crystal spatial modulator;

(c) a ferroelectric liquid crystal spatial light modulator;

(d) a non-polymeric electro-optical spatial light modulator; and

(e) an electro-optical polymer spatial light modulator.

13. The method of claim 1, further comprising the steps of:

(a) monitoring a viewer'"'"'s eyes to detect at least one of a plurality of different parameters while the image is being viewed, producing a feedback signal, wherein the plurality of different parameters include an accommodation, a vergence, a pupil size, and an eye position; and

(b) modifying the display of the image in response to the feedback signal.

14. The method of claim 1, wherein the step of varying the wavefront divergence comprises the steps of:

(a) providing a plurality of different channels, each channel having a different focal distance range that is a subset of a full focal distance range for the image to be displayed and being selected so that the full focal distance range of the image is encompassed by the different focal distance ranges of the plurality of different channels; and

(b) varying the focal distance in appropriate channels, within the focal distance range of the appropriate channels, where an appropriate channel is selected and the focal distance for the channel that is selected is varied within the focal distance range of the channel that is selected, based upon the depth map for the voxels being displayed by the channels in the image.

15. The method of claim 1, further comprising the step of statically shifting a full range of accommodation for the image to compensate for vision errors in a viewer'"'"'s eye, so that the viewer is able to clearly view the image with the eye without wearing any corrective lens.

16. The method of claim 1, wherein each voxel of the image is processed separately based upon the indication of fixation viewing distance included in the depth map, to provide a voxel-by-voxel focus modulation when displaying the image.

17. The method of claim 14, wherein the viewer'"'"'s eyes interpolate focal distances between the focal distance range of the plurality of channels in response to other visual cues included in the image.

18. The method of claim 1, further comprising the step of scaling a focal distance for the voxels of the image to one of:

(a) increase a total focal distance range for objects in the image, so that the objects at different focal distances appear farther apart from each other; and

(b) decrease the total focal distance range for objects in the image, so that the objects at different focal distances appear closer to each other.

1-1. A method for producing a multi-focal three-dimensional visually perceptible image that includes objects at different fixation viewing distances and in which a fixation viewing distance of each object in the image substantially corresponds to its vergence when the image is viewed, comprising the steps of:

(a) providing both two-dimensional image data and a depth map for voxels of the multi-focal three-dimensional visually perceptible image to be displayed, the depth map indicating a fixation viewing distance of objects in the image;

(b) in response to the two-dimensional image data, producing light for displaying voxels of the image;

(c) varying a wavefront divergence of the light for the voxels of the image in response to the fixation viewing distance indicated by the depth map; and

(d) displaying the image, so that as a function of a viewing distance in the image of objects represented by the voxels, the voxels of the image are displayed with a corresponding divergence, wherein the viewing distance of objects in the image can vary within a maximum range that extends from less than 7 cm to infinity.

2-2. The method of claim 1, wherein the step of producing the light for displaying voxels of the image comprises the steps of:

(a) for each voxel of the image, varying an intensity of the light used to produce the image, as a function of the two-dimensional image data for the voxel, so that an intensity of the light at the voxel in the image corresponds to the two-dimensional data for the voxel; and

(b) directing the light to produce the image so that it is visually perceptible.

3-3. The method of claim 2, wherein the image is multi-color, and wherein for each voxel, the step of producing the light for displaying voxels of the image comprises the step varying an intensity of the light from a plurality of different wavelength light sources, before the step of directing the light to produce the image.

4-4. The method of claim 1, further comprising the step of scanning the light to produce the image, wherein the step of varying the wavefront divergence comprises the step of modulating a focal distance of the light for the voxels, so that the voxels comprising the image are at different focal distances generally corresponding to different viewing distances of the objects in the image, as indicated by the depth map.

5-5. The method of claim 4, wherein the step of varying the wavefront divergence comprises the step of employing at least one optical element to produce a plurality of different focal distances of the light for the voxels of the image, wherein the at least one optical element has a focus characteristic that is one of fixed and dynamically variable.

6-6. The method of claim 5, further comprising the steps of:

(a) providing a plurality of light sources, so that each of the plurality of different focal distances has at least one separate light source;

(b) separately luminance modulating the plurality of light sources over time to produce an individual voxel stream for each separate light source;

(c) optically combining and superimposing images formed of the individual voxel streams that are at different focal distances; and

(d) scanning the images that were superimposed, to produce the multi-focal three-dimensional visually perceptible image as a composite of images formed with the light from the plurality of light sources at the plurality of different focal distances.

7-7. The method of claim 6, wherein the composite of images enables objects at a greater focal distance to be seen through an image formed at a nearer focal distance that is semi-transparent.

8-8. The method of claim 6, wherein the composite of images enables reflections from objects at different focal distances to be visible in the image.

9-9. The method of claim 6, further comprising the step of providing different path lengths along which the light from the plurality of light sources travels, to produce the different focal distances.

10-10. The method of claim 5, wherein the at least one optical element modifies the focal distance of voxels after at least one scanning of the light has occurred.

11-11. The method of claim 4, wherein the step of modulating the focal distance of the light comprises the step of dynamically varying a focus of a deformable membrane mirror before the light is scanned.

12-12. The method of claim 4, wherein the step of modulating the focal distance of the light comprises the step of dynamically varying the focal distance of the light using one of:

(a) a fluid lens;

(b) a nematic liquid crystal spatial modulator;

(c) a ferroelectric liquid crystal spatial light modulator;

(d) a non-polymeric electro-optical spatial light modulator; and

(e) an electro-optical polymer spatial light modulator.

13-13. The method of claim 1, further comprising the steps of:

(a) monitoring a viewer'"'"'s eyes to detect at least one of a plurality of different parameters while the image is being viewed, producing a feedback signal, wherein the plurality of different parameters include an accommodation, a vergence, a pupil size, and an eye position; and

(b) modifying the display of the image in response to the feedback signal.

14-14. The method of claim 1, wherein the step of varying the wavefront divergence comprises the steps of:

(a) providing a plurality of different channels, each channel having a different focal distance range that is a subset of a full focal distance range for the image to be displayed and being selected so that the full focal distance range of the image is encompassed by the different focal distance ranges of the plurality of different channels; and

(b) varying the focal distance in appropriate channels, within the focal distance range of the appropriate channels, where an appropriate channel is selected and the focal distance for the channel that is selected is varied within the focal distance range of the channel that is selected, based upon the depth map for the voxels being displayed by the channels in the image.

15-15. The method of claim 1, further comprising the step of statically shifting a full range of accommodation for the image to compensate for vision errors in a viewer'"'"'s eye, so that the viewer is able to clearly view the image with the eye without wearing any corrective lens.

16-16. The method of claim 1, wherein each voxel of the image is processed separately based upon the indication of fixation viewing distance included in the depth map, to provide a voxel-by-voxel focus modulation when displaying the image.

17-17. The method of claim 14, wherein the viewer'"'"'s eyes interpolate focal distances between the focal distance range of the plurality of channels in response to other visual cues included in the image.

18-18. The method of claim 1, further comprising the step of scaling a focal distance for the voxels of the image to one of:

(a) increase a total focal distance range for objects in the image, so that the objects at different focal distances appear farther apart from each other; and

(b) decrease the total focal distance range for objects in the image, so that the objects at different focal distances appear closer to each other.

19. A system for producing a multi-focal, three-dimensional visually perceptible image that includes objects at different fixation viewing distances and in which a fixation viewing distance of each object in the image substantially corresponds to its vergence when the image is viewed, comprising:

(a) a signal conveying two-dimensional image data and a fixation viewing distance for voxels comprising objects of the multi-focal, three-dimensional visually perceptible image to be displayed;

(b) at least one light source that produces light for the image;

(c) means for modulating the light from the at least one light source based upon the two-dimensional image data;

(d) means for varying a wavefront divergence of the light from the at least one light source, in response to the fixation viewing distance of the voxels of the image; and

(e) means for displaying the image, so that as a function of a viewing distance in the image of objects represented by the voxels, the voxels of the image are displayed with a corresponding divergence, wherein the viewing distance of objects in the image can vary within a maximum range that extends from less than 7 cm to infinity.

20. The system of claim 19, wherein the means for modulating the light from the at least one light source comprises means for modulating at least one of:

(a) a luminosity of the light; and

(b) a luminosity of each of a plurality of different wavelengths that determine a color of each voxel in the image.

21. The system of claim 19, wherein the means for varying the wavefront divergence comprises at least one of the following components:

(a) a dynamically variable focus mirror;

(b) a dynamically variable focus lens;

(c) a plurality of fixed focus mirrors of different focal distances;

(d) a plurality of fixed focus lenses of different focal distances;

(e) a plurality of different length optical paths for each of a plurality of different focal distance ranges; and

(f) at least one spatial light modulator.

22. The system of claim 19, wherein the means for varying the wavefront divergence modulates a focal distance of the light for the voxels, so that the voxels comprising the image are at different focal distances generally corresponding to the different viewing distances of the objects in the image.

23. The system of claim 19, further comprising:

(a) a plurality of light sources, wherein a separate light source is provided for each of a plurality of different focal distances;

(b) means for modulating the plurality of light sources separately over time to produce an individual voxel stream for each separate light source;

(c) means for optically combining and superimposing images formed of individual voxel streams at different focal distances; and

(d) a two-dimensional scanner that scans the images that were superimposed, to produce the multi-focal three-dimensional visually perceptible image as a composite of images formed with the light from the plurality of light sources and at the plurality of different focal distances.

24. The system of claim 23, wherein the two-dimensional scanner comprises a resonant scanner.

25. The system of claim 23, wherein the composite of images enables objects at a greater focal distance to be seen through an image formed at a nearer focal distance that is semi-transparent.

26. The system of claim 23, wherein the composite of images enables reflections from objects at different focal distances to be visible in the image.

27. The system of claim 19, wherein the means for displaying include a first scanner for scanning the light over a first axis of the image, and a second scanner for scanning the light over a second axis of the image, wherein the first axis and the second axis are generally orthogonal to each other.

28. The system of claim 27, wherein the means for varying the wavefront divergence by modifying the focal distance of voxels comprising the image are disposed downstream of at least one of the first scanner and the second scanner.

29. The system of claim 27, wherein the means for varying the wavefront divergence by modifying the focal distance of voxels comprising the image is disposed upstream of both the first scanner and the second scanner.

30. The system of claim 19, wherein the means for varying the wavefront divergence comprises one of the following components:

(a) a fluid lens;

(b) a nematic liquid crystal spatial modulator;

(c) a ferroelectric liquid crystal spatial light modulator;

(d) an electro-optical polymer spatial light modulator; and

(e) a non-polymeric electro-optical spatial light modulator.

31. The system of claim 19, wherein the means for displaying the image comprise a head-mounted display.

32. The system of claim 19, further comprising means for monitoring a viewer'"'"'s eyes to detect at least one of an accommodation, a vergence, a pupil size, and an eye position while the image is being viewed, producing a feedback signal, wherein the means for displaying the image further modify the image as displayed in response to the feedback signal.

33. The system of claim 19, wherein the means for varying the wavefront divergence comprise:

(a) a plurality of different channels, each channel having a focal distance range that is a subset of a full focal distance range for the image to be displayed and being selected so that the full focal distance range of the image is encompassed by the plurality of different channels; and

(b) means for varying the focal distance in appropriate channels, within the focal distance range of the appropriate channels, where an appropriate channel is selected and the focal distance for the channel that is selected is varied within the focal distance range of the channel that is selected, based upon the depth map for the voxels being displayed by the channels in the image.

34. The system of claim 19, wherein the means for displaying the image comprise a display that is spaced apart from a viewer of the image and is not supported by a viewer'"'"'s head.

35. The system of claim 19, wherein to display the image, each voxel of the image is processed separately by the means for varying the wavefront divergence, based upon the two-dimensional image data and the fixation depth data.

19-19. A system for producing a multi-focal, three-dimensional visually perceptible image that includes objects at different fixation viewing distances and in which a fixation viewing distance of each object in the image substantially corresponds to its vergence when the image is viewed, comprising:

(a) a signal conveying two-dimensional image data and a fixation viewing distance for voxels comprising objects of the multi-focal, three-dimensional visually perceptible image to be displayed;

(b) at least one light source that produces light for the image;

(c) means for modulating the light from the at least one light source based upon the two-dimensional image data;

(d) means for varying a wavefront divergence of the light from the at least one light source, in response to the fixation viewing distance of the voxels of the image; and

(e) means for displaying the image, so that as a function of a viewing distance in the image of objects represented by the voxels, the voxels of the image are displayed with a corresponding divergence, wherein the viewing distance of objects in the image can vary within a maximum range that extends from less than 7 cm to infinity.

20-20. The system of claim 19, wherein the means for modulating the light from the at least one light source comprises means for modulating at least one of:

(a) a luminosity of the light; and

(b) a luminosity of each of a plurality of different wavelengths that determine a color of each voxel in the image.

21-21. The system of claim 19, wherein the means for varying the wavefront divergence comprises at least one of the following components:

(a) a dynamically variable focus mirror;

(b) a dynamically variable focus lens;

(c) a plurality of fixed focus mirrors of different focal distances;

(d) a plurality of fixed focus lenses of different focal distances;

(e) a plurality of different length optical paths for each of a plurality of different focal distance ranges; and

(f) at least one spatial light modulator.

22-22. The system of claim 19, wherein the means for varying the wavefront divergence modulates a focal distance of the light for the voxels, so that the voxels comprising the image are at different focal distances generally corresponding to the different viewing distances of the objects in the image.

23-23. The system of claim 19, further comprising:

(a) a plurality of light sources, wherein a separate light source is provided for each of a plurality of different focal distances;

(b) means for modulating the plurality of light sources separately over time to produce an individual voxel stream for each separate light source;

(c) means for optically combining and superimposing images formed of individual voxel streams at different focal distances; and

(d) a two-dimensional scanner that scans the images that were superimposed, to produce the multi-focal three-dimensional visually perceptible image as a composite of images formed with the light from the plurality of light sources and at the plurality of different focal distances.

24-24. The system of claim 23, wherein the two-dimensional scanner comprises a resonant scanner.

25-25. The system of claim 23, wherein the composite of images enables objects at a greater focal distance to be seen through an image formed at a nearer focal distance that is semi-transparent.

26-26. The system of claim 23, wherein the composite of images enables reflections from objects at different focal distances to be visible in the image.

27-27. The system of claim 19, wherein the means for displaying include a first scanner for scanning the light over a first axis of the image, and a second scanner for scanning the light over a second axis of the image, wherein the first axis and the second axis are generally orthogonal to each other.

28-28. The system of claim 27, wherein the means for varying the wavefront divergence by modifying the focal distance of voxels comprising the image are disposed downstream of at least one of the first scanner and the second scanner.

29-29. The system of claim 27, wherein the means for varying the wavefront divergence by modifying the focal distance of voxels comprising the image is disposed upstream of both the first scanner and the second scanner.

30-30. The system of claim 19, wherein the means for varying the wavefront divergence comprises one of the following components:

(a) a fluid lens;

(b) a nematic liquid crystal spatial modulator;

(c) a ferroelectric liquid crystal spatial light modulator;

(d) an electro-optical polymer spatial light modulator; and

(e) a non-polymeric electro-optical spatial light modulator.

31-31. The system of claim 19, wherein the means for displaying the image comprise a head-mounted display.

32-32. The system of claim 19, further comprising means for monitoring a viewer'"'"'s eyes to detect at least one of an accommodation, a vergence, a pupil size, and an eye position while the image is being viewed, producing a feedback signal, wherein the means for displaying the image further modify the image as displayed in response to the feedback signal.

33-33. The system of claim 19, wherein the means for varying the wavefront divergence comprise:

(a) a plurality of different channels, each channel having a focal distance range that is a subset of a full focal distance range for the image to be displayed and being selected so that the full focal distance range of the image is encompassed by the plurality of different channels; and

(b) means for varying the focal distance in appropriate channels, within the focal distance range of the appropriate channels, where an appropriate channel is selected and the focal distance for the channel that is selected is varied within the focal distance range of the channel that is selected, based upon the depth map for the voxels being displayed by the channels in the image.

34-34. The system of claim 19, wherein the means for displaying the image comprise a display that is spaced apart from a viewer of the image and is not supported by a viewer'"'"'s head.

35-35. The system of claim 19, wherein to display the image, each voxel of the image is processed separately by the means for varying the wavefront divergence, based upon the two-dimensional image data and the fixation depth data.

36. A method for producing a multi-focal three-dimensional visually perceptible image in which a convergence of a viewer'"'"'s eye on an object in the image corresponds with an accommodation of the viewer'"'"'s eye on the object for objects at different fixation viewing distances within the image, comprising the steps of:

(a) using a depth map associated with voxels of the image to control a viewing distance of the voxels in the image, the depth map indicating a fixation viewing distance of objects in the image; and

(b) displaying the voxels of the image with different wavefront divergence based upon the fixation viewing distance of the voxels, so that objects with different fixation viewing distances require a viewer'"'"'s eyes to focus at different viewing distances and with a corresponding convergence, and so that objects disposed at a greater viewing distance and behind an object at a closer viewing distance are not visible through the voxels of the object at a closer viewing distance in the image, where the object at the closer viewing distance is intended to be completely opaque and not transparent.

37. The method of claim 36, wherein the step of displaying comprises the step of dynamically varying a focal distance for voxels in the image in response to the depth map, by employing at least one of:

(a) a variable focus mirror;

(b) a variable focus lens;

(c) a plurality of fixed focus mirrors of different focal distances;

(d) a plurality of fixed focus lenses of different focal distances;

(e) a plurality of different length optical paths for each of a plurality of different focal distance ranges; and

(f) at least one spatial light modulator.

38. The method of claim 36, wherein the step of displaying comprises the step of optically directing light used to form the image toward one of a display screen and toward retinas of the viewer'"'"'s eyes.

39. The method of claim 36, wherein the image is displayed as a monocular image that is expected to be visually perceivable by only one eye of a viewer.

40. The method of claim 36, wherein the step of displaying comprises the step of displaying the image as one of a monocular image and a stereo image using a head-mounted display.

36-36. A method for producing a multi-focal three-dimensional visually perceptible image in which a convergence of a viewer'"'"'s eye on an object in the image corresponds with an accommodation of the viewer'"'"'s eye on the object for objects at different fixation viewing distances within the image, comprising the steps of:

(a) using a depth map associated with voxels of the image to control a viewing distance of the voxels in the image, the depth map indicating a fixation viewing distance of objects in the image; and

(b) displaying the voxels of the image with different wavefront divergence based upon the fixation viewing distance of the voxels, so that objects with different fixation viewing distances require a viewer'"'"'s eyes to focus at different viewing distances and with a corresponding convergence, and so that objects disposed at a greater viewing distance and behind an object at a closer viewing distance are not visible through the voxels of the object at a closer viewing distance in the image, where the object at the closer viewing distance is intended to be completely opaque and not transparent.

37-37. The method of claim 36, wherein the step of displaying comprises the step of dynamically varying a focal distance for voxels in the image in response to the depth map, by employing at least one of:

(a) a variable focus mirror;

(b) a variable focus lens;

(c) a plurality of fixed focus mirrors of different focal distances;

(d) a plurality of fixed focus lenses of different focal distances;

(e) a plurality of different length optical paths for each of a plurality of different focal distance ranges; and

(f) at least-one spatial light modulator.

38-38. The method of claim 36, wherein the step of displaying comprises the step of optically directing light used to form the image toward one of a display screen and toward retinas of the viewer'"'"'s eyes.

39-39. The method of claim 36, wherein the image is displayed as a monocular image that is expected to be visually perceivable by only one eye of a viewer.

40-40. The method of claim 36, wherein the step of displaying comprises the step of displaying the image as one of a monocular image and a stereo image using a head-mounted display.