SYSTEM, METHOD, AND COMPUTER PROGRAM PRODUCT TO PRODUCE IMAGES FOR A NEAR-EYE LIGHT FIELD DISPLAY HAVING A DEFECT

• US 20140340390A1
• Filed: 07/03/2014
• Published: 11/20/2014
• Est. Priority Date: 05/17/2013
• Status: Active Grant

• Current Assignee: NVIDIA Corporation
• Original Assignee: NVIDIA Corporation
• Inventors: Luebke, David Patrick, Lanman, Douglas Robert
• Granted Patent: US 9,519,144 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G09G 3/00 Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
• CPC Class Codes: H04N 13/106 Processing image signals for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/398 Synchronisation thereof; Control thereof

Abstract

A system, method, and computer program product are provided for producing images for a near-eye light field display. Defect information for a first pixel of a microdisplay of a near-eye light field display device is received and a second pixel of the microdisplay is identified, where the first pixel and the second pixel contribute to a portion of the retinal image. Based on the defect information, a value of the second pixel within an array of elemental images is modified to produce a corrected array of elemental images for display by the microdisplay. An optical apparatus of the near-eye light field display device may, for example, be a microlens of a microlens array positioned between a viewer and an emissive microdisplay or a pinlight of a pinlight array positioned behind a transmissive microdisplay relative to the viewer.

Claims

1. A method comprising:

receiving defect information for a first pixel of a microdisplay of a near-eye light field display device, the first pixel contributing to a portion of a retinal image;

identifying a second pixel of the microdisplay that also contributes to the portion of the retinal image; and

modifying, based on the defect information, a value of the second pixel within an array of elemental images to produce a corrected array of elemental images for display by the microdisplay.

2. The method of claim 1, wherein the defect information indicates that the first pixel is a hot pixel comprising at least one continuously illuminated subpixel.

3. The method of claim 1, wherein the defect information indicates that the first pixel is a dead pixel comprising at least one continuously dark subpixel.

4. The method of claim 1, wherein identifying the second pixel comprises:

identifying a first ray defined by the first pixel and a first optical apparatus of the near-eye light field display device;

determining the portion of the retinal image intersected by the first ray; and

identifying a second ray defined by the portion of the retinal image and a second optical apparatus of the near-eye light field display device, the second ray intersecting the second pixel.

5. The method of claim 4, further comprising identifying additional rays defined by the portion of the retinal image and additional optical apparatus of the near-eye light field display device, the additional rays intersecting additional pixels that also contribute to the portion of the retinal image.

6. The method of claim 1, wherein the retinal image appears to a user of the near-eye light field display device at a two-dimensional virtual display plane beyond a near focus plane.

7. The method of claim 6, wherein a depth of the two-dimensional virtual display plane relative to the user'"'"'s eye is based on an eyeglass prescription.

8. The method of claim 1, further comprising storing the modified value of the second pixel in a data structure for the corrected array of elemental images in a memory.

9. The method of claim 1, wherein the corrected array of elemental images corresponds to a left-eye image or a right-eye image of a stereoscopic image pair.

10. The method of claim 1, wherein multiple processing threads are configured to execute a fragment shader program to compute modified values for the second pixel and additional pixels of the corrected array of elemental images in parallel.

11. The method of claim 1, wherein modifying the value of the second pixel comprises minimizing an error between an ideal retinal image corresponding to the array of elemental images and the retinal image.

12. The method of claim 1, wherein each elemental image in the array of elemental images is a view of the retinal image corresponding to one optical apparatus within an array of optical apparatus included in the near-eye light field display device.

13. The method of claim 12, wherein the array of optical apparatus is an array of pinlights.

14. The method of claim 12, wherein the array of optical apparatus is an array of microlenses.

16. The system of claim 1, wherein the processor is further configured to identify the second pixel by:

identifying a first ray defined by the first pixel and a first optical apparatus of the near-eye light field display device;

determining the portion of the retinal image intersected by the first ray; and

identifying a second ray defined by the portion of the retinal image and a second optical apparatus of the near-eye light field display device, the second ray intersecting the second pixel.

17. The system of claim 16, wherein the processor is further configured to identify additional rays defined by the portion of the retinal image and additional optical apparatus of the near-eye light field display device, the additional rays intersecting additional pixels that also contribute to the portion of the retinal image

15. A system comprising:

microdisplay of a near-eye light field display device, the microdisplay positioned between a near focus plane and a retina; and

a processor configured to;

receive defect information for a first pixel of the microdisplay, the first pixel contributing to a portion of a retinal image;

identify a second pixel of the microdisplay that also contributes to the portion of the retinal image; and

adapt, based on the defect information, a value of the second pixel within an array of elemental images to produce a corrected array of elemental images for display by the microdisplay.

18. The system of claim 15, further comprising an optical apparatus positioned between the microdisplay and the retina.

19. The system of claim 15, wherein the processor is further configured to process multiple processing threads that each execute a fragment shader program to compute values for the second pixel and additional pixels of the corrected array of elemental images in parallel.

20. A computer-readable storage medium storing instructions that, when executed by a processor, causes the processor to perform steps comprising:

receiving defect information for a first pixel of a microdisplay of a near-eye light field display device, the first pixel contributing to a portion of a retinal image;

identifying a second pixel of the microdisplay that also contributes to the portion of the retinal image; and

modifying, based on the defect information, a value of the second pixel within an array of elemental images to produce a corrected array of elemental images for display by the microdisplay.