Display with gaze-adaptive resolution enhancement

• US 10,636,340 B2
• Filed: 08/03/2018
• Issued: 04/28/2020
• Est. Priority Date: 04/16/2018
• Status: Active Grant

• Current Assignee: META Platforms Technologies LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Original Assignee: Facebook Technologies, LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Inventors: Lu, Lu, Peng, Fenglin, Mounier, Michael Sean, MacKenzie, Kevin James, Lam, Wai Sze Tiffany
• Primary Examiner(s): Patel, Nitin
• Assistant Examiner(s): Onyekaba, Amy
• Application Number:
  US16/054,962
• Publication Number:
  US 20190318677A1
• Time in Patent Office: 634 Days
• CPC Class Codes: G02B 2027/0147 comprising a device modifying the resolution of the displayed image
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/0101 characterised by optical features G02B27/0172 takes precedence
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/1066 for enhancing image performance, like resolution, pixel numbers, dual magnifications or dynamic range, by tiling, slicing or overlapping fields of view
• CPC Class Codes: G02B 30/25 using polarisation techniques
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G09G 2310/04 Partial updating of the display screen
• CPC Class Codes: G09G 2320/0285 using tables for spatial correction of display data
• CPC Class Codes: G09G 2340/0407 Resolution change, inclusive of the use of different resolutions for different screen areas
• CPC Class Codes: G09G 2340/0435 Change or adaptation of the frame rate of the video stream
• CPC Class Codes: G09G 2340/0464 Positioning
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 3/001 using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background slide projectors per se G03B23/00 = 42 HP
• CPC Class Codes: G09G 3/002 to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT
• CPC Class Codes: G09G 3/003 to produce spatial visual effects
• CPC Class Codes: G09G 3/007 Use of pixel shift techniques, e.g. by mechanical shift of the physical pixels or by optical shift of the perceived pixels
• CPC Class Codes: G09G 3/3611 : Control of matrices with row and column drivers

Abstract

A display apparatus includes a pixelated display, an eye tracking system, and a segmented resolution enhancing device coupled to the eye tracking system and operable to selectively enhance effective image resolution for the electronic display in the gaze direction. The segmented resolution enhancing device may include a polarization switch in series with a polarization gratings, at least one of which is segmented, for shifting image pixels in the gaze direction between offset positions while displaying a frame.

Claims

1. An apparatus comprising:

an electronic display comprising a pixel array configured to display a frame, the pixel array comprising a plurality of display pixels;

an eye tracking device configured to determine a gaze direction of a user of the electronic display; and

,

an optical block for forming a frame image from the frame displayed by the electronic display, the optical block comprising a segmented resolution enhancing device (SRED) disposed in an optical path of light from the display pixels and operatively coupled to the eye tracking device, the frame image comprising an array of image pixels disposed at an image pixel pitch p;

wherein the SRED comprises a plurality of device segments configured to be individually activated in response to a signal from the eye tracking device, each device segment being optically coupled to a different group of display pixels of the pixel array, for selectively enhancing effective resolution for a portion of the frame image in the gaze direction;

wherein each device segment from the plurality of device segments comprises a light steering switch (LSS) configured to switch, upon activation in response to the signal, between two or more light steering states while the frame is being displayed, so as to provide a sequence of offset images for the portion of the frame image in the gaze direction.

2. The apparatus of claim 1 comprising a controller configured to activate a selected device segment in the gaze direction from the array of device segments based on the signal from the eye tracking device.

3. The apparatus of claim 2 comprising a display processor configured to:

provide pixel data to the electronic display for displaying the frame;

identify a selected group of display pixels, from the different groups of display pixels of the electronic display, that is optically coupled to the selected device segment; and

,

perform a selective pixel update of the selected group of pixels with new pixel data at least once while the frame is being displayed by the electronic display.

4. The apparatus of claim 3 wherein the selective pixel update is performed without updating a peripheral group of display pixels located outside of the selected group of display pixels with new pixel data while the frame is being displayed.

5. The apparatus of claim 1 wherein the selective pixel update is performed in coordination with switching the LSS of the selected device segment.

6. The apparatus of claim 5 wherein the LSS is operable, upon activation by the controller, to provide the sequence of mutually offset images comprising two images having a spatial offset s therebetween equal to a fraction of the image pixel pitch p.

7. The apparatus of claim 6 wherein the frame image comprises interstitial areas between adjacent image pixels that are at least equal to the image pixels in size, and wherein the spatial offset s is such that at least some of the image pixels of a first of the two mutually offset images are located in the interstitial spaces of the second of the mutually offset images.

8. The apparatus of claim 6 wherein the display processor is configured to selectively sub-sample input frame data for the selected group of display pixels at two or more sampling grids with a sampling offset therebetween matching the spatial offset s.

9. The apparatus of claim 1 wherein the SRED comprises:

a first polarization grating, and

a first polarization switch disposed in sequence with the first polarization grating,

wherein at least one of the first polarization grating or the first polarization switch comprises a plurality of first individually addressable segments defining the plurality of device segments, and

wherein at least one of the electronic display or the optics block comprises a polarization element configured to provide polarized pixel light from the electronic display to the SRED.

10. The apparatus of claim 9 wherein the electronic display is configured to transmit polarized light.

11. The apparatus of claim 9 comprising polarization device disposed optically between the electronic display and the SRED and configured to provide light from the electronic display to the SRED in a circular polarization state.

12. The apparatus of claim 9 wherein at least one of the first polarization switch or the first polarization grating comprises an active LC device comprising an LC layer disposed between two electrodes, at least one of which is transparent, and wherein at least one of the two electrodes comprises a plurality of electrically separate electrode segments.

13. The apparatus of claim 9 wherein the SRED further comprises a second polarization grating disposed in sequence with a second polarization switch, and wherein at least one of the second polarization grating or the second polarization switch comprises a plurality of second individually addressable segments matching the first individually addressable segments of the at least one of the first polarization grating or the first polarization switch.

14. The apparatus of claim 13 wherein the second polarization grating is disposed with a rotational offset relative to the first polarization grating.

15. The apparatus of claim 13 wherein the rotational offset is substantially 90°

.

16. The apparatus of claim 13 wherein at least one of the first polarization grating or the second polarization grating comprises at least one of:

a liquid crystal (LC) Pancharatnam Berry Phase (PBP) grating or an LC volume holographic grating.

17. The apparatus of claim 13 wherein at least one of the first polarization grating or the second polarization grating comprises at least one of:

a segmented active liquid crystal LC PBP grating, or a segmented active LC holographic grating.

18. The apparatus of claim 13 wherein at least one of the first polarization switch or the second polarization switch comprises an active liquid crystal (LC) half-wave plate (HWP).