Variable resolution virtual reality display system

• US 10,083,538 B2
• Filed: 03/28/2017
• Issued: 09/25/2018
• Est. Priority Date: 06/30/2015
• Status: Active Grant

• Current Assignee: KAYA Dynamics LLC
• Original Assignee: Ariadne's Thread (Usa), Inc.
• Inventors: Li, Adam
• Primary Examiner(s): Du, Haixia
• Application Number:
  US15/472,243
• Publication Number:
  US 20170200304A1
• Time in Patent Office: 546 Days
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0147 comprising a device modifying the resolution of the displayed image
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0181 Adaptation to the pilot/driver
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 15/10 Geometric effects
• CPC Class Codes: G06T 15/20 : Perspective computation
• CPC Class Codes: G06T 19/00 : Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/006 : Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2215/16 : Using real world measurements to influence rendering
• CPC Class Codes: G06T 2219/2016 : Rotation, translation, scaling
• CPC Class Codes: G06T 3/18 : Image warping, e.g. rearranging pixels individually
• CPC Class Codes: G06T 3/40 : Scaling of whole images or parts thereof, e.g. expanding or contracting

Abstract

A virtual reality display system that renders images at different resolutions in different parts of a display. Reduces rendering latency by rendering at a lower resolution in selected regions, for example on the sides of a display where human vision has lower resolution than in the center. Pixels in low resolution regions are combined into grid elements, and rendering may generate grid element values rather than individual pixel values. Rendering may use ray casting, rasterization, or both. Variable resolution rendering may be combined with variable level of detail geometry models to further reduce rendering time. Selected objects may be designed as high resolution objects that are rendered at a high resolution even in low resolution display regions.

Claims

1. A variable resolution virtual reality display system comprising:

at least one display viewable by a user and partitioned into at least two display regions having different resolutions;

a pose analyzer that calculates a pose of one or more body parts of said user;

a scene renderer coupled to said at least one display and to said pose analyzer, wherein said scene renderer

receives said pose from said pose analyzer; and

,

based on said pose, calculates a rendered image for each of said at least two display regions; and

,

an image warper coupled to said at least one display, said scene renderer, and said pose analyzer, wherein said image warper

receives said pose from said pose analyzer;

calculates a change in pose from said pose; and

,

when said change in pose is below a threshold value,

generates a rerendering approximation of said rendered image for each of said at least two display regions based on said change in pose; and

,

modifies one or more pixels of said at least one display based on said rerendering approximation.

2. The system of claim 1, wherein

said at least two display regions comprise

a high resolution center region located in a center of a field of view of said user;

a low resolution left region located in a left portion of said field of view; and

,

a low resolution right region located in a right portion of said field of view.

3. The system of claim 1, further comprising a 3D model of a scene, wherein

at least one object in said 3D model comprises at least two geometry models at different levels of detail;

for each display region of said at least two display regions, said scene renderer

selects a level of detail for said at least one object; and

,

calculates said rendered image using a geometry model for said level of detail associated with said at least one object.

4. The system of claim 1, further comprising

at least one sensor that generates sensor data that measures one or more aspects of said pose of said one or more body parts of said user;

and,

wherein said pose analyzer calculates said pose of said one or more body parts of said user based on said sensor data generated by said at least one sensor.

5. The system of claim 1, wherein said rerendering approximation comprises

calculating a pixel translation vector;

and,

translating one or more pixels of said rendered image for each of said at least two display regions by said pixel translation vector.

6. The system of claim 5, wherein calculating said pixel translation vector comprises

approximating said change in pose as a rotation around a unit vector {circumflex over (ω

)} comprising {circumflex over (ω

)}_y and {circumflex over (ω

)}_x by an angle Δ

θ

;

calculating a spatial translation vector ({circumflex over (ω

)}_yΔ

θ

,−

{circumflex over (ω

)}_xΔ

θ

);

calculating a scaling factor to convert spatial distances to pixels based on pixel dimensions and fields of view of said rendered image for each of said at least two display regions; and

,

calculating said pixel translation vector by scaling said spatial translation vector by said scaling factor.