Dynamic distortion correction for optical compensation

• US 10,241,545 B1
• Filed: 06/01/2017
• Issued: 03/26/2019
• Est. Priority Date: 06/01/2017
• Status: Active Grant

• Current Assignee: META Platforms Technologies LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Original Assignee: Facebook Technologies, LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Inventors: Richards, Evan M., Lewis, Brant Carlton, Yoon, Youngshik, Shen, Shizhe
• Primary Examiner(s): Schnurr, John R
• Application Number:
  US15/611,720
• Time in Patent Office: 663 Days
• CPC Class Codes: G02B 2027/011 comprising device for correcting geometrical aberrations, distortion
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/1637 Details related to the display arrangement, including those related to the mounting of the display in the housing
• CPC Class Codes: G06F 1/1675 Miscellaneous details related to the relative movement between the different enclosures or enclosure parts
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G09G 2320/0261 in the context of movement of objects on the screen or movement of the observer relative to the screen
• CPC Class Codes: G09G 2320/0693 Calibration of display systems
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T

Abstract

A distortion calibration system generates a distortion map for a head-mounted display (HMD). The system includes a camera that takes pictures of a test pattern displayed by the HMD. The images are taken at different camera positions and/or states of the HMD. The system determines a distortion map using the captured images, and uploads it to the HMD as part of, e.g., an optical model. The HMD comprises an electronic display, an eye tracking unit, an optics block, and a module. The module estimates distortion values based on eye tracking information from the eye tracking unit, an optical model, and a state of the HMD (e.g., a distance between the optics block and the electronic display). The module determines an adjusted state of the HMD using the estimated distortion values, the eye tracking information, and the optical model, and adjusts the state of the HMD to the adjusted state.

Claims

1. A head mounted display (HMD) comprising:

an electronic display configured to present images to a user of the HMD;

an eye tracking unit configured to determine eye tracking information for one or more eyes of the user as the user views the images presented by the electronic display;

an optics block configured to modify the images presented by the electronic display when viewed by the user;

a distortion correction module configured to;

estimate one or more distortion values by inputting the eye tracking information and a state of the HMD into an optical model, wherein the HMD includes a plurality of states, and each state is associated with a different image plane for the electronic display,

determine an adjusted state of the HMD using one or more of the estimated distortion values, the eye tracking information, and the optical model, the adjusted state being different from the state, and

adjust the optical element in accordance with the adjusted state.

2. The HMD of claim 1, wherein the distortion correction module is further configured to determine the adjusted state by:

using the optical model to determine a state of the HMD at which associated distortion values are less than a threshold distortion value.

3. The HMD of claim 1, wherein the optical model is a distortion map that maps values for one or more types of distortion to the eye tracking information for each of the plurality of states of the HMD.

4. The HMD of claim 1, wherein the optical model is a ray-tracing program that dynamically determines values for one or more types of distortion using the eye tracking information for each of the plurality of states of the HMD.

5. The HMD of claim 1, wherein the distortion correction module is further configured to pre-distort the images presented by the electronic display to reduce the estimated distortion values based on the eye tracking information.

6. The distortion calibration system of claim 1, wherein the distance between the electronic display and the optical element is adjusted by the optical element changing its position relative to the electronic display.

7. The distortion calibration system of claim 1, wherein the distance between the electronic display and the optical element is adjusted by the electronic display changing its position relative to the optical element.