HMD Calibration with Direct Geometric Modeling

• US 20160012643A1
• Filed: 06/16/2015
• Published: 01/14/2016
• Est. Priority Date: 07/10/2014
• Status: Active Grant

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: Kezele, Irina, Chenchev, Margarit Simeonov, Yuan, Stella, Szeto, Simon, Abadpour, Arash
• Granted Patent: US 10,198,865 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 30/34 Stereoscopes providing a stereoscopic pair of separated images corresponding to parallactically displaced views of the same object, e.g. 3D slide viewers
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 13/156 Mixing image signals
• CPC Class Codes: H04N 13/275 from 3D object models, e.g. computer-generated stereoscopic image signals
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays

Abstract

An optical see-through (OST) head-mounted display (HMD) uses a calibration matrix having a fixed sub-set of adjustable parameters within all its parameters. Initial values for the calibration matrix are based on a model head. A predefined set of incremental adjustment values is provided for each adjustable parameter. During calibration, the calibration matrix is cycled through its predefined incremental parameter changes, and a virtual object is projected for each incremental change. The resultant projected virtual object is aligned to a reference real object, and the projected virtual object having the best alignment is identified. The setting values of the calibration matrix that resulted in the best aligned virtual object are deemed the final calibration matrix to be used with the OST HMD.

Claims

1. A calibration method of an optical see-through (OST) head-mounted display (HMD) of an augmented reality (AR) system, comprising:

providing a real target object in a scene where the OST HMD is to be used, said real target object being of known dimensions, location and orientation within said scene;

defining a default calibration matrix set based on a predefined head model;

generating a virtual object representation of the real target object using the default calibration matrix set, and projecting a view of the virtual object;

centrally aligning the projected view of virtual object onto the real target object as much as possible;

cycling through a predefined number of incremental changes in at least one predefined parameter within said default calibration matrix set;

for each incremental change of the predefined parameter, generating and projecting a new virtual object, re-aligning the new virtual object to the real target object, and comparing the current re-alignment of the current new virtual object to a previous alignment of a previously projected virtual object;

identifying the projected virtual object having the best alignment with the real target object and designated it a calibrated virtual object; and

setting the OST HMD to use the calibration matrix set with the value of the predefined parameter that corresponds to the calibrated virtual object.

2. The method of claim 1, wherein the OST HMD is a stereo AR system providing a binocular image of the projected virtual object.

3. The method of claim 2, wherein:

the binocular image is comprised of a left image and a right image;

the step of re-aligning the new virtual object to the real target object is conducted separately for each of the left image and right image;

and the calibrated virtual object is further defined as the projected virtual object whose left image virtual object projection and right image virtual object projection best aligned with each other.

4. The method of claim 1, wherein:

the default calibration matrix set includes an extrinsic parameter matrix and an intrinsic parameter matrix, both matrices being defined from direct geometric measures of the OST HMD and head model; and

the extrinsic parameter matrix and intrinsic parameter matrix are based on a pin-hole camera model.

5. The method of claim 4, wherein updates to the extrinsic parameter matrix are obtained from direct geometric measures of the 3D pose (3D rotation+3D translation) of a projected and aligned virtual object by using the known dimensions, location and orientation of the real target object and a coordinate system of the real target object.

6. The method of claim 5, wherein values in the coordinate system of the real target object are converted to values in a coordinate system of the projected virtual object.

7. The method of claim 5, wherein each projected virtual object is generated using a virtual camera.

8. The method of claim 4, wherein:

the projected virtual object is based on the pin-hole camera model;

parameters within the intrinsic parameter matrix are defined using the following geometric measures;

a distance from a center of the virtual pin-hole camera to a center of a virtual image plane of the virtual pin-hole camera is used to define a focal length; and

the center position of the virtual image plane with respect to the head model, and an image resolution of the projected image to define a center pixel position of the projected image, and pixel density of the projected virtual object.

9. The method of claim 8, wherein:

the OST HMD is a stereo AR system providing a binocular image of the projected virtual object;

the binocular image is comprised of a left image and a right image, each having a respective extrinsic parameter matrix and an intrinsic parameter matrix; and

the defined focal length is used for both left image and right image.

10. The method of claim 8, wherein said one predefined parameter is an interpupillary distance (IPD), and an optimal (IPD) is found by:

conducting the step of re-aligning the new virtual object to the real target object separately for each of the left image and right image;

and identifying the parameter value that results in the re-aligned new virtual object of the left image being aligned to the re-aligned new virtual object of the right image.

11. The method of claim 10, wherein the optimal IPD does not match a users true IPD.

12. The method of claim 11, wherein the cycling through the predefined number of incremental changes is responsive to a user-controlled input.

13. The method of claim 1, wherein:

said one predefined parameter is one of a plurality of said predefined parameters;

incremental changes in said plurality of predefined parameters result in translation of the projected virtual object, and a difference in the orientation of the projected virtual object relative to the real target object is corrected by adjusting the translation of the projected virtual object.

14. A non-transient computer readable media embodying instructions to execute the method of claim 1 by a computing device.