AMBIENT ELECTROMAGNETIC DISTORTION CORRECTION FOR ELECTROMAGNETIC TRACKING

• US 20200133405A1
• Filed: 10/16/2019
• Published: 04/30/2020
• Est. Priority Date: 10/26/2018
• Status: Active Grant

• Current Assignee: Magic Leap, Inc.
• Original Assignee: Magic Leap, Inc.
• Inventors: Chan, Richmond B.
• Granted Patent: US 11,157,090 B2
• Time in Patent Office: Days
• CPC Class Codes: G01R 33/0082 Compensation, e.g. compensating for temperature changes
• CPC Class Codes: G01R 33/02 Measuring direction or magnitude of magnetic fields or magnetic flux G01R33/20 takes precedence
• CPC Class Codes: G02B 27/0179 Display position adjusting means not related to the information to be displayed
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

Head-mounted augmented reality (AR) devices can track pose of a wearer'"'"'s head to provide a three-dimensional virtual representation of objects in the wearer'"'"'s environment. An electromagnetic (EM) tracking system can track head or body pose. A handheld user input device can include an EM emitter that generates an EM field, and the head-mounted AR device can include an EM sensor that senses the EM field (e.g., for determining head pose). The generated EM field may be distorted due to nearby electrical conductors or ferromagnetic materials, which may lead to error in the determined pose. Systems and methods are disclosed that measure the degree of EM distortion, as well as correct for the EM distortion. The EM distortion correction may be performed in real time by the EM tracking system without the need for additional data from imaging cameras or other sensors.

Claims

1. An augmented reality display system comprising:

a display configured to project virtual images to eyes of a wearer;

a frame configured to mount the display in front of the eyes of the wearer;

an electromagnetic (EM) emitter configured to generate a magnetic field;

an EM sensor configured to sense the magnetic field and provide EM sensor data, wherein one of the EM emitter or the EM sensor is mechanically coupled to the frame and the other of the EM emitter or the EM sensor is mechanically coupled to a component of the augmented reality display system that is independently movable relative to the frame;

a hardware processor programmed to;

receive EM sensor data from the EM sensor, the EM sensor data comprising an EM field matrix;

detect presence of EM distortion based at least partly on the EM field matrix;

calculate distortion compensating matrices;

calculate, based at least partly on the distortion compensating matrices, an undistorted EM field matrix;

calculate, based at least partly on the undistorted EM field matrix, a pose of the EM sensor; and

cause the display to project the virtual images based at least in part on the calculated pose.

2. The augmented reality display system of claim 1, wherein the EM sensor and the EM emitter each comprise three mutually orthogonal coils.

3. The augmented reality display system of claim 1, wherein the pose comprises a six degree of freedom pose having three spatial coordinates and three angular coordinates.

4. The augmented reality display system of claim 1, wherein to detect the presence of EM distortion, the hardware processor is programmed to calculate a deviation of eigenvalues of the EM field matrix from eigenvalues for an undistorted EM field matrix.

5. The augmented reality display system of claim 1, wherein to calculate the distortion compensating matrices, the hardware processor is programmed to apply constraints that the determinant of each of the distortion compensating matrices is close to unity and eigenvalues of the undistorted EM field matrix are proportional to [1, ½

, ½

].

6. The augmented reality display system of claim 5, wherein the hardware processor is programmed to perform a singular value decomposition (SVD) to calculate the eigenvalues of the undistorted EM field matrix.

7. The augmented reality display system of claim 5, wherein the constraints comprise the eigenvalues of the undistorted EM field matrix being close to [1, ½

, ½

].

8. The augmented reality display system of claim 1, wherein to calculate the pose of the EM sensor, the hardware processor is programmed to perform a singular value decomposition (SVD) of the undistorted EM field matrix.

9. The augmented reality display system of claim 8, wherein the hardware processor is programmed to calculate a three degree of freedom position of the EM sensor with respect to the EM emitter from the SVD.

10. The augmented reality display system of claim 1, wherein the hardware processor is programmed to calculate a three degree of freedom orientation of the EM sensor with respect to the EM emitter based at least partly on the undistorted EM field matrix, a diagonal matrix, and a rotation matrix based on azimuth and pitch.

11. The augmented reality display system of claim 1, wherein to calculate the pose of the EM sensor, the hardware processor is programmed to calculate the pose in a world frame of the augmented reality display system.

12. The augmented reality display system of claim 11, wherein to calculate the pose in the world frame, the hardware processor is programmed to apply spherical harmonics or higher order polynomials.

13. The augmented reality display system of claim 1, wherein the hardware processor is programmed to calculate the distortion compensating matrices only when the detected EM distortion is above a first threshold or a change in the detected EM distortion is above a second threshold.

14. The augmented reality display system of claim 1, wherein

the component of the augmented reality display system comprises a user-input totem,

the EM sensor is mechanically coupled to the frame, and

the EM emitter is mechanically coupled to the user-input totem.

15. The augmented reality display system of claim 14, further comprising an outward-facing camera mechanically coupled to the frame.

16. The augmented reality display system of claim 15, wherein the hardware processor is further programmed to:

analyze images obtained by the outward-facing camera;

determine, based at least partly on the analyzed images, a pose of the user-input totem; and

calculate the distortion compensating matrices based at least in part on the determined pose of the user-input totem.

17. The augmented reality display system of claim 14, wherein the user-input totem further comprises a light source or an optically-recognizable fiducial marker configured to assist determining pose of the user-input totem.

18. The augmented reality display system of claim 1, wherein the hardware processor is further programmed to calculate a degree of EM distortion in the ambient environment using a metric based on eigenvalues of the EM field matrix.

19. The augmented reality display system of claim 18, wherein the metric is based on a combination of differences of the eigenvalues from a reference value.

20. The augmented reality display system of claim 19, wherein the eigenvalues are normalized based on a determinant of the EM field matrix.