Stereo-based calibration apparatus

• US 9,805,512 B1
• Filed: 11/13/2015
• Issued: 10/31/2017
• Est. Priority Date: 11/13/2015
• Status: Active Grant

• Current Assignee: META Platforms Technologies LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Original Assignee: Oculus VR, LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Inventors: Katz, Dov, Dolson, Jennifer Leigh, Hallam, Simon, Levin, Kieran Tobias, Vaughan, Eric Loren, Ivmark, Klas Petter, Melim, Andrew
• Primary Examiner(s): Hoang, Peter
• Application Number:
  US14/941,334
• Time in Patent Office: 718 Days
• Field of Search: None
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06T 7/571 from focus
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A virtual reality (VR) headset calibration system calibrates a VR headset, which includes a plurality of locators and an inertial measurement unit (IMU) generating output signals indicative of motion of the VR headset. The system comprises a calibration controller configured to receive a headset model of the VR headset that identifies expected positions of each of the locators. The controller controls cameras to capture images of the VR headset while the headset is moved along a predetermined path. The images detect actual positions of the locators during the movement along the predetermined path. Calibration parameters for the locators are generated based on differences between the actual positions and the expected positions. Calibration parameters for the IMU are generated based on the calibration parameters for the locators and differences between expected and actual signals output by the IMU. The calibration parameters are stored to the VR headset.

Claims

1. A method comprising:

accessing a memory that stores a set of expected positions of a plurality of locators on a virtual reality (VR) headset;

capturing a plurality of images of the VR headset in a plurality of positions as the VR headset is moved along a predetermined path through a space imaged by a plurality of cameras, the captured images detecting actual positions of the plurality of locators on the VR headset;

using images captured at each position of the VR headset;

identifying the respective position of the VR headset in the space;

determining positions of at least a subset of the locators in the space based on the images;

computing a difference between an actual position of each locator in the subset on the VR headset and the expected position of the respective locator based on the determined position of the locator in the space and the identified position of the VR headset;

generating calibration parameters for the plurality of locators based on the computed differences; and

storing the calibration parameters for the plurality of locators to the VR headset, the calibration parameters usable to identify positions and motion of the VR headset during use of the VR headset.

2. The method of claim 1, wherein determining the positions of the subset of locators in the space comprises, for a locator of the subset:

triangulating the position in the space of the locator using at least one image captured simultaneously by two or more of the cameras; and

determining the actual position of the locator based on the triangulated position of the locator in the space and the identified position of the VR headset.

3. The method of claim 1, wherein the calibration parameters for the plurality of locators comprise a calibration parameter representing a transformation that when applied to a position of a locator in the space yields the expected position of the respective locator on the VR headset.

4. The method of claim 1, wherein the VR headset further comprises an inertial measurement unit (IMU) generating output signals indicative of motion of the VR headset, the method further comprising:

generating calibration parameters for the IMU based on the calibration parameters for the plurality of locators and differences between expected signals output by the IMU and actual signals output by the IMU; and

storing the calibration parameters for the IMU to the VR headset, the calibration parameters for the IMU usable to identify motion of the VR headset during use of the VR headset.

5. The method of claim 4, further comprising:

capturing a second plurality of images of the VR headset while the VR headset is moved in one or more predetermined manners; and

determining positions of the VR headset in the space using the second plurality of images.

6. The method of claim 5, wherein the one or more predetermined manners of movement comprise a rotation of the VR headset to each of a plurality of orientations in the space, and wherein generating the calibration parameters for the IMU comprises:

identifying the orientations of the VR headset based on the captured images and the calibration parameters for the plurality of locators;

receiving the actual signals output by the IMU at each of the plurality of orientations of the VR headset;

determining the expected IMU signals based on the identified orientations; and

identifying for each of the orientations, the difference between the actual signals output from the IMU at the orientation and the expected signals.

7. The method of claim 6, wherein identifying the orientations of the VR headset comprises, for an orientation of the VR headset:

determining positions of a plurality of the locators of the VR headset in the space using at least one of the captured images;

translating the determined positions of the locators in the space to the expected positions of the corresponding locators based on the calibration parameters; and

determining the orientation based on the expected positions.

8. The method of claim 6, wherein the calibration parameters for the IMU comprise a calibration parameter representing a transformation that when applied to an actual signal output by the IMU yields an expected signal.

9. The method of claim 5, wherein the one or more predetermined manners of movement comprises rotating the VR headset around one or more axes of the VR headset, and wherein generating the calibration parameters for the IMU comprises:

receiving actual signals output by the IMU while the VR headset is rotated;

calculating a rate of rotation of the IMU around each of the one or more axes based on the second plurality of captured images and the calibration parameters for the plurality of locators;

determining the expected IMU signals based on the calculated rates of rotation; and

identifying for the rotation around each of the one or more axes, the difference between the actual signals and the expected signals.

10. A system comprising:

a plurality of cameras;

an imaging space imaged by the plurality of cameras;

a support arm configured to couple to a virtual reality (VR) headset and move the VR headset along a predetermined path within the imaging space, the VR headset including a plurality of locators; and

a calibration controller configured to;

accessing a memory that stores a set of expected positions of each of the locators on the VR headset;

control the cameras to capture a plurality of images of the VR headset in each of a plurality of positions while the VR headset is moved along the predetermined path, the captured images detecting actual positions of the plurality of locators on the VR headset;

using images captured at each position of the VR headset;

identify the respective position of the VR headset in the space;

determine positions of at least a subset of the locators in the space based on the images;

compute a difference between an actual position of each locator in the subset on the VR headset and the expected position of the respective locator based on the determined position of the locator in the space and the identified position of the VR headset;

generate calibration parameters for the plurality of locators based on the computed differences; and

store the calibration parameters for the plurality of locators to the VR headset, the calibration parameters usable to identify positions of the VR headset during use of the VR headset.

11. The system of claim 10, wherein the calibration controller is configured to determine the positions of the subset of locators in the space by, for a locator of the subset:

triangulating the position in the space of the locator using at least one image captured simultaneously by two or more of the cameras; and

determining the actual position of the locator based on the triangulated position of the locator in the space and the identified position of the VR headset.

12. The system of claim 10, wherein the calibration parameters for the plurality of locators comprise a calibration parameter representing a transformation that when applied to a position of a locator in the space yields the expected position of the respective locator on the VR headset.

13. The system of claim 10, wherein the VR headset further comprises an inertial measurement unit (IMU) generating output signals indicative of motion of the VR headset, and wherein the calibration controller is further configured to:

generate calibration parameters for the IMU based on the calibration parameters for the plurality of locators and differences between expected signals output by the IMU and actual signals output by the IMU; and

store the calibration parameters for the IMU to the VR headset, the calibration parameters for the IMU usable to identify motion of the VR headset during use of the VR headset.

14. The system of claim 13, wherein the calibration controller is further configured to:

control the cameras to capture a second plurality of images of the VR headset while the VR headset is moved in one or more predetermined manners; and

determine positions of the VR headset in the space using the second plurality of images.

15. The system of claim 13, wherein the one or more predetermined manners of movement comprise a rotation of the VR headset to each of a plurality of orientations in the space, and wherein the calibration controller is configured to generate the calibration parameters for the IMU by:

identifying the orientations of the VR headset based on the captured images and the calibration parameters for the plurality of locators;

receiving the actual signals output by the IMU at each of the plurality of orientations of the VR headset;

determining the expected IMU signals based on the identified orientations; and

identifying for each of the orientations, the difference between the actual signals output from the IMU at the orientation and the expected signals.

16. The system of claim 15, wherein the calibration controller is configured to identify the orientations of the VR headset by:

determining, for an orientation of the VR headset, positions of a plurality of the locators of the VR headset in the space using at least one of the captured images;

translating the determined positions of the locators in the space to the expected positions of the corresponding locators based on the calibration parameters; and

determining the orientation based on the expected positions.

17. The system of claim 13, wherein the calibration parameters for the IMU comprise a calibration parameter representing a transformation that when applied to an actual signal output by the IMU yields an expected signal.

18. The system of claim 13, wherein the one or more predetermined manners of movement comprises rotating the VR headset around one or more axes of the VR headset, and wherein generating the calibration parameters for the IMU comprises:

receiving actual signals output by the IMU while the VR headset is rotated;

calculating a rate of rotation of the IMU around each of the one or more axes based on the second plurality of captured images and the calibration parameters for the plurality of locators;

determining the expected IMU signals based on the calculated rates of rotation; and

identifying for the rotation around each of the one or more axes, the difference between the actual signals and the expected signals.

19. The system of claim 10, wherein the plurality of cameras each has a field of view, and wherein the imaging space comprises regions of overlap of the fields of view of at least two of the cameras.

20. The system of claim 10, wherein the support arm couples to the VR headset without occluding the locators.