User-mounted device calibration using external data

• US 9,588,935 B2
• Filed: 12/03/2012
• Issued: 03/07/2017
• Est. Priority Date: 06/14/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Liu, James Chia-Ming, Vaught, Benjamin Isaac, Malabuyo, Paolo
• Primary Examiner(s): Suglo, Janet
• Application Number:
  US13/692,855
• Publication Number:
  US 20130096863A1
• Time in Patent Office: 1,555 Days
• US Class Current: 1/1
• CPC Class Codes: G01C 25/005 initial alignment, calibration or starting-up of inertial devices
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 15/00 Digital computers in general details G06F1/00 – G06F13/00; Data processing equipment in general
• CPC Class Codes: G09G 2320/0693 Calibration of display systems

Abstract

Systems, methods, and computer media for calibrating user-mounted devices are provided. An external device capable of providing calibration data to a user-mounted device worn by a user is identified. An identification acknowledgement is received from the external device. A device calibration mode is entered in which calibration data describing the user-mounted device is received by the user-mounted device. The calibration data is based at least in part on sensor data acquired and normalized by the external device. The calibration data is then received. The calibration data includes at least one determined pose or body measurement of the user and a calculated alignment of the user-mounted device relative to the user. The user-mounted device is calibrated using the received calibration data.

Claims

1. One or more computer storage hardware devices storing computer-executable instructions for performing a method for calibrating a first user-mounted device worn by a first user, the method comprising:

identifying an external device capable of providing calibration data to the first user-mounted device;

receiving an identification acknowledgement from the external device;

entering a device calibration mode in which calibration data describing the first user-mounted device is received by the first user-mounted device, wherein the calibration data is based at least in part on sensor data acquired and normalized by the external device;

receiving the calibration data, wherein the calibration data includes at least one determined pose or body measurement of the first user and the calibration data also includes a calculated alignment of the first user-mounted device relative to the first user; and

calibrating the first user-mounted device using the received calibration data.

2. The one or more computer storage hardware devices of claim 1, wherein the method is performed as an automatic silent calibration.

3. The one or more computer storage hardware devices of claim 1, wherein the method is repeated when at least one of a calibration threshold or a specified time period is exceeded.

4. The one or more computer storage hardware devices of claim 1, wherein the alignment of the first user-mounted device includes rotation of the first user-mounted device relative to the first user.

5. The one or more computer storage hardware devices of claim 1, wherein the sensor data includes data from at least one of:

an inertial measurement unit (IMU);

a red, green, blue (RGB) camera;

a depth sensor;

assisted GPS;

or a microphone.

6. The one or more computer storage hardware devices of claim 1, wherein the first user-mounted device is a head-mounted device.

7. The one or more computer storage hardware devices of claim 1, wherein the external device is in a fixed location.

8. The one or more computer storage hardware devices of claim 1, further comprising receiving input from a passive object.

9. The one or more computer storage hardware devices of claim 1, wherein the input is one of:

a captured reflection of the first user-mounted device or first user or a captured image or video showing the first user-mounted device or first user.

10. The one or more computer storage hardware devices of claim 1, wherein the external device is a second user-mounted device worn by a second user.

11. The one or more computer storage hardware devices of claim 10, wherein the calibration data is transmitted from the second user-mounted device to the first user-mounted device.

12. The one or more computer storage hardware devices of claim 11, further comprising:

acquiring sensor data describing the second user-mounted device;

normalizing the sensor data describing the second user-mounted device;

based at least in part on the normalized sensor data describing the second user-mounted device, (1) determining pose and body measurements of the second user and (2) calculating an alignment of the second user-mounted device relative to the second user; and

transmitting second calibration data to the second user-mounted device, the second calibration data including at least one determined pose and body measurement of the second user and the calculated alignment of the second user-mounted device.

13. One or more computer storage hardware devices having a system embodied thereon including computer-executable instructions that, when executed, perform a method for calibrating a first user-mounted device, the system comprising:

a session component that communicates with an external device to enter a device calibration mode in which calibration data describing the first user-mounted device is received by the first user-mounted device, the calibration data based at least in part on sensor data acquired by the external device;

an intake component that receives the calibration data, the calibration data including at least one determined pose or body measurement of the first user and the calibration data also including a calculated alignment of the first user-mounted device relative to the first user; and

a calibration component that calibrates the first user-mounted device using the received calibration data.

14. The one or more computer storage hardware devices of claim 13, wherein the first user-mounted device is a head-mounted device.

15. The one or more computer storage hardware devices of claim 13, further comprising a passive data component that receives input from a passive object, where the input is one of:

a captured reflection of the first user-mounted device or first user or a captured image or video showing the first user-mounted device or first user.

16. The one or more computer storage hardware devices of claim 13, further comprising a joint calibration component that provides second calibration data to the external device, the second calibration data based on acquired sensor data that describes the external device.

17. The one or more computer storage hardware devices of claim 16, wherein the external device is a second user-mounted device worn by a second user, and wherein the joint calibration component provides the second calibration data by:

normalizing the acquired sensor data that describes the second user-mounted device;

based at least in part on the normalized sensor data describing the second user-mounted device, determining pose and body measurements of the second user;

calculating an alignment of the second user-mounted device relative to the second user; and

transmitting the second calibration data to the second user-mounted device, the second calibration data including at least one determined pose and body measurement of the second user and the calculated alignment of the second user-mounted device.

18. One or more computer storage hardware devices storing computer-executable instructions for performing a method for calibrating a first user-mounted device worn by a first user, the method comprising:

identifying a second user-mounted device worn by a second user, the second user-mounted device capable of providing calibration data to the first user-mounted device;

receiving an identification acknowledgement from the second user-mounted device;

entering a joint device calibration mode in which first calibration data describing the first user-mounted device is received by the first user-mounted device and second calibration data describing the second user-mounted device is transmitted by the first user-mounted device, wherein the first calibration data is based at least in part on first sensor data acquired and normalized by the second user-mounted device, and wherein the second calibration data is based at least in part on second sensor data acquired and normalized by the first user-mounted device;

receiving the first calibration data,

wherein the first calibration data includes at least one determined pose or body measurement of the first user and the calibration data also includes a calculated alignment of the first user-mounted device relative to the first user, and

wherein the first sensor data includes data from at least one of;

an inertial measurement unit (IMU);

a red, green, blue (RGB) camera;

a depth sensor;

assisted GPS;

or a microphone;

calibrating the first user-mounted device using the first calibration data;

acquiring the second sensor data, the second sensor data including data from at least one of;

an IMU;

an RGB camera;

a depth sensor;

assisted GPS;

or a microphone;

normalizing the second sensor data;

based at least in part on the normalized second sensor data, (1) determining pose and body measurements of the second user and (2) calculating an alignment of the second user-mounted device relative to the second user; and

transmitting the second calibration data, the second calibration data including the at least one determined pose and body measurement of the second user and the calculated alignment of the second user-mounted device.

19. The one or more computer storage hardware devices of claim 18, wherein the first and second user-mounted devices are head-mounted devices.

20. The one or more computer storage hardware devices of claim 18, wherein the method is performed as an automatic silent calibration.