Automatic boresighting of head-worn display

• US 9,891,705 B1
• Filed: 08/25/2016
• Issued: 02/13/2018
• Est. Priority Date: 08/25/2016
• Status: Active Grant

• Current Assignee: Rockwell Collins, Inc. (Raytheon Technologies Corporation d/b/a RTX)
• Original Assignee: Rockwell Collins, Inc. (Raytheon Technologies Corporation d/b/a RTX)
• Inventors: Lahr, Weston J., Kirchner, William T., Han, David I.
• Primary Examiner(s): Abdulselam, Abbas
• Application Number:
  US15/247,284
• Time in Patent Office: 537 Days
• CPC Class Codes: F41G 3/225 Helmet sighting systems
• CPC Class Codes: G01C 21/165 combined with non-inertial navigation instruments
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 2027/0198 System for aligning or maintaining alignment of an image in a predetermined direction
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements

Abstract

A system and method for automatic boresighting of a head-tracking system for a HWD or HUD accesses georeferenced pose (position/orientation) data of an aircraft or other mobile platform from a georeferenced head-tracker and platform-referenced head pose data (based on fiducial markers within the aircraft cockpit) from the inertial measurement units of an optical, magnetic, or optical/magnetic platform-referenced head tracker (PRHT) and determines misalignments of the PRHT relative to the boresighted inertial reference system (IRS) of the aircraft. Based on the determined misalignments and accessed data, the IMUs of the PRHT may be updated and aligned with the aircraft boresight. Misalignments of head-mounted fiducial markers with the boresighted IRS may be determined for updating the alignment of an optical head-tracker fixed relative to the aircraft cockpit.

Claims

1. A head-tracking system, comprising:

one or more fiducial markers positioned in a fixed pattern within a mobile platform;

a georeferenced head tracker (GHT) configured to be associated with a head of a user, the GHT configured to provide georeferenced pose data of the mobile platform;

at least one platform-referenced head tracker (PRHT) configured to be associated with the head, the PRHT including at least one inertial management unit (IMU) configured to determine platform-referenced head pose data based on the one or more fiducial markers;

at least one controller coupled to the GHT and the at least one PRHT and configured to be coupled to at least one inertial reference system (IRS) of the mobile platform, the IRS configured to be aligned with a boresight of the mobile platform and to provide georeferenced pose data of the mobile platform, the controller including at least one processor coupled with a non-transitory processor-readable medium and configured to;

engage a calibration mode of the head-tracking system;

access the georeferenced pose data of the mobile platform and the platform-referenced head pose data;

determine at least one misalignment of the PRHT relative to at least one of the IRS and the mobile platform based on the georeferenced aircraft pose data and the platform-referenced head pose data;

determine, based on the at least one of the misalignment, the georeferenced pose data of the mobile platform, and the platform-referenced head pose data, alignment data based on the alignment of the head-tracking system relative to the boresight; and

update the at least one IMU based on the determined alignment data.

2. The system of claim 1, wherein:

the mobile platform includes an aircraft; and

the head-tracking system is embodied in an avionics display system, the avionics display system including at least one of a head-worn display (HWD) worn by the user and a head-up display (HUD) of the aircraft.

3. The system of claim 1, wherein the mobile platform includes at least one of a ground-based vehicle, a submersible vehicle, a ship, and an unmanned aerial vehicle (UAV).

4. The system of claim 1, wherein the controller is configured to:

in response to the engaging of the calibration mode, signal a display system of the mobile platform to provide the user with at least one calibration indication;

determine a confidence level of the alignment data; and

in response to the confidence level exceeding a predetermined confidence threshold, a) store the alignment data to the non-transitory processor-readable medium;

b) disengage the calibration mode; and

c) signal the display unit to terminate the at least one calibration indication.

5. The system of claim 1, wherein:

the at least one PRHT comprises at least one of a) an optical head tracker (OHT) configured to provide optical platform-referenced head pose data and b) a magnetic head tracker (MHT) configured to provide magnetic platform-referenced head pose data; and

the controller is configured to determine at least one of a) a misalignment of the OHT relative to at least one of the IRS and the mobile platform and b) a misalignment of the MHT relative to at least one of the IRS and the mobile platform based on the georeferenced aircraft pose data and at least one of the optical platform-referenced head pose data and the magnetic platform-referenced head pose data.

6. The system of claim 1, wherein the controller is further configured to:

store the alignment data in the non-transitory processor-readable medium;

monitor at least one error associated with the georeferenced aircraft pose data and the platform-referenced head pose data;

in response to the at least one error exceeding a predetermined error threshold, re-engage the calibration mode.

7. The system of claim 1, wherein:

the one or more fiducial markers are fixedly mounted relative to the head;

the at least one PRHT is fixedly mounted to the mobile platform; and

the controller is configured to determine at least one misalignment of the fiducial markers relative to at least one of the IRS and the mobile platform based on the georeferenced aircraft pose data and the platform-referenced head pose data.

8. A head-worn display (HWD), comprising:

at least one display unit configured to be attached to a head of a user, the display unit configured to provide georeferenced images and symbols to the user; and

a head-tracking system comprising;

a georeferenced head tracker (GHT) configured to be associated with the head and to provide georeferenced head pose data;

at least one platform-referenced head tracker (PRHT) configured to be associated with the head, the PRHT configured to determine platform-referenced head pose data based on one or more fiducial markers positioned within a mobile platform;

a controller coupled to the GHT and the at least one PRHT and configured to be coupled with at least one IRS of the mobile platform, the IRS configured to be aligned with a boresight of the mobile platform and to provide georeferenced pose data of the mobile platform, the controller including at least one processor coupled with a non-transitory processor-readable medium, the at least one processor configured to;

access the georeferenced aircraft pose data and the platform-referenced head pose data;

determine at least one misalignment of the PRHT relative to at least one of the IRS and the mobile platform based on the georeferenced aircraft pose data and the platform-referenced head pose data;

determine, based on the at least one misalignment, the georeferenced pose data of the mobile platform, and the platform-referenced head pose data, alignment data based on the alignment of the head-tracking system relative to the boresight; and

update the at least one PRHT based on the determined alignment data.

9. The HWD of claim 8, wherein the at least one PRHT includes at least one of:

an optical head-tracker configured to determine optical platform-referenced head pose data; and

a magnetic head-tracker configured to determine magnetic platform-referenced head pose data.

10. The HWD of claim 8, wherein the at least one processor is further configured to:

in response to the accessing the georeferenced aircraft pose data and the platform-referenced head pose data, signal the display unit to provide the user with at least one calibration indication;

determine at least one confidence level of the determined alignment data; and

in response to the confidence level exceeding a predetermined confidence threshold, store the determined alignment data to the non-transitory processor-readable medium and signal the display unit to terminate the at least one calibration indication.

11. The HWD of claim 8, wherein:

the one or more fiducial markers are fixedly mounted relative to the head;

the at least one PRHT is fixedly mounted to the mobile platform; and

the controller is configured to determine at least one misalignment of the fiducial markers relative to at least one of the IRS and the mobile platform based on the georeferenced aircraft pose data and the platform-referenced head pose data.

12. The HWD of claim 8, wherein the mobile platform includes at least one of a ground-based vehicle, a submersible vehicle, a ship, and a UAV.

13. A method for automatic boresighting of a head-tracking system, the method comprising:

engaging a calibration mode of a head-tracking system of a mobile platform;

generating georeferenced pose data of the mobile platform via an inertial reference system (IRS) of the mobile platform, the IRS aligned with a boresight of the mobile platform;

generating platform-referenced head pose data associated with a head of a user via at least one inertial measurement unit (IMU) of a platform-referenced head-tracker (PRHT) of the head-tracking system and a plurality of fiducial markers mounted in a fixed pattern within the mobile platform;

determining, via a processor coupled to the PRHT, at least one misalignment of the PRHT relative to at least one of the IRS and the mobile platform based on the georeferenced pose data of the mobile platform and the platform-referenced head pose data;

determining, via the processor, alignment data based on the alignment of the head-tracking system relative to the boresight based on the at least one misalignment, the georeferenced pose data of the mobile platform, and the platform-referenced head pose data; and

updating the at least one IMU based on the determined alignment data.

14. The method of claim 13, wherein generating platform-referenced head pose data associated with a head of a user via at least one inertial measurement unit (IMU) of a platform-referenced head-tracker (PRHT) of the head-tracking system and a plurality of fiducial markers mounted in a fixed pattern within the mobile platform includes:

generating platform-referenced head pose data associated with a head of a user via at least one inertial measurement unit (IMU) of a platform-referenced head-tracker (PRHT) fixedly mounted to the mobile platform and a plurality of fiducial markers fixedly mounted relative to the head.

15. The method of claim 14, wherein determining, via a processor coupled to the PRHT, at least one misalignment of the PRHT relative to at least one of the IRS and the mobile platform based on the georeferenced pose data of the mobile platform and the platform-referenced head pose data includes:

determining, via a processor coupled to the PRHT, at least one misalignment of the fiducial markers relative to at least one of the IRS and the mobile platform and the platform-referenced head pose data.

16. The method of claim 13, wherein generating platform-referenced head pose data associated with a head of a user via at least one inertial measurement unit (IMU) of a platform-referenced head-tracker (PRHT) of the head-tracking system and a plurality of fiducial markers mounted in a fixed pattern within the mobile platform includes:

generating platform-referenced head pose data associated with a head of the user via at least one IMU of one or more of an optical head-tracker (OHT) and a magnetic head-tracker (MHT) of a head-tracking system and a plurality of fiducial markers mounted in a fixed pattern within the mobile platform.

17. The method of claim 13, wherein generating platform-referenced head pose data associated with a head of a user via at least one inertial measurement unit (IMU) of a platform-referenced head-tracker (PRHT) of the head-tracking system and a plurality of fiducial markers mounted in a fixed pattern within the mobile platform includes:

generating platform-referenced head pose data associated with a head of a user via at least one IMU of a platform-referenced head-tracker (PRHT) of at least one of a HWD worn by the user and a HUD of the mobile platform.

18. The method of claim 13, further comprising:

monitoring at least one error associated with the georeferenced aircraft pose data and the platform-referenced head pose data;

in response to the error exceeding a predetermined error threshold, re-engaging the calibration mode.

19. The method of claim 13, wherein engaging a calibration mode of a head-tracking system of a mobile platform includes:

signaling, via the processor, at least one display unit coupled to the head-tracking system to provide at least one calibration notification to the user.

20. The method of claim 19, further comprising:

determining, via the processor, at least one confidence level of the determined alignment data; and

in response to the confidence level exceeding a predetermined confidence threshold;

storing the determined alignment data to the non-transitory processor-readable medium;

disengaging, via the processor, the calibration mode; and

signaling the display unit to terminate the at least one calibration indication.