Adjustment of a mixed reality display for inter-pupillary distance alignment

• US 9,025,252 B2
• Filed: 08/30/2011
• Issued: 05/05/2015
• Est. Priority Date: 08/30/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Lewis, John R., Perez, Kathryn Stone, Crocco, Robert L., Kipman, Alex Aben-Athar
• Primary Examiner(s): PINKNEY, DAWAYNE
• Application Number:
  US13/221,707
• Publication Number:
  US 20130050833A1
• Time in Patent Office: 1,344 Days
• Field of Search: 359630-634, 359 13- 14, 310/49.R, 310/156.32, 310/156.33, 310/156.34, 310/156.35, 310266-268, 310/156.02, 349/11, 353 11- 12, 353/28, 353/119, 345/7, 345/9, 340/438, 340/980, 340/995.1, 340/815.47, 340/815.74, 351/200, 351205-206, 351/210, 351/221
• US Class Current: 359/630
• CPC Class Codes: A61B 3/111 for measuring interpupillary distance
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0176 characterised by mechanical features
• CPC Class Codes: G02B 30/36 using refractive optical elements, e.g. prisms, in the optical path between the images and the observer
• CPC Class Codes: G02B 7/12 Adjusting pupillary distance of binocular pairs
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/383 for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
• CPC Class Codes: H04N 2213/001 Constructional or mechanical details

Abstract

The technology provides for adjusting a see-through, near-eye, mixed reality display device for alignment with an inter-pupillary distance (IPD) of a user by different examples of display adjustment mechanisms. The see-through, near-eye, mixed reality system includes for each eye a display optical system having an optical axis. Each display optical system is positioned to be seen through by a respective eye, and is supported on a respective movable support structure. A display adjustment mechanism attached to the display device also connects with each movable support structure for moving the structure. A determination is automatically made as to whether the display device is aligned with an IPD of a user. If not aligned, one or more adjustment values for a position of at least one of the display optical systems is automatically determined. The display adjustment mechanism moves the at least one display optical system in accordance with the adjustment values.

Claims

1. A system for adjusting a near-eye, mixed reality display for alignment with an inter-pupillary distance (IPD) comprising:

a near-eye, mixed reality display device including, for each of two eyes, a display optical system, each display optical system having an optical axis and being positioned to be seen through by a respective eye;

the display device including a respective movable support structure for supporting each display optical system;

at least one sensor for each display optical system attached to the display device, the sensor having a detection area at a position to capture data of the respective eye;

a memory for storing software and data including the captured data of each eye;

one or more processors having access to the memory for determining one or more position adjustment values for each display optical system based on the captured data and the position of the respective detection area for alignment with the IPD; and

a respective display adjustment mechanism attached to each display optical system and communicatively coupled to the one or more processors, each display adjustment mechanism adapted to independently move the respective display optical system by moving a respective movable support structure to adjust a position of the respective display optical system in accordance with the one or more position adjustment values.

2. The system of claim 1 wherein the respective movable support structure is a movable eyeglass frame portion.

3. The system of claim 1 wherein the respective movable support structure further supports an image generation unit in alignment with the display optical system.

4. The system of claim 1 wherein the one or more processors for determining one or more position adjustment values for each display optical system based on the captured data and the position of the respective detection area further comprises the one or more processors determining a pupil position for each eye with respect to the optical axis of its respective display optical system.

5. The system of claim 1 wherein the at least one display adjustment mechanism is automatically controlled by the one or more processors for adjusting a position of the respective display optical system in accordance with the one or more position adjustment values.

6. The system of claim 1 further comprising the one or more processors causing the display device to electronically provide instructions for user activation of the at least one display adjustment mechanism for moving the at least one movable support structure in accordance with the one or more position adjustment values.

7. The system of claim 6 wherein the at least one display adjustment mechanism includes a mechanical controller which has a calibration for user activation of the controller to correspond to a predetermined distance and direction for movement of at least one display optical system;

and

the one or more processors determine the content of the instructions based on the calibration.

8. The system of claim 7 wherein the at least one display adjustment mechanism is a mechanical adjustment mechanism.

9. The system of claim 8 wherein the mechanical adjustment mechanism comprises at least one of the following:

a ratcheting mechanism;

a turn wheel mechanism; and

a sliding mechanism.

10. In a near-eye, mixed reality system including, for each eye of a wearer, a display optical system, each display optical system having an optical axis and being positioned to be seen through by a respective eye, a method for aligning a near-eye, mixed reality display device with an inter-pupillary distance (IPD) for the wearer, the method comprising:

automatically determining whether the near-eye, mixed reality display is aligned with a wearer IPD in accordance with an alignment criteria; and

responsive to the display device not being aligned with the wearer IPD in accordance with the alignment criteria,

determining one or more adjustment values for at least one display optical system for satisfying the alignment criteria, and

adjusting one or both display optical systems based on the one or more adjustment values, each display optical system supported by an independently movable support structure, by moving at least one independently movable support structure based on the one or more adjustment values.

11. The method of claim 10 wherein the one or more adjustment values comprise a horizontal adjustment value and a vertical adjustment value.

12. The method of claim 11 wherein adjusting the at least one display optical system based on the one or more adjustment values further comprises adjusting the at least one display optical system in accordance with a depth adjustment value.

13. The method of claim 10 wherein the one or more adjustment values are determined based on a three-dimensional position difference vector representing a position difference between current and reference gaze vectors to an object in a three-dimensional user field of view.

14. The method of claim 10 wherein the one or more adjustment values are determined based on a mapping criteria of at least one respective sensor of the display device for each display optical system, the at least one respective sensor capturing data of the respective eye of the display optical system.

15. A system for adjusting a near-eye display system in any of three dimensions for alignment with an inter-pupillary distance of a user comprising:

a near-eye, mixed reality display device including for each eye;

a display optical system, each display optical system having an optical axis and being positioned to be seen through by a respective eye;

the display device including a respective independently movable support structure for supporting each display optical system, the respective independently movable support structure being capable of movement in three dimensions;

at least one sensor for each display optical system attached to the display device, the sensor having a detection area at a position for capturing data of the respective eye;

a memory for storing software and data including the captured data of each eye;

one or more processors having access to the memory for determining one or more position adjustment values in any of three dimensions for each display optical system based on the captured data and the position of the respective detection area for alignment with the IPD; and

at least one display adjustment mechanism attached to the display device and communicatively coupled to the one or more processors for moving at least one independently movable support structure in any of three dimensions under control of the one or more processors in accordance with the one or more position adjustment values.

16. The system of claim 15 wherein:

the display device system further comprises a respective image generation unit for creating images for display for each display optical system; and

the at least one display adjustment mechanism under the control of the processor moving the image generation unit in any of three dimensions for maintaining an optical path between the image generation unit and the respective display optical system.

17. The system of claim 16 wherein each image generation unit is attached to the respective movable support structure for the respective display optical system.

18. The system of claim 16 wherein each respective movable support structure supports an arrangement of gaze detection elements positioned with respect to each display optical system for forming a spatial relationship between the gaze detection elements and the respective eye.

19. The system of claim 15 wherein the respective movable support structure is an inner eyeglass frame portion attached to a fixed eyeglass frame portion.

20. The system of claim 15 wherein the at least one display adjustment mechanism comprises an XYZ transport mechanism including one or more motors.