Hybrid display system

• US 9,977,493 B2
• Filed: 06/17/2015
• Issued: 05/22/2018
• Est. Priority Date: 06/17/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Lanier, Jaron, Kollin, Joel S.
• Primary Examiner(s): Patel, Premal
• Application Number:
  US14/742,592
• Publication Number:
  US 20160370855A1
• Time in Patent Office: 1,070 Days
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: H04N 13/279 the virtual viewpoint locations being selected by the viewers or determined by tracking
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 25/00 Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 9/3173 wherein the projection device is specially adapted for enhanced portability

Abstract

Embodiments are disclosed for a display system having multiple fields of view. An example display system comprises a first display configured to produce images across a first field of view, a second display configured to produce images across a second field of view, the second field of view being larger than the first field of view, and a controller configured to selectively present a virtual object via one or more of the first display and the second display based on a feature of the virtual object as displayed within a user'"'"'s viewspace.

Claims

1. A head-mounted display system comprising:

a first transparent display configured to produce images across a first field of view;

a second transparent display configured to produce images across a second field of view, the second field of view being larger than the first field of view; and

a controller configured to selectively present a virtual object in an overlapped region of the first field of view and the second field of view, such selective presentation including determining whether to use one or both of the first transparent display and the second transparent display based on occlusion to be provided by the virtual object.

2. The head-mounted display system of claim 1, wherein the first transparent display comprises a waveguide-type or birdbath display, and wherein the second transparent display comprises a compressed lightfield or Maxwellian-view display.

3. The head-mounted display system of claim 2, wherein the first transparent display comprises a microprojector that projects into a waveguide and a plurality of gratings for replicating a pupil in a plurality of directions, and wherein the second transparent display comprises a pinlight display.

4. The head-mounted display system of claim 1, wherein the first transparent display has a field of view that is less than or equal to 60 degrees, and wherein the second transparent display has a field of view that is greater than 60 degrees.

5. The head-mounted display system of claim 4, wherein the field of view of the first transparent display is between 30 and 60 degrees, inclusive, wherein the field of view of the second transparent display is above 60 degrees, and wherein the second transparent display provides selective occlusion of a real-world environment as viewed through the second transparent display.

6. The head-mounted display system of claim 1, wherein the first transparent display has a higher resolution than the second display.

7. The head-mounted display system of claim 1, wherein selectively presenting the virtual object via the first and second transparent display comprises displaying at least a portion of the virtual object via the second transparent display responsive to determining that a transparency of the virtual object is below a threshold.

8. The head-mounted display system of claim 1, wherein the head-mounted display system comprises, in order along an optical axis extending from a user'"'"'s eye, the first transparent display, an air gap, and the second transparent display.

9. A method of producing an image in a head-mounted display device, the method comprising:

directing light from a first light source to a waveguide to selectively present virtual objects via a first transparent display, the first transparent display having a first field of view;

directing light from one or more second light sources to a user'"'"'s eye to selectively present virtual objects via a second transparent display, the second display having a second field of view that is larger than the first field of view;

generating a virtual object for display in a user'"'"'s viewspace;

determining a location for the virtual object within the user'"'"'s viewspace;

if the location for the virtual object is outside of the first field of view, displaying the virtual object with the second transparent display; and

if the location for the virtual object is within an overlapped region of the first field of view and the second field of view, selectively displaying the virtual object with one or both of the first transparent display and the second transparent display based on occlusion to be provided by the virtual object.

10. The method of claim 9, wherein selectively displaying the virtual object with one or both of the first transparent display and the second transparent display comprises displaying the virtual object with one or both of the first transparent display and the second transparent display if the occlusion to be provided by the virtual object is below a threshold and displaying the virtual object with at least the second transparent display if the occlusion to be provided by the virtual object is above the threshold.

11. The method of claim 9, wherein selectively displaying the virtual object with one or both of the first transparent display and the second transparent display is further based on a designation of the virtual object as an interactive object for a user interface such that the virtual object corresponds to a virtual or real-world element that is used to provide a gesture input to the user interface.

12. The method of claim 9, wherein selectively displaying the virtual object with one or both of the first transparent display and the second transparent display further comprises determining a resolution of the virtual object, then displaying the virtual object with the first transparent display if the resolution of the virtual object is above a threshold and displaying the virtual object with the second transparent display if the resolution of the virtual object is below the threshold.

13. The method of claim 9, further comprising displaying at least a first portion of the virtual object with the first transparent display and at least a second portion of the virtual object with the second transparent display responsive to determining that the virtual object is located within the first field of view and that a transparency of the virtual object is above a threshold.

14. The method of claim 9, wherein the second transparent display comprises a pinlight display or a Maxwellian-view display.

15. A head-mounted display device comprising:

a first transparent display configured to produce images across a first field of view, the first transparent display comprising a waveguide;

a second transparent display configured to produce images across a second field of view, the second field of view being larger than the first field of view, the second transparent display comprising an aperture mask display and being separated from the first transparent display by an air gap; and

a circuit configured to;

present a virtual object in an overlapped region of the first field of view and the second field of view to a wearer of the head-mounted display device via a selected one of a plurality of display modes based on occlusion to be provided by the virtual object as displayed within a user'"'"'s viewspace, the display modes including a first display mode in which the virtual object is displayed via only the first transparent display, a second display mode in which the virtual object is displayed via only the second transparent display, and a third display mode in which the virtual object is displayed via the first transparent display and the second transparent display.

16. The head-mounted display device of claim 15, wherein the first transparent display and the second transparent display form a stacked display in which the first transparent display is positioned nearer to an eye of a wearer of the head-mounted display device than the second transparent display.

17. The head-mounted display device of claim 15, wherein the first transparent display has a field of view that is less than or equal to 60 degrees, wherein the second transparent display has a field of view that is greater than 60 degrees, and wherein the second transparent display provides selective occlusion of a real-world environment as viewed through the second transparent display.

18. The head-mounted display device of claim 15, wherein presenting the virtual object to a wearer of the head-mounted display device via a selected one of a plurality of display modes comprises displaying at least a portion of the virtual object via the second display mode responsive to determining that a transparency of the virtual object is below a threshold.

19. The head-mounted display device of claim 15, wherein presenting the virtual object to a wearer of the head-mounted display device via a selected one of a plurality of display modes comprises displaying at least a portion of the virtual object via the second display mode responsive to determining that the virtual object is to be presented in a peripheral region of the user'"'"'s viewspace and displaying at least a portion of the virtual object via the first display mode responsive to determining that the virtual object is to be presented in a central region of the user'"'"'s viewspace.

20. The head-mounted display device of claim 15, wherein the first transparent display has a higher resolution than the second transparent display.

21. The head-mounted display device of claim 15 wherein the aperture mask display comprises a pinlight display or a Maxwellian-view display.