Head mounted display system

• US 9,596,456 B2
• Filed: 07/16/2014
• Issued: 03/14/2017
• Est. Priority Date: 07/16/2014
• Status: Active Grant

• Current Assignee: Quanta Computer
• Original Assignee: Quanta Computer
• Inventors: Yang, Wen-Chu, Li, Chung-Te
• Primary Examiner(s): DINH, DUC Q
• Application Number:
  US14/332,939
• Publication Number:
  US 20160021358A1
• Time in Patent Office: 972 Days
• Field of Search: 345 7- 9
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G09G 3/002 to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/383 for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
• CPC Class Codes: H04N 13/398 Synchronisation thereof; Control thereof

Abstract

A head mounted display system comprises a lens set, an image capturing unit and a processing circuit. The lens set comprises a first liquid crystal panel and a second liquid crystal panel. The first liquid crystal panel comprises first liquid crystal blocks, and the second liquid crystal panel comprises second liquid crystal blocks. The image capturing unit captures front image data having a first dynamic range. The processing circuit performs tone mapping according to the front image data to generate mapping image data having a second dynamic range smaller than the first dynamic range. The processing circuit calculates regulated values according to the mapping image data. A driving circuit drives the first liquid crystal blocks and the second liquid crystal blocks according to the regulated values, respectively.

Claims

1. A head mounted display system, comprising:

a lens set, comprising;

a first liquid crystal panel comprising a plurality of first liquid crystal blocks; and

a second liquid crystal panel comprising a plurality of second liquid crystal blocks;

an image capturing unit for capturing front image data having a first dynamic range;

a processing circuit for performing tone mapping according to the front image data to generate mapping image data having a second dynamic range smaller than the first dynamic range, the processing circuit calculating a plurality of regulated values according to the mapping image data, wherein the processing circuit performs stereo matching according to a first region of interest (ROI) and a second ROI to generate depth information and a plurality of real image coordinates, calculates a plurality of first projection positions of the real image coordinates projected onto the first liquid crystal panel, and calculates a plurality of second projection positions of the real image coordinates projected onto the second liquid crystal panel; and

a driving circuit for driving the first liquid crystal blocks and the second liquid crystal blocks according to the first projection positions, the second projection positions and the regulated values.

2. The head mounted display system according to claim 1, wherein the image capturing unit comprises:

a first image capturing device for capturing a first front image of the front image data; and

a second image capturing device for capturing a second front image of the front image data,

wherein the processing circuit performs the tone mapping according to the first front image to generate a first mapping image of the mapping image data, performs the tone mapping according to the second front image to generate a second mapping image of the mapping image data, and calculates the regulated values according to the first mapping image and the second mapping image.

3. The head mounted display system according to claim 2, wherein the processing circuit crops the first front image according to horizontal offset information and vertical offset information to generate the first ROI, and performs the tone mapping on the first ROI to generate the first mapping image, and the processing circuit crops the second front image according to the horizontal offset information and the vertical offset information to generate the second ROI, and performs the tone mapping on the second ROI to generate the second mapping image.

4. The head mounted display system according to claim 1, further comprising:

an eye image capturing unit for capturing eye image data,

wherein the processing circuit calculates pupil offset information according to the eye image data, corrects the first projection positions according to the pupil offset information, and corrects the second projection positions according to the pupil offset information.

5. The head mounted display system according to claim 4, wherein the eye image capturing unit comprises:

a first eye camera for capturing a first eye image of the eye image data; and

a second eye camera for capturing a second eye image of the eye image data,

wherein the processing circuit calculates a first pupil offset of the pupil offset information according to the first eye image, corrects the first projection positions according to the first pupil offset, calculates a second pupil offset of the pupil offset information according to the second eye image, and corrects the second projection positions according to the second pupil offset.

6. The head mounted display system according to claim 3, wherein the processing circuit performs a ROI calibration to generate the horizontal offset information and the vertical offset information.

7. The head mounted display system according to claim 1, wherein the image capturing unit comprises:

a first image capturing device for capturing a first front image of the front image data,

wherein the processing circuit performs the tone mapping according to the first front image to generate a first mapping image of the mapping image data, and calculates the regulated values according to the first mapping image.

8. The head mounted display system according to claim 7, wherein the processing circuit crops the first front image according to horizontal offset information and vertical offset information to generate the first ROI, and performs the tone mapping on the first ROI to generate the first mapping image.

9. The head mounted display system according to claim 8, further comprising:

a time of flight (TOF) camera for providing the depth information, wherein the processing circuit calculates a plurality of real image coordinates according to the depth information and the first front image.

10. The head mounted display system according to claim 8, further comprising:

a structured light camera for providing the depth information, wherein the processing circuit calculates a plurality of real image coordinates according to the depth information and the first front image.

11. The head mounted display system according to claim 8, wherein the processing circuit performs a ROI calibration to generate the horizontal offset information and the vertical offset information.

12. The head mounted display system according to claim 1, wherein the lens set further comprises a first polarization structure, a second polarization structure, a first polarizing beam splitter (PBS) and a second PBS, the first liquid crystal panel is disposed between the first polarization structure and the first PBS, and the second liquid crystal panel is disposed between the second polarization structure and the second PBS.

13. The head mounted display system according to claim 12, wherein the first polarization structure and the second polarization structure are polarizers.

14. The head mounted display system according to claim 12, wherein the first polarization structure and the second polarization structure are PBSs.

15. The head mounted display system according to claim 1, wherein the image capturing unit is a native HDR sensor.

16. The head mounted display system according to claim 1, wherein the image capturing unit is an alternate row sensor.

17. The head mounted display system according to claim 1, wherein the image capturing unit is a staggered sensor.

18. The head mounted display system according to claim 1, wherein the image capturing unit is an image sensor.

19. The head mounted display system according to claim 1, wherein the tone mapping is global tone mapping.

20. The head mounted display system according to claim 1, wherein the tone mapping is local tone mapping.