HEAD MOUNTED DISPLAY AND CONTROL METHOD THEREOF

• US 20150212330A1
• Filed: 04/24/2014
• Published: 07/30/2015
• Est. Priority Date: 01/24/2014
• Status: Active Grant

• Current Assignee: Quanta Computer
• Original Assignee: Quanta Computer
• Inventors: LI, Chung-Te, YANG, Wen-Chu
• Granted Patent: US 9,430,878 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0179 Display position adjusting means not related to the information to be displayed
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A head mounted display and a control method thereof are disclosed. The control method comprises following steps. An application processor controls a pico projector unit to project a virtual image having a virtual object located on a virtual image coordinate in a virtual image coordinate system. An eye image sensing unit captures an eye image data. A sensing apparatus senses a touch object to output a sensing data. An ASIC obtains a real image coordinate of the touch object in a real image coordinate system according to the sensing data. The ASIC obtains a pupil position according to the eye image data, and controls the adjustment unit to adjust an imaging position of the virtual image according to the pupil position. The ASIC determines whether the touch object touched the virtual object according to the pupil position, the real coordinate and the virtual coordinate.

Claims

1. A head mounted display, comprising:

a pico projector unit;

an application processor for controlling the pico projector unit to project a virtual image having a virtual object located on a virtual image coordinate in a virtual image coordinate system;

an eye image sensing unit for capturing an eye image data;

an adjustment unit;

a sensing apparatus for sensing a touch object to output a sensing data;

an application specific integrated circuit (ASIC) for obtaining a real coordinate of the touch object in a real image coordinate system according to the sensing data, wherein the ASIC obtains a pupil position according to the eye image data, controls the adjustment unit to adjust an imaging position of the virtual image according to the pupil position, and determines whether the touch object touched the virtual object according to the pupil position, the real coordinate and the virtual coordinate; and

a glasses frame for supporting the adjustment unit, the pico projector unit, the application processor, the eye image sensing unit, the real coordinate capturing device and the ASIC.

2. The head mounted display according to claim 1, wherein, when the pupil position is located at the center of the eye, the ASIC converts the real coordinate into a first relative coordinate in the virtual coordinate system, and determines whether the first relative coordinate matches the virtual image coordinate, and the ASIC determines that the touch object touched the virtual object if the first relative coordinate matches the virtual image coordinate.

3. The head mounted display according to claim 2, wherein, when the pupil position is not located at the center of the eye, the ASIC keeps the virtual coordinate unchanged, and correspondingly converts the real coordinate into a second relative coordinate, which differs with the first relative coordinate by a touch displacement, in the virtual coordinate system.

4. The head mounted display according to claim 3, wherein, the ASIC determines whether the second relative coordinate matches the virtual image coordinate, and the ASIC determines that the touch object touched the virtual object if the second relative coordinate matches the virtual image coordinate.

5. The head mounted display according to claim 1, further comprising a beam splitter unit, wherein the application processor controls the pico projector unit to project a 3D calibrated image, the ASIC controls the adjustment unit to adjust an imaging position of the virtual image according to the pupil position such that a beam position of the pico projector unit matches the pupil position, the ASIC requests the application processor to send an inquiry message to inquire whether the presented 3D image is of good quality, the ASIC controls the adjustment unit to adjust an imaging position of the virtual image if the presented 3D image is not of good quality, and the ASIC decides and records a calibrated displacement according to the adjusted beam splitter unit if the presented 3D image is of good quality.

6. The head mounted display according to claim 1, wherein the application processor records a moveable range of the pupil position.

7. The head mounted display according to claim 1, wherein the sensing apparatus comprises a first image sensor and a second image sensor.

8. The head mounted display according to claim 7, wherein the first image sensor and the second image sensor comprise wide angle lens.

9. The head mounted display according to claim 1, wherein the sensing apparatus comprises a first image sensor and a time of flight (TOF) camera.

10. The head mounted display according to claim 1, wherein the sensing apparatus comprises a first image sensor and a structured light camera.

11. The head mounted display according to claim 1, further comprising:

an infrared light emitting unit for providing an auxiliary light source required by the eye image sensing unit.

12. The head mounted display according to claim 11, further comprising a beam splitter unit comprising a first beam splitter, wherein the pico projector unit comprises a first pico projector, the eye image sensing unit comprises a first eye image sensor, the adjustment unit comprises a first adjustment apparatus, the infrared light emitting unit comprises a first infrared LED, the first pico projector projects the virtual image, the first eye image sensor captures the eye image data, the first adjustment apparatus adjusts the first beam splitter, and the first infrared LED provides an auxiliary light source required by the first eye image sensor.

13. The head mounted display according to claim 11, wherein the pico projector unit comprises a first pico projector, the eye image sensing unit comprises a first eye image sensor, the adjustment unit comprises a first adjustment apparatus, the infrared light emitting unit comprises a first infrared LED, the first pico projector projects the virtual image, the first eye image sensor captures the eye image data, the first adjustment apparatus adjusts the first pico projector, and the first infrared LED provides an auxiliary light source required by the first eye image sensor.

14. The head mounted display according to claim 11, further comprising a beam splitter unit comprising a first beam splitter and a second beam splitter, wherein the pico projector unit comprises a first pico projector and a second pico projector, the eye image sensing unit comprises a first eye image sensor and a second eye image sensor, the adjustment unit comprises a first adjustment apparatus and a second adjustment apparatus, the infrared light emitting unit comprises a first infrared LED and a second infrared LED, the first pico projector and the second pico projector project the virtual image, the first eye image sensor and the second eye image sensor capture the eye image data, the first adjustment apparatus adjusts the first beam splitter, the second adjustment apparatus adjusts the second beam splitter, the first infrared LED provides an auxiliary light source required by the first eye image sensor, and the second infrared LED provides an auxiliary light source required by the second eye image sensor.

15. A control method of a head mounted display, comprising:

controlling a pico projector unit to project a virtual image having a virtual object located on a virtual image coordinate in a virtual image coordinate system;

capturing an eye image data by an eye image sensing unit;

sensing a touch object by a sensing apparatus to output a sensing data;

obtaining a real coordinate of the touch object in a real image coordinate system according to the sensing data ;

obtaining a pupil position according to the eye image data, and controlling an adjustment unit to adjust an imaging position of the virtual image according to the pupil position; and

determining whether the touch object touched the virtual object according to the pupil position, the real coordinate and the virtual coordinate.

16. The control method according to claim 15, wherein in the determination step, when the pupil position is located at the center of the eye, the real coordinate is converted into a first relative coordinate in the virtual coordinate system, and whether the first relative coordinate matches the virtual image coordinate is determined, and it is determined that the touch object touched the virtual object if the first relative coordinate matches the virtual image coordinate.

17. The control method according to claim 16, wherein in the determination step, when the pupil position is not located at the center of the eye, the virtual coordinate is kept unchanged, and the real coordinate is correspondingly converted into a second relative coordinate, which differs with the first relative coordinate by a touch displacement, in the virtual coordinate system.

18. The control method according to claim 17, wherein in the determination step, whether the second relative coordinate matches the virtual image coordinate is determined, and it is determined that the touch object touched the virtual object if the second relative coordinate matches the virtual image coordinate.

19. The control method according to claim 15, further comprising:

projecting a 3D calibrated image;

controlling the adjustment unit to adjust an imaging position of the virtual image according to the pupil position, such that a beam position of the pico projector unit matches the pupil position;

sending an inquiry message to inquire whether the presented 3D image is of good quality;

adjusting an imaging position of the virtual image if the presented 3D image is not of good quality; and

determining and recording a calibrated displacement according to the adjusted beam splitter unit if the presented 3D image is of good quality.

20. The control method according to claim 15, further comprising:

recording a moveable range of the pupil position.

21. The control method according to claim 15, wherein the sensing apparatus comprises a first image sensor and a second image sensor.

22. The control method according to claim 21, wherein the first image sensor and the second image sensor comprise wide angle lens.

23. The control method according to claim 15, wherein the sensing apparatus comprises a first image sensor and a time of flight (TOF) camera.

24. The control method according to claim 15, wherein the sensing apparatus comprises a first image sensor and a structured light camera.

25. The control method according to claim 15, wherein the head mounted display further comprises:

an infrared light emitting unit for providing an auxiliary light source required by the eye image sensing unit.

26. The control method according to claim 25, wherein the head mounted display further comprises a beam splitter unit comprising a first beam splitter, the pico projector unit comprises a first pico projector, the eye image sensing unit comprises a first eye image sensor, the adjustment unit comprises a first adjustment apparatus, the infrared light emitting unit comprises a first infrared LED, the first pico projector projects the virtual image, the first eye image sensor captures the eye image data, the first adjustment apparatus adjusts the first beam splitter, and the first infrared LED provides an auxiliary light source required by the first eye image sensor.

27. The control method according to claim 25, wherein the pico projector unit comprises a first pico projector, the eye image sensing unit comprises a first eye image sensor, the adjustment unit comprises a first adjustment apparatus, the infrared light emitting unit comprises a first infrared LED, the first pico projector projects the virtual image, the first eye image sensor captures the eye image data, the first adjustment apparatus adjusts the first pico projector, and the first infrared LED provides an auxiliary light source required by the first eye image sensor.

28. The control method according to claim 25, wherein the head mounted display further comprises a beam splitter unit comprising a first beam splitter and a second beam splitter, the pico projector unit comprises a first pico projector and a second pico projector, the eye image sensing unit comprises a first eye image sensor and a second eye image sensor, the adjustment unit comprises a first adjustment apparatus and a second adjustment apparatus, the infrared light emitting unit comprises a first infrared LED and a second infrared LED, the first pico projector and the second pico projector project the virtual image, the first eye image sensor and the second eye image sensor capture the eye image data, the first adjustment apparatus adjusts the first beam splitter, the second adjustment apparatus adjusts the second beam splitter, the first infrared LED provides an auxiliary light source required by the first eye image sensor, and the second infrared LED provides an auxiliary light source required by the second eye image sensor.