HEAD-MOUNTED DISPLAY SYSTEM AND OPERATION METHOD THEREOF

• US 20160091717A1
• Filed: 06/23/2015
• Published: 03/31/2016
• Est. Priority Date: 09/25/2014
• Status: Abandoned Application

• Current Assignee: Coretronic Corporation
• Original Assignee: Coretronic Corporation
• Inventors: Wu, Chia-Chien, Hsiao, Chi-Wu, Chen, Chien-Ting, Hsu, Fu-Yu
• Application Number:
  US14/746,846
• Publication Number:
  US 20160091717A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0176 characterised by mechanical features
• CPC Class Codes: G02B 27/646 compensating for small deviations, e.g. due to vibration or shake movement of one or more optical elements for control of motion blur in cameras, projectors or printers G03B2205/0007; image stabilisation in cameras peculiar to the presence or use of an electronic image sensor H04N23/68
• CPC Class Codes: G08G 1/00 Traffic control systems for road vehicles arrangement of road signs or traffic signals E01F9/00 ; automatic vehicle control B62D

Abstract

A display system and an operation method thereof are provided. The head-mounted display (HMD) device of the display system includes a camera, a monitor, a communication circuit and a processor. The processor is coupled to the monitor, the camera and the communication circuit. The processor receives and processes sensing data provided by a motion sensor of the display system through the communication circuit, so as to obtain a motion trajectory of the user. The camera photographs a person in a view field of the user. The processor performs image recognition to the video data provided by the camera, so as to obtain a motion trajectory of the person. The processor controls the monitor to display the motion trajectory of the user or the motion trajectory of the person.

Claims

1. A display system, comprising:

a first motion sensor, adapted to be disposed on at least one of limbs of a user or on an object held by the user; and

a head-mounted display (HMD) device, adapted to be worn on a head of the user, and the HMD device comprising;

a camera, adapted to photograph a person in a view field of the user to generate video data;

a monitor, adapted to display an image for the user to view;

a communication circuit, adapted to establish connection with the first motion sensor; and

a processor, coupled to the monitor, the camera and the communication circuit, wherein the processor is adapted to receive and process first sensing data provided by the first motion sensor through the communication circuit, so as to obtain a motion trajectory of the user, and the processor is adapted to receive the video data provided by the camera, and perform image recognition to the video data to obtain a motion trajectory of the person, and display the motion trajectory of the user or the motion trajectory of the person on the image.

2. The display system as claimed in claim 1, wherein the motion trajectory of the user comprises a motion trajectory of the least one of limbs of the user or a motion trajectory of the object held by the user, and the motion trajectory of the person comprises a motion trajectory of at least one of limbs of the person or a motion trajectory of another object held by the person.

3. The display system as claimed in claim 1, wherein the first motion sensor comprises a g-sensor, a gyroscope, an accelerometer, an electronic compass, or an altitude meter.

4. The display system as claimed in claim 1, wherein the motion trajectory of the user and the motion trajectory of the person are simultaneously displayed on the image.

5. The display system as claimed in claim 1, wherein the HMD device comprises:

a sound sensor, coupled to the processor, and adapted to sense a sound sent by the user to generate sound data to the processor,

wherein the processor is adapted to perform voice recognition to the sound data to obtain a sound control instruction.

6. The display system as claimed in claim 1, wherein the HMD device comprises:

a touch sensor, coupled to the processor, and adapted to sense a touch operation of the user to generate touch data to the processor, wherein the processor is adapted to generate a touch instruction according to the touch data.

7. The display system as claimed in claim 1, wherein the HMD device comprises:

a second motion sensor, coupled to the processor, and adapted to sense a head motion of the user to generate second sensing data to the processor.

8. The display system as claimed in claim 7, wherein the second motion sensor comprises an electronic compass, a gyroscope, a geomagnetic sensor, an accelerometer, or a g-sensor.

9. The display system as claimed in claim 7, wherein the processor is adapted to generate a menu instruction according to the second sensing data.

10. The display system as claimed in claim 1, wherein the processor is adapted to correct the motion trajectory of the person according to a direction of the person in the video data.

11. An operation method of a display system, comprising:

sensing a motion of a user to generate first sensing data by a first motion sensor;

analyzing the first sensing data transmitted to a processor of a head-mounted display (HMD) device from the first motion sensor by the processor to obtain a motion trajectory of the user;

photographing a person in a view field of the user by a camera of the HMD device to provide video data and transmit the video data to the processor;

performing image recognition to the video data to obtain a motion trajectory of the person by the processor; and

displaying the motion trajectory of the user or the motion trajectory of the person on an image by a monitor of the HMD device.

12. The operation method of the display system as claimed in claim 11, wherein the HMD device is adapted to be worn on a head of the user, the first motion sensor is adapted to be disposed on at least one of limbs of the user or on an object held by the user, and the motion trajectory of the user comprises a motion trajectory of the least one of limbs of the user or a motion trajectory of the object held by the user.

13. The operation method of the display system as claimed in claim 11, wherein the motion trajectory of the person comprises a motion trajectory of at least one of limbs of the person or a motion trajectory of another object held by the person.

14. The operation method of the display system as claimed in claim 11, wherein the first motion sensor comprises a g-sensor, a gyroscope, an accelerometer, an electronic compass, or an altitude meter.

15. The operation method of the display system as claimed in claim 11, wherein the motion trajectory of the user and the motion trajectory of the person are simultaneously displayed on the image.

16. The operation method of the display system as claimed in claim 11, further comprising:

sensing a sound sent by the user by a sound sensor of the HMD device to generate sound data to the processor; and

performing voice recognition to the sound data to obtain a sound control instruction by the processor.

17. The operation method of the display system as claimed in claim 11, further comprising:

sensing a touch operation of the user by a touch sensor of the HMD device to generate touch data to the processor; and

generating a touch instruction by the processor according to the touch data.

18. The operation method of the display system as claimed in claim 11, further comprising:

sensing a head motion of the user by a second motion sensor of the HMD device to generate second sensing data to the processor.

19. The operation method of the display system as claimed in claim 18, wherein the second motion sensor comprises an electronic compass, a gyroscope, a geomagnetic sensor, an accelerometer, or a g-sensor.

20. The operation method of the display system as claimed in claim 18, further comprising:

generating a menu instruction by the processor according to the second sensing data.

21. The operation method of the display system as claimed in claim 11, further comprising:

correcting the motion trajectory of the person by the processor according to a direction of the person in the video data.