Head Mounted Display Device and Processing Method of Head Mounted Display Device

• US 20190206093A1
• Filed: 06/22/2016
• Published: 07/04/2019
• Est. Priority Date: 06/22/2016
• Status: Active Grant

• Current Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Inventors: CHEN, Xin, Gao, Wenmei
• Granted Patent: US 10,672,149 B2
• Time in Patent Office: Days
• CPC Class Codes: G02B 2027/0134 of stereoscopic type
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0147 comprising a device modifying the resolution of the displayed image
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06F 3/1423 controlling a plurality of local displays, e.g. CRT and flat panel display
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 3/04 Context-preserving transformations, e.g. by using an importance map panospheric to cylindrical image transformations G06T3/12
• CPC Class Codes: G09G 2320/0261 in the context of movement of objects on the screen or movement of the observer relative to the screen
• CPC Class Codes: G09G 2340/0464 Positioning
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 2370/16 Use of wireless transmission of display information
• CPC Class Codes: G09G 5/38 with means for controlling the display position

Abstract

This application relates to the communications field, and in particular, to a head mounted display device for extending a display area of an electronic device and a processing method of the head mounted display device. A display of the electronic device displays first content of an image. The image further includes second content that is not displayed on the display of the electronic device. The head mounted display device obtains, by using an image sensor, the first content displayed on the display of the electronic device, performs projection transformation on the second content, and displays the obtained first content and the second content obtained after the projection transformation on the display of the head mounted display device. Therefore, when image resolution of a to-be-displayed image is greater than display resolution of the display of the electronic device, the entire image is clear without needing to be scaled down or cropped.

Claims

1. A method, applied to a head mounted display device with a display and an image sensor, wherein the method comprises:

establishing a connection to an electronic device, wherein the electronic device has a display, a defined three-dimensional coordinate system, and a feature point that has known three-dimensional coordinates in the three-dimensional coordinate system, wherein the display of the electronic device displays first content of an image, wherein image resolution of the image is greater than display resolution of the display of the electronic device, and wherein the image further comprises second content that is not displayed on the display of the electronic device;

receiving data sent by the electronic device, wherein the data comprises a grayscale value of each pixel in the second content

performing projection transformation on the second content according to the three-dimensional coordinates of the feature point and the grayscale value;

obtaining, by using the image sensor, the first content displayed on the display of the electronic device; and

displaying, on the display of the head mounted display device, the obtained first content and the second content obtained after the projection transformation.

2. The method according to claim 1, further comprising, before the first content and the second content are displayed on the display of the head mounted display device:

tracking the feature point; and

calculating a posture of the image sensor relative to the display of the electronic device.

3. The method according to claim 1, wherein the image comprises a document and a virtual keyboard used to edit the document.

4. The method according to claim 1, wherein the image comprises a static image or a dynamic image.

12. The method according to claim 1, wherein the image is a web page and the electronic device comprises a smartwatch.

5-11. -11. (canceled)

13. A head-mounted display device, comprising:

a display;

an image sensor;

a memory comprising instructions; and

at least one processor coupled to the memory, the instructions being executed by the at least one processor to cause the head-mounted display device to be configured to;

establish a connection to an electronic device, wherein the electronic device has a display, a defined three-dimensional coordinate system, and a feature point that has known three-dimensional coordinates in the three-dimensional coordinate system, wherein the display of the electronic device displays first content of an image, wherein image resolution of the image is greater than display resolution of the display of the electronic device, and wherein the image further comprises second content that is not displayed on the display of the electronic device;

receive data sent by the electronic device, wherein the data comprises a grayscale value of each pixel in the second content;

perform projection transformation on the second content according to the three-dimensional coordinates of the feature point and the grayscale value;

obtain, by using the image sensor, the first content displayed on the display of the electronic device; and

display, on the display of the head mounted display device, the obtained first content and the second content obtained after the projection transformation.

14. The head-mounted display device of claim 13, the instructions further cause the head-mounted display device to:

track the feature point; and

calculate a posture of the image sensor relative to the display of the electronic device.

15. The head-mounted display device of claim 13, wherein the image comprises a document and a virtual keyboard used to edit the document.

16. The head-mounted display device of claim 13, wherein the image comprises a static image or a dynamic image.

17. The head-mounted display device of claim 13, wherein the image is a web page and the electronic device comprises a smartwatch.