METHOD AND APPARATUS FOR COMPENSATING FOR MOVEMENT OF HEAD-MOUNTED DISPLAY

• US 20170084074A1
• Filed: 09/20/2016
• Published: 03/23/2017
• Est. Priority Date: 09/21/2015
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Hwang, Seokjoong, Shin, Youngsam, Lee, Wonjong
• Granted Patent: US 10,217,271 B2
• Time in Patent Office: Days
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 15/06 Ray-tracing
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G09G 2320/0252 Improving the response speed
• CPC Class Codes: G09G 2320/0261 in the context of movement of objects on the screen or movement of the observer relative to the screen
• CPC Class Codes: G09G 3/003 to produce spatial visual effects

Abstract

A method of compensating for a movement of a head-mounted display (HMD) includes sensing the movement of the HMD, predicting a movement of the HMD based on the sensed movement, projecting and rendering an image based on the predicted movement, re-sensing the movement of the HMD, re-predicting the movement of the HMD based on the re-sensed movement, determining an error value based on a comparison between the predicted movement of the HMD and the re-predicted movement of the HMD, and re-projecting and rendering the image based on the determined error value.

Claims

1. A method of compensating for a movement of a head-mounted display (HMD), the method comprising:

sensing the movement of the HMD;

predicting a movement of the HMD based on the sensed movement;

projecting and rendering an image based on the predicted movement of the HMD;

re-sensing the movement of the HMD;

re-predicting the movement of the HMD based on the re-sensed movement;

determining an error value based on a comparison between the predicted movement of the HMD and the re-predicted movement of the HMD; and

re-projecting and rendering the image based on the determined error value.

2. The method of claim 1, further comprising performing image space transformation on the re-projected and rendered image.

3. The method of claim 1, wherein the determined error value comprises a horizontal error value, and

wherein the re-projecting and rendering comprises re-projecting and rendering the image in a horizontal direction based on the determined error value.

4. The method of claim 1, wherein the determined error value comprises a vertical error value, and

wherein the re-projecting and rendering comprises re-projecting and rendering the image in a vertical direction based on the determined error value.

5. The method of claim 1, wherein the determined error value comprises a depth error value, and

wherein the re-projecting and rendering comprises re-projecting and rendering the image in a depth direction based on the determined error value.

6. The method of claim 1, wherein the determined error value comprises at least one of horizontal error value, a vertical error value, and a depth error value, and

wherein the re-projecting and rendering comprises:

determining a sequence of a horizontal direction re-projection based on the horizontal error value, a vertical direction re-projection based on the vertical error value, and a depth direction re-projection based on the depth error value; and

re-projecting and rendering the image in the determined sequence.

7. The method of claim 1, wherein the re-projecting and rendering comprises re-projecting and rendering the image with respect to one view point or two view points.

8. The method of claim 1,

wherein the predicting comprises predicting the movement of the HMD at a one-eye view point or at both-eye view points, and

wherein the rendering of the image comprises rendering the image at the one-eye view point, at a view point between the both-eye view points, or at an intermediate value view point of the both-eye view points.

9. The method of claim 1, wherein the rendering of the image comprises rasterizing and rendering the image or ray tracing and rendering the image.

10. The method of claim 6, wherein the determining of the sequence comprises:

in response to the horizontal error value being greater than the vertical error value, determining the sequence such that the image is re-projected in a horizontal direction and then in a vertical direction; and

in response to the horizontal error value being less than the vertical error value, determining the sequence such that the image is re-projected in the vertical direction and then in the horizontal direction.

11. The method of claim 6, wherein the determining of the sequence comprises determining the sequence based on whether at least one of the horizontal error value and the vertical error value exceeds a threshold value.

12. The method of claim 6, wherein the determining of the sequence comprises determining the sequence for each image region.

13. The method of claim 1, further comprising outputting the rendered image on the HMD.

27. A non-transitory computer-readable recording medium having recorded thereon a computer program for executing the method of claim 1.

14. An apparatus for compensating for a movement of a head-mounted display (HMD), the apparatus comprising:

a sensor configured to sense the movement of the HMD; and

a processor configured to predict a movement of the HMD based on the movement sensed by the sensor, and project and render an image based on the predicted movement of the HMD,

wherein the processor is further configured to re-predict the movement of the HMD based on the re-sensed movement, determine an error value by comparing the predicted movement of the HMD and the re-predicted movement of the HMD, and re-project and render the image based on the determined error value.

15. The apparatus of claim 14, wherein the processor is further configured to perform image space transformation on the re-projected and rendered image.

16. The apparatus of claim 14, wherein the determined error value comprises a horizontal error value, and

wherein the processor is further configured to re-project and render the image in a horizontal direction based on the determined error value.

17. The apparatus of claim 14, wherein the determined error value comprises a vertical error value, and

wherein the processor is further configured to re-project and render the image in a vertical direction based on the determined error value.

18. The apparatus of claim 14, wherein the determined error value comprises a depth error value, and

wherein the processor is further configured to re-project and render the image in a depth direction based on the determined error value.

19. The apparatus of claim 14, wherein the determined error value comprises at least one of a horizontal error value, a vertical error value, and a depth error value, and

wherein the processor is further configured to determine a sequence of a horizontal direction re-projection based on the horizontal error value, a vertical direction re-projection based on the vertical error value, and a depth direction re-projection based on the depth error value, and re-project and render the image in the determined sequence.

20. The apparatus of claim 14, wherein the processor is further configured to re-project and render the image with respect to one view point or two view points.

21. The apparatus of claim 14,

wherein the processor is further configured to predict the movement of the HMD at a one-eye view point or at both-eye view points, and render the image at the one-eye view point, at a view point between the both-eye view points, or at an intermediate value view point of the both-eye view points.

22. The apparatus of claim 14, wherein the processor is further configured to rasterize and render the image or to ray trace and render the image.

23. The apparatus of claim 19, wherein the processor is further configured to, in response to the horizontal error value being greater than the vertical error value, determine the sequence such that the image is re-projected in a horizontal direction and then in a vertical direction, and, in response to the horizontal error value being smaller than the vertical error value, determine the sequence such that the image is re-projected in the vertical direction and then in the horizontal direction.

24. The apparatus of claim 19, wherein the processor is further configured to determine the sequence based on whether at least one of the horizontal error value or the vertical error value exceeds a threshold value.

25. The apparatus of claim 19, wherein the processor is further configured to determine the sequence for each image region.

26. The apparatus of claim 14, further comprising an outputter configured to output the rendered image on the HMD.