System and method for rendering in accordance with location of virtual objects in real-time

• US 9,129,644 B2
• Filed: 06/23/2009
• Issued: 09/08/2015
• Est. Priority Date: 06/23/2009
• Status: Active Grant

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: Gay, Michael, Thiel, Aaron
• Primary Examiner(s): Nguyen, Kimbinh T
• Application Number:
  US12/456,813
• Publication Number:
  US 20100321389A1
• Time in Patent Office: 2,268 Days
• Field of Search: 345/632, 345/633, 345/419, 345/630, 345/474, 463/1, 463/4, 348/157
• US Class Current: 1/1
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G11B 27/036 Insert-editing

Abstract

There is provided a system and method for rendering in accordance with location of virtual objects in real-time. There is provided a method for persistent association of a graphic overlay with a virtual object in a displayable environment, comprising receiving a first three-dimensional coordinate of the virtual object in the displayable environment, determining a three-dimensional coordinate of the graphic overlay in accordance with the first three-dimensional coordinate of the virtual object, tracking a movement of the virtual object in the displayable environment by receiving one or more second three-dimensional coordinates of the virtual object, and modifying the three-dimensional coordinate of the graphic overlay in accordance with the one or more second three-dimensional coordinates of the virtual object.

Claims

1. A method for persistent association of a graphic overlay with a first virtual object in a displayable environment, the method comprising:

receiving a first three-dimensional coordinate of the first virtual object in the displayable environment;

determining a three-dimensional coordinate of the graphic overlay in accordance with the first three-dimensional coordinate of the first virtual object, wherein the determining the three-dimensional coordinate of the graphic overlay applies positional rules so that the graphic overlay does not obscure both the first virtual object and one or more other objects in the displayable environment;

tracking a movement of the first virtual object in the displayable environment by receiving one or more second three-dimensional coordinates of the first virtual object from a remote virtual simulator;

modifying the three-dimensional coordinate of the graphic overlay in accordance with the one or more second three-dimensional coordinates of the first virtual object, wherein modifying the three-dimensional coordinate of the graphic overlay includes repositioning the graphic overlay to avoid obscuring the first virtual object and the one or more other objects in the displayable environment; and

rendering the graphic overlay and the displayable environment for displaying on a display.

2. The method of claim 1 further comprising:

receiving a first three-dimensional coordinate of a real object in the displayable environment further-modifying the three-dimensional coordinate of the graphic overlay in accordance with the first three-dimensional coordinate of the real object.

3. The method of claim 2, wherein the real object is a static object, and the one or more other_objects are virtual objects.

4. The method of claim 1 further comprising:

receiving a first three-dimensional coordinate of a second virtual object in the displayable environment further modifying the three-dimensional coordinate of the graphic overlay in accordance with the first three-dimensional coordinate of the second virtual object.

5. The method of claim 1 , wherein the determining the three-dimensional coordinate of the graphic overlay further uses metadata of the first virtual object.

6. The method of claim 5, wherein the metadata includes translation data.

7. The method of claim 5, wherein the metadata includes motion data.

8. The method of claim 5, wherein the metadata includes priority data.

9. The method of claim 1, wherein the graphic overlay comprises text window.

10. The method of claim 1, wherein the graphic overlay comprises a graphical arrow.

11. A rendering device for rendering a displayable environment with persistent association of a graphic overlay with a first virtual object, the rendering device comprising:

a processor configured to;

receive a first three-dimensional coordinate of the first virtual object in the displayable environment;

determine a three-dimensional coordinate of the graphic overlay in accordance with the first three-dimensional coordinate of the first virtual object, wherein the three-dimensional coordinate of the graphic overlay is determined using positional rules so that the graphic overlay does not obscure both the first virtual object and one or more other objects in the displayable environment;

track a movement of the first virtual object in the displayable environment by receiving one or more second three-dimensional coordinates of the first virtual object from a remote-virtual simulator;

modify the three-dimensional coordinate of the graphic overlay in accordance with the one or more second three-dimensional coordinates of the first virtual object, wherein modifying the three-dimensional coordinate of the graphic overlay includes repositioning the graphic overlay to avoid obscuring the first virtual object and the one or more other objects in the displayable environment; and

render the graphic overlay and displayable environment on a display.

12. The rendering device of claim 11, wherein the processor is further configured to:

receive a first three-dimensional coordinate of a real object in the displayable environment further modifying the three-dimensional coordinate of the graphic overlay in accordance with the first three-dimensional coordinate of the real object.

13. The rendering device of claim 12, wherein the real object is a static object, and the one or more other objects are virtual objects.

14. The rendering device of claim 11, wherein the processor is further configured to:

receive a first three-dimensional coordinate of a second virtual object in the displayable environment further modifying the three-dimensional coordinate of the graphic overlay in accordance with the first three-dimensional coordinate of the second virtual object.

15. The rendering device of claim 11, wherein the processor is further configured to determine the three-dimensional coordinate of the graphic overlay by using metadata of the first virtual object.

16. The rendering device of claim 15, wherein the metadata includes translation data.

17. The rendering device of claim 15, wherein the metadata includes motion data.

18. The rendering device of claim 15, wherein the metadata includes priority data.

19. The rendering device of claim 11, wherein the graphic overlay comprises a text window.

20. The rendering device of claim 11, wherein the graphic overlay comprises a graphical arrow.