Spatially correlated rendering of three-dimensional content on display components having arbitrary positions

• US 8,253,649 B2
• Filed: 04/15/2009
• Issued: 08/28/2012
• Est. Priority Date: 09/02/2008
• Status: Expired due to Fees

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Imai, Francisco, Kim, Seung Wook, Marti, Stefan
• Primary Examiner(s): Wu, Xiao M.
• Assistant Examiner(s): He, Yingchun
• Application Number:
  US12/424,117
• Publication Number:
  US 20100053164A1
• Time in Patent Office: 1,231 Days
• Field of Search: 345/419, 345/427, 345/1.3
• US Class Current: 345/1.3
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/1446 display composed of modules, e.g. video walls
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling
• CPC Class Codes: G06T 2219/2021 Shape modification
• CPC Class Codes: G09G 2356/00 Detection of the display position w.r.t. other display screens
• CPC Class Codes: G09G 3/002 to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT
• CPC Class Codes: H04M 2203/359 Augmented reality
• CPC Class Codes: H04N 13/279 the virtual viewpoint locations being selected by the viewers or determined by tracking

Abstract

Two or more display components are used to provide spatially correlated displays of 3D content. Three-dimensional content is rendered on multiple displays where the 3D content refers to the same virtual 3D coordinates, in which the relative position of the displays to each other determines the 3D virtual camera position for each display. Although not required, one of the displays may be mobile, such as a cell phone, and the other stationary or nomadic, such as a laptop. Each display shows a view based on a virtual camera into 3D content, such as an online virtual world. By continuously sensing and updating the relative physical distances and orientations of each device to one another, the devices show the user a view into the 3D content that is spatially correlated. Each device has a virtual camera that uses a common pool of 3D geometrical data and renders this data to display images.

Claims

1. A method of displaying 3D content on a first display component and a second display component utilizing a host server, the method comprising:

obtaining, at the host server, first physical position data of the first display component indicating a relative position to the host server and second physical position data of the second display component indicating a relative position to the host server;

executing a multiple-display rendering engine on the host server for spatially-correlating multiple displays, the engine containing virtual cameras for determining specific rendering perspectives into a virtual world provided by the first display component and the second display component, thereby creating a first set of rendered data and a second set of rendered data;

receiving virtual world coordinates; and

transmitting the first set of rendered data to the first display component and the second set of rendered data to the second display component;

wherein the first set of rendered data is displayed on the first display component and the second set of rendered data is displayed on the second display component; and

wherein the first display component and the second display component enable a spatially correlated rendering of the 3D content and provide an extended field-of-view for a user viewing the 3D content using both the first and second display components.

2. A method as recited in claim 1 further comprising:

obtaining 3D content data representing the 3D content from a local memory on the host server.

3. A method as recited in claim 1 further comprising:

obtaining 3D content data representing the 3D content from a network component.

4. A method as recited in claim 1 wherein obtaining first physical position data and second physical position data further comprises:

receiving the first physical position data from the first display component; and

receiving the second physical position data from the second display component.

5. A method as recited in claim 1 wherein obtaining first physical position data and second physical position data further comprises:

detecting the first physical position data and the second physical position data using sensors in the host server.

6. A method as recited in claim 1 further comprising:

creating a third set of rendered data; and

displaying the third set of rendered data on the host server, wherein the host server, the first display component, and the second display component enable a spatially correlated rendering of the 3D content.

7. A method as recited in claim 1 further comprising:

sensing the first physical position data of the first display and the second physical position data of the second display, wherein the sensing is performed continuously, thereby enabling the spatially correlated rendering.

8. A method as recited in claim 1 further comprising:

executing a one-to-one mapping transformation of the first physical position data to a first virtual position and orientation data for the first display and of the second physical position data to a second virtual position and orientation data for the second display.

9. A method of displaying 3D content on a first display component and a second display component, the method comprising:

sensing a first physical position of the first display component, wherein the first physical position is relative to the second display component; and

displaying a first set of rendering data on the first display component;

sensing a second physical position of the second display component, wherein the second physical position is relative to the first display component; and

displaying a second set of rendering data on the second display component;

wherein a first physical position data of the first display component is obtained by the second display component and a second physical position data of the second display component is obtained by the first display component;

executing a multiple-display rendering engine on the first display component, the engine containing a virtual camera for determining a first specific rendering perspective for the first display component into a virtual world provided by the first display component and for determining a second specific rendering perspective for the second display component into the virtual world by the second display component;

receiving virtual world coordinates; and

wherein the first display component and second display component provide a spatially correlated rendering of the 3D content, thereby enabling an extended field-of-view for a user viewing the 3D content using the first display component and the second display component.

10. A method as recited in claim 9 further comprising:

measuring first egomotion data of the first display component.

11. A method as recited in claim 9 further comprising:

receiving second egomotion data from the second display component.

12. A method as recited in claim 11 further comprising:

propagating the first egomotion data of the first display component to the second display component.

13. A method as recited in claim 9 further comprising:

receiving second physical position data relating to the second display component.

14. A method as recited in claim 9 wherein the first display component and the second display component store the 3D content and wherein the first display component and second display component synchronize when there is a change in the 3D content.

15. A method as recited in claim 9 further comprising:

transmitting a portion of the 3D content to the second display component, wherein the first display component stores all the 3D content and the second display component initially stores no 3D content.

16. An apparatus for displaying 3D content on a first display component and a second display component, the apparatus comprising:

means for obtaining a first physical position data of the host server indicating a position relative to the second display component and a second physical position data of the second display component indicating a position relative to the host server;

means for executing a multiple-display rendering engine on the host server for spatially-correlating multiple displays, the engine containing virtual cameras for determining specific rendering perspectives into a virtual world provided by the first display component and the second display component, thereby creating a first set of rendered data and a second set of rendered data;

means for receiving virtual world coordinates; and

means for transmitting the first set of rendered data to the first display component and the second set of rendered data to the second display component;

wherein the first set of rendered data is displayed on the first display component and the second set of rendered data is displayed on the second display component; and

wherein the first display component and the second display component enable a spatially correlated rendering of the 3D content and provide an extended field-of-view for a user viewing the 3D content using both the first and second display components.

17. An apparatus as recited in claim 16 further comprising:

means for obtaining 3D content data representing the 3D content from a network component.

18. An apparatus as recited in claim 16 further comprising:

means for rendering the first set of rendered data and the second set of rendered data.

19. An apparatus as recited in claim 16 wherein means for obtaining a first physical position data and a second physical position data further comprises:

means for receiving the first physical position data from the first display component and the second physical position data from the second display component.

20. An apparatus as recited in claim 16 wherein means for obtaining a first physical position data and a second physical position data further comprises:

means for detecting the first physical position data and the second physical position data.

21. An apparatus as recited in claim 16 wherein means for obtaining a first physical position data and a second physical position data further comprises:

means for receiving the first physical position data from a first sensor on the first display component and the second physical position data from a second sensor on the second display component.

22. An apparatus as recited in claim 16 further comprising:

means for creating a third set of rendered data; and

means for displaying the third set of rendered data, wherein the apparatus, the first display component, and the second display component enable a spatially correlated rendering of the 3D content.

23. An apparatus as recited in claim 16 further comprising:

means for sensing the first physical position data of the first display and the second physical position data of the second display, wherein the sensing is performed continuously, thereby enabling the spatially correlated rendering.

24. An apparatus as recited in claim 16 further comprising:

means for executing a one-to-one mapping transformation of the first physical position data to a first virtual position and orientation data for the first display and of the second physical position data to a second virtual position and orientation data for the second display.

25. An apparatus for displaying 3D content in conjunction with a second display component, the apparatus comprising:

means for sensing a first physical position of the apparatus, wherein the first physical position is relative to the second display component;

means for displaying a first set of rendering data on the apparatus;

means for sensing a second physical position of the second display component, wherein the second physical position is relative to the first display component; and

means for displaying a second set of rendering data on the second display component;

wherein a first physical position data of the apparatus is obtained by the second display component and a second physical position data of the second display component is obtained by the first display component;

means for executing a multiple-display rendering engine on the first display component, the engine containing a virtual camera for determining a first specific rendering perspective for the first display component into a virtual world provided by the first display component and for determining a second specific rendering perspective for the second display component into the virtual world by the second display component; and

means for receiving virtual world coordinates;

wherein the apparatus and second display component provide a spatially correlated rendering of the 3D content, thereby enabling an extended field-of-view for a user viewing the 3D content using the apparatus and the second display component.

26. An apparatus as recited in claim 25 further comprising:

means for creating the first set of rendering data.

27. An apparatus as recited in claim 25 further comprising:

means for measuring first egomotion data of the apparatus.

28. An apparatus as recited in claim 25 further comprising:

means for propagating the first egomotion data of the apparatus to the second display component.

29. An apparatus as recited in claim 25 further comprising:

means for transmitting a portion of the 3D content to the second display component, wherein the apparatus stores all the 3D content and the second display component initially stores no 3D content.