SPATIALLY CORRELATED RENDERING OF THREE-DIMENSIONAL CONTENT ON DISPLAY COMPONENTS HAVING ARBITRARY POSITIONS

• US 20100053164A1
• Filed: 04/15/2009
• Published: 03/04/2010
• Est. Priority Date: 09/02/2008
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Marti, Stefan, Kim, Seung Wook, Imai, Francisco
• Granted Patent: US 8,253,649 B2
• Time in Patent Office: Days
• US Class Current: 345/427
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/1446 display composed of modules, e.g. video walls
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling
• CPC Class Codes: G06T 2219/2021 Shape modification
• CPC Class Codes: G09G 2356/00 Detection of the display position w.r.t. other display screens
• CPC Class Codes: G09G 3/002 to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT
• CPC Class Codes: H04M 2203/359 Augmented reality
• CPC Class Codes: H04N 13/279 the virtual viewpoint locations being selected by the viewers or determined by tracking

Abstract

Two or more display components are used to provide spatially correlated displays of 3D content. Three-dimensional content is rendered on multiple displays where the 3D content refers to the same virtual 3D coordinates, in which the relative position of the displays to each other determines the 3D virtual camera position for each display. Although not required, one of the displays may be mobile, such as a cell phone, and the other stationary or nomadic, such as a laptop. Each display shows a view based on a virtual camera into 3D content, such as an online virtual world. By continuously sensing and updating the relative physical distances and orientations of each device to one another, the devices show the user a view into the 3D content that is spatially correlated. Each device has a virtual camera that uses a common pool of 3D geometrical data and renders this data to display images.

Claims

1. A method of displaying 3D content on a first display component and a second display component utilizing a host server, the method comprising:

obtaining, at the host server, first physical position data of the first display component indicating a relative position to the second display component and second physical position data of the second display component indicating a relative position to the first display component; and

transmitting a first set of rendered data to the first display component and a second set of rendered data to the second display component;

wherein the first set of rendered data is displayed on the first display component and the second set of rendered data is displayed on the second display component; and

wherein the first display component and the second display component enable a spatially correlated rendering of the 3D content and provide an extended field-of-view for a user viewing the 3D content using both the first and second display components.

2. A method as recited in claim 1 further comprising:

obtaining 3D content data representing the 3D content from a local memory on the host server.

3. A method as recited in claim 1 further comprising:

obtaining 3D content data representing the 3D content from a network component.

4. A method as recited in claim 1 further comprising:

executing a rendering engine on the host server, thereby creating the first set of rendered data and the second set of rendered data.

5. A method as recited in claim 1 wherein obtaining first physical position data and second physical position data further comprises:

receiving the first physical position data from the first display component; and

receiving the second physical position data from the second display component.

6. A method as recited in claim 1 wherein obtaining first physical position data and second physical position data further comprises:

detecting the first physical position data and the second physical position data using sensors in the host server.

7. A method as recited in claim 1 further comprising:

creating a third set of rendered data; and

displaying the third set of rendered data on the host server, wherein the host server, the first display component, and the second display component enable a spatially correlated rendering of the 3D content.

8. A method as recited in claim 1 further comprising:

sensing the first physical position data of the first display and the second physical position data of the second display, wherein the sensing is performed continuously, thereby enabling the spatially correlated rendering.

9. A method as recited in claim 1 further comprising:

executing a one-to-one mapping transformation of the first physical position data to a first virtual position and orientation data for the first display and of the second physical position data to a second virtual position and orientation data for the second display.

10. A method of displaying 3D content on a first display component and second display component, the method comprising:

sensing a first physical position of the first display component, wherein the first physical position is in reference to the second display component; and

displaying a first set of rendering data on the first display component;

wherein first physical position data of the first display component is obtained by the second display component; and

wherein the first display component and second display component provide a spatially correlated rendering of the 3D content, thereby enabling an extended field-of-view for a user viewing the 3D content using the first display component and the second display component.

11. A method as recited in claim 10 further comprising:

executing a rendering engine on the first display component

12. A method as recited in claim 10 further comprising:

measuring first egomotion data of the first display component.

13. A method as recited in claim 10 further comprising:

receiving second egomotion data from the second display component.

14. A method as recited in claim 13 further comprising:

propagating the first egomotion data of the first display component to the second display component.

15. A method as recited in claim 10 further comprising:

receiving second physical position data relating to the second display component.

16. A method as recited in claim 10 wherein the first display component and the second display component store the 3D content and wherein the first display component and second display component synchronize when there is a change in the 3D content.

17. A method as recited in claim 10 further comprising:

transmitting a portion of the 3D content to the second display component, wherein the first display component stores all the 3D content and the second display component initially stores no 3D content.

18. A system of display devices for displaying 3D content, the system comprising:

a first display having at least one first location sensor and a first monitor; and

a second display having at least one second location sensor and a second monitor;

wherein rendered pixels of 3D content are transmitted from the first display to the second display, the rendered pixels calculated based on the position data of the second display, such that the first display and the second display enable a spatially correlated rendering of the 3D content, thereby providing an extended field-of-view for a user utilizing the first display and the second display for viewing the 3D content; and

wherein the first display obtains relative position data of the second display and the second display obtains relative position data of the first display.

19. A system as recited in claim 18 wherein the first display executes a multiple-display rendering engine.

20. A system as recited in claim 18 wherein position data is location data and orientation data.

21. An apparatus for displaying 3D content on a first display component and a second display component, the apparatus comprising:

means for obtaining a first physical position data of the first display component indicating a position relative to the second display component and a second physical position data of the second display component indicating a position relative to the first display component; and

means for transmitting a first set of rendered data to the first display component and a second set of rendered data to the second display component;

wherein the first set of rendered data is displayed on the first display component and the second set of rendered data is displayed on the second display component; and

wherein the first display component and the second display component enable a spatially correlated rendering of the 3D content and provide an extended field-of-view for a user viewing the 3D content using both the first and second display components.

22. An apparatus as recited in claim 21 further comprising:

means for obtaining 3D content data representing the 3D content from a network component.

23. An apparatus as recited in claim 21 further comprising:

means for rendering the first set of rendered data and the second set of rendered data.

24. An apparatus as recited in claim 21 wherein means for obtaining a first physical position data and a second physical position data further comprises:

means for receiving the first physical position data from the first display component and the second physical position data from the second display component.

25. An apparatus as recited in claim 21 wherein means for obtaining a first physical position data and a second physical position data further comprises:

means for detecting the first physical position data and the second physical position data.

26. An apparatus as recited in claim 21 wherein means for obtaining a first physical position data and a second physical position data further comprises:

means for receiving the first physical position data from a first sensor on the first display component and the second physical position data from a second sensor on the second display component.

27. An apparatus as recited in claim 21 further comprising:

means for creating a third set of rendered data; and

means for displaying the third set of rendered data, wherein the apparatus, the first display component, and the second display component enable a spatially correlated rendering of the 3D content.

28. An apparatus as recited in claim 21 further comprising:

means for sensing the first physical position data of the first display and the second physical position data of the second display, wherein the sensing is performed continuously, thereby enabling the spatially correlated rendering.

29. An apparatus as recited in claim 21 further comprising:

means for executing a one-to-one mapping transformation of the first physical position data to a first virtual position and orientation data for the first display and of the second physical position data to a second virtual position and orientation data for the second display.

30. An apparatus for displaying 3D content in conjunction with a second display component, the apparatus comprising:

means for sensing a first physical position of the apparatus, wherein the first physical position is in reference to the second display component; and

means for displaying a first set of rendering data on the apparatus;

wherein first physical position data of the apparatus is obtained by the second display component; and

wherein the apparatus and second display component provide a spatially correlated rendering of the 3D content, thereby enabling an extended field-of-view for a user viewing the 3D content using the apparatus and the second display component.

31. An apparatus as recited in claim 30 further comprising:

means for creating the first set of rendering data.

32. An apparatus as recited in claim 30 further comprising:

means for measuring first egomotion data of the apparatus.

33. An apparatus as recited in claim 30 further comprising:

means for propagating the first egomotion data of the apparatus to the second display component.

34. An apparatus as recited in claim 30 further comprising:

means for transmitting a portion of the 3D content to the second display component, wherein the apparatus stores all the 3D content and the second display component initially stores no 3D content.