Real-time view synchronization across multiple networked devices

• US 8,842,113 B1
• Filed: 01/13/2011
• Issued: 09/23/2014
• Est. Priority Date: 05/26/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Holt, Jason, Barcay, Daniel
• Primary Examiner(s): Xiao, Ke
• Assistant Examiner(s): Le, Sarah
• Application Number:
  US13/005,845
• Time in Patent Office: 1,349 Days
• Field of Search: 345/419, 345/2.1, 345/2.2, 345/1.1
• US Class Current: 345/419
• CPC Class Codes: G06F 3/1446 display composed of modules, e.g. video walls
• CPC Class Codes: G09G 2356/00 Detection of the display position w.r.t. other display screens
• CPC Class Codes: G09G 5/12 Synchronisation between the display unit and other units, e.g. other display units, video-disc players

Abstract

Embodiments relate to a method for synchronizing media playback across multiple computing devices on a network. Multiple display screens, each coupled to a computing device, can be arranged in a circular formation surrounding a user. For example, a special enclosure for the display screens may be constructed for this purpose. An immersive viewing experience is created for the user by synchronizing the views displayed on each screen. The multiple screens operate in unison to create a tiled view of a three-dimensional environment. This immersive experience can be referred to as a “Liquid Galaxy.” The experience is enhanced by allowing the user to view and navigate through such three-dimensional environments using an input device, such as, for example, a joystick or game controller.

Claims

1. A computer-implemented method for real-time view synchronization comprising:

receiving, by one or more second computing devices from a first computing device, a view synchronization message associated with a first view of media content from the first computing device, the first view rendered by the first computing device using a first display having a first physical location, wherein the first view is rendered from a first perspective of a first three-dimensional virtual camera located in a geographic information system (GIS) environment and wherein the view synchronization message includes a set of camera parameters, the set of camera parameters describing a position and orientation of the first virtual camera at the first perspective;

positioning, by the one or more second computing devices, a second three-dimensional virtual camera in the GIS environment according to the set of camera parameters included in the received view synchronization message associated with the first view;

obtaining, by the one or more second computing devices, from a configuration file a set of camera position settings associated with the one or more second computing devices, wherein the set of camera position settings includes one or more predefined camera offsets corresponding to a defined, static physical offset of a second physical location from the first physical location, the second physical location being associated with a second display, and wherein the set of camera position settings are pre-stored in the configuration file;

moving, by the one or more second computing devices, the second three-dimensional virtual camera according to the one or more camera offsets such that the second virtual camera has a second perspective providing a second view of the media content that is different from the first perspective; and

rendering, by the one or more second computing devices, the second view of the media content from the second perspective of the second three-dimensional virtual camera, the second view rendered using the second display having the second physical location, such that the second view is synchronized in real time with the first view of the media content displayed on the first.

2. The method of claim 1, wherein the set of camera parameters comprises a position value corresponding to the first perspective of the three-dimensional virtual camera,

wherein the positioning comprises:

positioning, by the one or more second computing devices, the second three-dimensional virtual camera according to the received position value corresponding to the first perspective of the three-dimensional virtual camera, and

wherein the one or more camera offsets comprise a rotation value and the moving comprises;

rotating, by the one or more second computing devices, the second three-dimensional virtual camera based on the rotation value.

3. The method of claim 1, wherein the media content includes multiple synchronized video streams.

4. The method of claim 1, wherein the media content includes real-time graphical application content.

5. The method of claim 1, wherein the received view synchronization message associated with the first view includes a plurality of rendering parameters, and

wherein the rendering further comprises:

rendering, by the one or more second computing devices, the second view of the media content based on the plurality of rendering parameters.

6. The method of claim 5, wherein the plurality of rendering parameters include lighting parameters corresponding to one or more frames of the media content.

7. The method of claim 5, wherein the plurality of rendering parameters include time parameters corresponding to one or more frames of the media content.

8. The method of claim 5, wherein the plurality of rendering parameters include feature content parameters corresponding to one or more frames of the media content.

9. The method of claim 1, wherein:

the view synchronization message further includes activation data associated with the first view of the media content, wherein the activation data identifies whether one or more content features of the first view are active or inactive; and

rendering, by the one or more second computing devices, the second view of the media content further comprises rendering in the second view of media content, by the one or more second computing devices, the content features identified as active by the activation data included in the view synchronization message.

10. The method of claim 9, wherein the one or more content features comprise at least one of sunlight rendering, historical imagery rendering, water surface rendering, and photo mode rendering.

11. A computer-implemented method for real-time view synchronization comprising:

receiving user input from a user at a first computing device;

rendering, by the first computing device, a first view of media content using a first display, the first display having a first physical location, wherein the first view is displayed from a first perspective of a first three-dimensional virtual camera located in a geographic information system (GIS) environment; and

transmitting, by the first computing device, a view synchronization message based on the rendered first view to one or more second computing devices having one or more respective second displays respectively located at one or more second physical locations having respective defined, static physical offsets with respect to the first physical location, wherein the view synchronization message includes a set of camera parameters describing a position and orientation of the first virtual camera at the first perspective;

wherein the one or more second computing devices are configured to;

respectively position one or more second virtual cameras in the GIS environment according to the set of camera parameters included in the view synchronization message;

respectively more the one or more second virtual cameras according to one or more predefined camera offsets corresponding to the one or more respective defined, static physical offsets, such that one or more second views from one or more second perspectives are obtained, the one or more second perspectives being different from the first perspective, wherein the predefined camera offsets are included in a set of camera position settings associated with the one or more second computing devices, and wherein the set of camera position settings are pre-stored in a configuration file; and

respectively render the one or more second views from the one or more second perspectives of the second three-dimensional virtual camera in the GIS environment, different from the first perspective of the first three-dimensional virtual camera in the GIS environment, wherein the one or more second views are rendered by the one or more second computing devices using the one or more respective second displays, whereby the first and one or more second displays are physically arranged to at least partially surround the user so as to create an immersive and interactive three-dimensional experience for the user.

12. A method for real-time synchronization of views across a plurality of computing devices coupled to a plurality of displays on a network within a geographic information system (GIS) environment comprising:

(a) arranging the plurality of displays coupled to the plurality of computing devices in a formation around a common point to partially or fully surround a user;

(b) sending a view synchronization message from a master device of the plurality of computing devices, wherein the view synchronization message corresponds to a master view of media content;

(c) displaying the master view using a master display, of the plurality of displays, coupled to the master device, the master view displayed at a first perspective of a first three-dimensional virtual camera in the geographic information system (GIS) environment;

(d) receiving the view synchronization message from the master device at one or more slave devices of the plurality of computing devices, wherein the view synchronization message includes a set of camera parameters describing a position and orientation of the first virtual camera at the first perspective;

(e) positioning one or more second three-dimensional virtual cameras respectively associated with the one or more slave devices according to the set of camera parameters included in the view synchronization message;

(f) obtaining, from one or more configuration files, one or more sets of camera position settings respectively associated with the one or more slave devices, wherein the one or more sets of camera position settings are respectively pre-stored in the one or more configuration files, and wherein the set of camera position settings for each slave device includes one or more predefined camera offsets corresponding to a position of a slave display coupled to such slave device in the formation relative to a position of the master display in the formation, wherein the position of each slave display relative to the position of the master display is defined and static;

(g) adjusting the one or more second three-dimensional virtual cameras based on the one or more sets of camera position settings, such that one or more slave views are adjusted to one or more second perspectives of the second three-dimensional virtual cameras in the GIS environment, different from the first perspective of the first three-dimensional virtual camera in the GIS environment; and

(h) displaying the one or more slave views of the media content synchronized with the master view using the one or more slave displays, of the plurality of displays, coupled to the one or more slave devices.

13. The method of claim 12, wherein the sending (b) comprises:

(i) determining a camera position and a rotation of a master virtual camera corresponding to the master view of the media content;

(ii) generating the view-synchronization message based on the determined camera position and rotation of the master virtual camera; and

(iii) sending the generated view-synchronization message.

14. The method of claim 13, wherein the adjusting (g) comprises:

(i) extracting the camera position and rotation of the master virtual camera from the view-synchronization message; and

(ii) adjusting one or more slave virtual cameras based on the extracted camera position and the rotation of the master virtual camera and the position of each or one or more slave displays in the formation relative to the position of the master display in the formation.

15. The method of claim 13, wherein the sending (b) further comprises:

(iv) determining a network address and a communication port,

wherein the generated view-synchronization message is sent to the determined network address and the determined communication port.

16. The method of claim 15, wherein the determining the network address and the communication port comprises accessing the network address and the communication port from a configuration file stored at the master device.

17. The method of claim 16, wherein the network address is a UDP broadcast address.

18. The method of claim 12, wherein the media content comprises digital video streams.

19. The method of claim 12,

wherein the displaying (c) comprises displaying a master section of a view frustum using the master display, of the plurality of displays, coupled to the master device, and

wherein the displaying (h) comprises displaying slave sections of the view frustum using the one or more slave displays, of the plurality of displays, coupled to the one or more slave devices.

20. A system for real-time synchronization of views across a plurality of computing devices coupled to a plurality of displays on a network within a geographic information system (GIS), the plurality of displays arranged in a formation around a common point to partially or fully surround a user, the system comprising:

a master device, of the plurality of computing devices, to send a view synchronization message, wherein the view synchronization message corresponds to a master view of media content and includes a set of camera parameters describing the position and orientation of a first virtual camera providing a first perspective in the geographic information system (GIS) environment, the first perspective corresponding to the master view of the media content;

a master display, coupled to the master device, to display the master view; and

one or more slave devices, of the plurality of computing devices, to receive the view synchronization message from the master device, the one or more slave devices coupled to one or more slave displays of the plurality of displays, to obtain one or more predefined camera offsets corresponding to a defined, static position of each of the one or more slave displays in the formation relative to a position of the master display in the formation, wherein the predefined camera offsets are included in a set of camera position settings associated with the one or more slave devices, the set of camera position settings being pre-stored in one or more configuration files from which they are respectively obtained by the one or more slave devices, and to adjust one or more second virtual cameras in the GIS environment from the first perspective based on the one or more predefined camera offsets, such that the one or more second virtual cameras are adjusted to one or more second perspectives in the GIS environment, different from the first perspective of the first three-dimensional virtual camera in the GIS environment,

wherein the one or more slave displays are configured to display one or more slave views of the media content, the one or more slave views respectively corresponding to the one or more second perspectives of the one or more second virtual cameras.