System for streaming databases serving real-time applications used through streaming interactive video

• US 10,434,425 B2
• Filed: 07/18/2017
• Issued: 10/08/2019
• Est. Priority Date: 12/10/2002
• Status: Expired due to Fees

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment America LLC (Sony Group Corp.)
• Inventors: Perlman, Stephen G., van der Laan, Roger
• Primary Examiner(s): Thai, Xuan M
• Assistant Examiner(s): Blaise, Malina D.
• Application Number:
  US15/653,463
• Publication Number:
  US 20170312624A1
• Time in Patent Office: 812 Days
• CPC Class Codes: A63F 13/12 involving interaction between a plurality of game devices, e.g. transmisison or distribution systems protocols for networked virtual reality, networked games H04L29/06034
• CPC Class Codes: A63F 13/30 Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
• CPC Class Codes: A63F 13/335 using Internet
• CPC Class Codes: A63F 13/352 involving special game server arrangements, e.g. regional servers connected to a national server or a plurality of servers managing partitions of the game world
• CPC Class Codes: A63F 13/533 for prompting the player, e.g. by displaying a game menu
• CPC Class Codes: A63F 13/87 Communicating with other players during game play, e.g. by e-mail or chat
• CPC Class Codes: A63F 2300/402 Communication between platforms, i.e. physical link to protocol
• CPC Class Codes: A63F 2300/513 server hierarchy, e.g. local, regional, national or dedicated for different tasks, e.g. authenticating, billing
• CPC Class Codes: A63F 2300/538 for performing operations on behalf of the game client, e.g. rendering
• CPC Class Codes: A63F 2300/552 for downloading to client devices, e.g. using OS version, hardware or software profile of the client device
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 65/65 Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/756 adapting media to device capabilities
• CPC Class Codes: H04L 65/765 intermediate
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04N 21/21 Server components or server architectures
• CPC Class Codes: H04N 21/2143 located in a single building, e.g. hotel, hospital or museum arrangements specially adapted for plural spots in a confined site in broadcast systems H04H20/63; adaptations for transmission by electric cable for domestic distribution in television systems H04N7/106
• CPC Class Codes: H04N 21/23 Processing of content or additional data; Elementary server operations; Server middleware
• CPC Class Codes: H04N 21/4781 Games
• CPC Class Codes: H04N 21/6125 : involving transmission via Internet transmission by internet of broadcast information H04H60/82
• CPC Class Codes: H04N 7/106 : for domestic distribution
• CPC Class Codes: H04W 84/12 : WLAN [Wireless Local Area Networks]
• CPC Class Codes: H04W 88/14 : Backbone network devices

Abstract

An apparatus includes one or more servers of a hosting service center operable to execute a twitch video game or real-time application for one or more users remotely located from the hosting service center. The execution of the twitch video game or real-time application on the one or more servers produces uncompressed video of 3D animation. The one or more servers are operable to integrate live video with the twitch video game or real-time application such that the live video appears within the uncompressed video of 3D animation. The live video is received at a hosting service center from one or more client devices correspondingly associated with the one or more users. The uncompressed video 3D animation is generated, at least partially, using data streamed from a high-speed storage unit of the hosting service center coupled to the one or more servers. The high-speed storage unit is configured to load geometry for efficient loading and rendering of objects of the 3D animations. A compression unit compresses the uncompressed video 3D animation integrated with the live video, and compressed streaming interactive video being produced therefrom. An outbound routing network device coupled to the compression unit that transmits the compressed streaming interactive video over a packetized network to the one or more users.

Claims

1. An apparatus, comprising:

one or more servers of a hosting service center operable to execute a twitch video game or real-time application for one or more users remotely located from the hosting service center;

the execution of the twitch video game or real-time application on the one or more servers producing uncompressed video of 3D animation, the one or more servers being further operable to integrate live video with the twitch video game or real-time application such that the live video appears within the uncompressed video of 3D animation;

the live video being received at the hosting service center from one or more client devices correspondingly associated with the one or more users, the receiving of the live video being via an inbound routing network of the hosting service center for direct transfer to the one or more servers;

the uncompressed video of 3D animation is generated, at least partially, using data streamed from a high-speed storage unit of the hosting service center coupled to the one or more servers via said inbound routing, the high-speed storage unit is configured to load geometry for efficient loading and rendering of objects of the 3D animations;

a compression unit that compresses the uncompressed video of 3D animation integrated with the live video, compressed streaming interactive video being produced therefrom; and

outbound routing network devices coupled to the compression unit that transmits the compressed streaming interactive video over a packetized network to the one or more users.

2. The apparatus of claim 1 wherein the uncompressed video of 3D animation is produced in substantial real-time in response to a user control input.

3. The apparatus of claim 1, wherein the data streamed from the high-speed storage unit is determined, at least in part, by interactive user input coupled to the one or more servers.

4. The apparatus of claim 1, wherein the data streamed from the high-speed storage unit comprises a 3D animation of a human face that is rendered as a textured surface.

5. The apparatus of claim 1, wherein the high-speed storage unit is a Storage Area Network (SAN).

6. The apparatus of claim 1, wherein user interaction input is coupled to the one or more servers through the inbound routing network.

7. A computer-implemented method for streaming interactive video comprising:

executing a twitch video game or real-time application on one or more servers of a hosting service center to produce uncompressed video of 3D animation, the uncompressed video of 3D animation is produced, at least partially, using data streamed from a high-speed storage unit coupled to the one or more servers of the hosting service center, the high-speed storage unit is configured to load geometry for rendering objects of said 3D animation;

compressing, by a compression unit of the hosting service center, the uncompressed video of 3D animation, thereby producing compressed streaming interactive video;

transmitting the compressed streaming interactive video over a packetized network to a plurality of client devices associated with a corresponding plurality of users, the users being geographically remote to the hosting service center, a perspective view of the 3D animation being generated for at least one of said plurality of users that is an active user of the twitch video game or real-time application, and

changing, by the one or more servers, responsive to a spectator user control input received from a client device of the spectator user, the perspective view to a new perspective view that is generated by the one or more servers for the spectator user, the new perspective view being of the twitch video game or real-time application that is interactively controllable by the spectator user via the control input and is different than the perspective view of the active user, the control input being communicated back to the hosting service center via an inbound routing that feeds the control input to the one or more servers of the hosting service center, and said one or more servers controlling actions to compress and stream video frames of said twitch video game or real-time application to the spectator user for said new perspective view.

8. The computer-implemented method of claim 7, further comprising streaming audio received from the client devices.

9. The computer-implemented method of claim 8, wherein the audio is used for implementing a collaborative call among at least two of the users.

10. The computer-implemented method of claim 8, wherein the audio is used for implementing a collaborative call among at least one of the users and the spectator user, the collaborative call being processed during active use of the twitch video game or real-time application.

11. The computer-implemented method of claim 7, wherein the perspective view is transmitted to one or more of said client devices for display thereon, the perspective view being updated over time by each of said client devices for said users that are active players of the twitch video game or real-time application, and said new perspective view is updated over time by said spectator user.

12. The computer-implemented method of claim 7, wherein one or more additional perspective views are transmitted to one or more of said client devices for display thereon.