Hosting and broadcasting virtual events using streaming interactive video

• US 9,108,107 B2
• Filed: 12/05/2007
• Issued: 08/18/2015
• Est. Priority Date: 12/10/2002
• Status: Expired due to Term

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment America LLC (Sony Group Corp.)
• Inventors: Perlman, Stephen G., van der Laan, Roger
• Primary Examiner(s): SALCE, JASON P
• Application Number:
  US11/999,475
• Publication Number:
  US 20090118017A1
• Time in Patent Office: 2,813 Days
• Field of Search: 725/71, 725/133, 725/141, 725/153, 463/4, 463 40- 42
• US Class Current: 1/1
• CPC Class Codes: A63F 13/12 involving interaction between a plurality of game devices, e.g. transmisison or distribution systems protocols for networked virtual reality, networked games H04L29/06034
• CPC Class Codes: A63F 13/30 Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
• CPC Class Codes: A63F 13/335 using Internet
• CPC Class Codes: A63F 13/355 Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• CPC Class Codes: A63F 13/358 Adapting the game course according to the network or server load, e.g. for reducing latency due to different connection speeds between clients
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 2009/245 visual
• CPC Class Codes: A63F 2009/247 audible, e.g. using a loudspeaker
• CPC Class Codes: A63F 2009/2488 Remotely playable
• CPC Class Codes: A63F 2300/402 Communication between platforms, i.e. physical link to protocol
• CPC Class Codes: A63F 2300/407 Data transfer via internet
• CPC Class Codes: A63F 2300/552 for downloading to client devices, e.g. using OS version, hardware or software profile of the client device
• CPC Class Codes: A63F 2300/572 Communication between players during game play of non game information, e.g. e-mail, chat, file transfer, streaming of audio and streaming of video
• CPC Class Codes: A63F 2300/577 for watching a game played by other players
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: A63F 9/24 Electric games; Games using electronic circuits not otherwise provided for video games A63F13/00; computerized gaming systems G07F17/32
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 65/765 intermediate
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/131 : Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04N 19/107 : between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/132 : Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/146 : Data rate or code amount at the encoder output
• CPC Class Codes: H04N 19/169 : characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
• CPC Class Codes: H04N 19/172 : the region being a picture, frame or field
• CPC Class Codes: H04N 19/188 : the unit being a video data packet, e.g. a network abstraction layer [NAL] unit
• CPC Class Codes: H04N 19/436 : using parallelised computational arrangements
• CPC Class Codes: H04N 19/61 : in combination with predictive coding
• CPC Class Codes: H04N 21/2143 : located in a single building, e.g. hotel, hospital or museum arrangements specially adapted for plural spots in a confined site in broadcast systems H04H20/63; adaptations for transmission by electric cable for domestic distribution in television systems H04N7/106
• CPC Class Codes: H04N 21/233 : Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/2343 : involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/2381 : Adapting the multiplex stream to a specific network, e.g. an Internet Protocol [IP] network transmission of MPEG streams over ATM H04L12/5601
• CPC Class Codes: H04N 21/262 : Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists scheduling strategies for dispatcher in multiprogramming arrangements G06F9/4881; arrangements for scheduling broadcast services or broadcast-related services H04H60/06; flow control in packet networks H04L47/10; establishing a time schedule or organising the servicing of application requests H04L67/62
• CPC Class Codes: H04N 21/4307 : Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
• CPC Class Codes: H04N 21/43074 : of additional data with content streams on the same device, e.g. of EPG data or interactive icon with a TV program
• CPC Class Codes: H04N 21/439 : Processing of audio elementary streams
• CPC Class Codes: H04N 21/4781 : Games
• CPC Class Codes: H04N 21/6125 : involving transmission via Internet transmission by internet of broadcast information H04H60/82
• CPC Class Codes: H04N 21/6377 : directed to server one-way streaming services wherein the source is controlled by the destination H04L65/613
• CPC Class Codes: H04N 21/6405 : Multicasting data broadcast and multicast in packet switching networks H04L12/18
• CPC Class Codes: H04N 21/658 : Transmission by the client directed to the server
• CPC Class Codes: H04N 21/6587 : Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 7/106 : for domestic distribution
• CPC Class Codes: H04W 28/16 : Central resource management; Negotiation of resources or communication parameters, e.g. negotiating bandwidth or QoS [Quality of Service]
• CPC Class Codes: H04W 84/12 : WLAN [Wireless Local Area Networks]

Abstract

A method comprising broadcasting a live game tournament in the form of a multicasted streaming interactive video stream from a hosting service to a plurality of viewers over the Internet. Audio from an announcer is overlaid by the hosting service onto the multicasted video stream.

Claims

1. A method comprising:

running one or more twitch video games or real-time applications on servers of a hosting service center, the one or more twitch video games or real-time applications being played or used by one or more users on one or more client devices remote to the hosting service center via compressed streaming interactive video, the compressed streaming interactive video being compressed with a worst-case, round-trip latency of 90 ms, from a user control input to display of a response to the user control input on the client device of the user, over a transmission distance of up to 1500 miles;

broadcasting the one or more twitch video games or real-time applications as one or more streams of real-time compressed digital video from the hosting service center to an audience of viewers over the Internet.

2. The method of claim 1 further comprising selectively overlaying, by the hosting service center, audio from one or more announcers onto the one or more streams of real-time compressed digital video.

3. The method of claim 1 further comprising selectively delaying the one or more streams of real-time compressed digital video before broadcasting to the audience of viewers.

4. The method of claim 1 wherein the worst case latency is achieved in part by a frame computation process of the one or more of the servers responsive to the user control input, and a video compression process which produces the compressed streaming interactive video stream, that have a combined maximum latency of 22 milliseconds.

5. The method of claim 1 wherein one of the one or more twitch video games or real-time applications comprises a video game tournament.

6. The method of claim 1 wherein the one or more streams of real-time compressed digital video comprises one or more IP multicasts within the hosting service center.

7. The method of claim 1 further comprising sending, by the hosting service center, the one or more streams of real-time compressed digital video to a Content Delivery Network (CDN) for distribution to the audience of viewers over the Internet.

8. A method comprising:

running a twitch video game or real-time application at a hosting service center, the twitch video game or real-time application being played or used by a user on a client device remote to the hosting service center via compressed streaming interactive video, the hosting service center processing input received from the user wherein the compressed streaming interactive video is compressed with a worst-case, round-trip latency of 90 ms, from a user control input to display of a response to the user control input on the client device of the user, over a transmission distance of up to 1500 miles; and

broadcasting the twitch action video game or real-time application in one or more streams of real-time compressed digital video from the hosting service center to an audience of viewers over the Internet.

9. The method of claim 8 further comprising selectively delaying the one or more streams of real-time compressed digital video broadcast to the audience of viewers over the Internet.

10. The method of claim 8 further comprising selectively overlaying, by the hosting service center, audio from one or more announcers onto the one or more streams of real-time compressed digital video.

11. The method of claim 8 wherein the hosting service center comprises one or more servers that run the twitch video game or real-time application, wherein the user interacts with the twitch video game or real-time application via the compressed streaming interactive video encoded from uncompressed video generated by the one or more servers.

12. The method of claim 11 wherein the uncompressed video generated by the one or more servers is utilized by the hosting service center in one or more forms in addition to the compressed streaming interactive video.

13. The method of claim 12 wherein the one of the one or more forms comprises a compressed digital video stream generated without client feedback.

14. The method of claim 11 wherein the one or more streams of real-time compressed digital video adapts dynamically to characteristics of an Internet channel between the hosting service center and the audience of viewers.

15. The method of claim 8 wherein the one or more streams of real-time compressed digital video comprises one or more IP multicasts within the hosting service center.

16. The method of claim 8 further comprising sending, by the hosting service center, the one or more streams of real-time compressed digital video to a Content Delivery Network (CDN) for distribution to the audience of viewers.

17. A method for integrating video content with interactive features comprising:

hosting a twitch video game or real-time application from a hosting service center, each of one or more players or users playing the twitch video game or real-time application over the Internet via a client device remote to the hosting service center via compressed streaming interactive digital video compressed with a worst-case, round-trip latency of 90 ms, from a user control input to display of a response to the user control input on the client device of the user over a transmission distance of up to 1500 miles, the client device not running the twitch video game or real-time application;

generating, by the hosting service center, one or more streams of real-time compressed digital video of the twitch video game or application; and

broadcasting the one or more streams of real-time compressed digital video to an audience of viewers over the Internet.

18. The method of claim 17 further comprising:

selectively overlaying in real-time, by the hosting service center, audible speech from one or more announcers onto the one or more streams of real-time compressed digital video broadcast to the audience of viewers.

19. The method of claim 17 wherein the hosting service center comprises one or more servers that run the twitch video game or real-time application, wherein each remote player interacts with the twitch game or real-time application via the compressed streaming interactive digital video encoded from uncompressed video from the one or more servers.

20. The method of claim 19 wherein the uncompressed video generated by the one or more servers is utilized by the hosting service center in one or more forms in addition to the compressed streaming interactive digital video.

21. The method of claim 20 wherein one of the one or more forms comprises a compressed digital video stream generated without client feedback.

22. The method of claim 18 further comprising selectively delaying the one or more streams of real-time compressed digital video before broadcasting to the audience of viewers.

23. The method of claim 17 wherein the one or more streams of real-time compressed digital video comprises one or more IP multicasts within the hosting service center.

24. The method of claim 17 further comprising sending, by the hosting service center, the one or more streams of real-time compressed digital video to a Content Delivery Network (CDN) for mass distribution to an audience of viewers.

25. The method of claim 18 wherein the one or more streams of real-time compressed digital video adapts dynamically to characteristics of an Internet channel between the hosting service center and the audience of viewers.

26. The method of claim 25 wherein the characteristics of the Internet channel between the hosting service center and each of the one or more players is determined by the hosting service based on feedback from the client device.

27. The method of claim 22 further comprising selectively delaying the audible speech overlaid onto the one or more streams of real-time compressed digital video prior to the broadcast so as to remain in synchronization with video action of the selectively delayed one or more streams of real-time compressed digital video.

28. A method for multi-player video gaming comprising:

executing a twitch video game on one or more servers of a hosting service center, the twitch video game being played via compressed streaming interactive video over the Internet by a plurality of players via client devices located remote to the hosting service center, the compressed streaming interactive video being compressed with a worst-case, round-trip latency of 90 ms, from a player control input to display of a response to the player control input on the client device of the player over a transmission distance of up to 1500 mile;

generating, by the hosting service center, one or more streams of real-time compressed digital video of the twitch video game; and

broadcasting the twitch video game live to an audience of viewers as one or more streams of real-time compressed digital video over the Internet.

29. The method of claim 28 wherein the worst case latency is achieved in part by a frame computation process of the one or more of the servers responsive to the user control input, and a video compression process which produces the compressed streaming interactive video stream, that have a combined maximum latency of 22 milliseconds.

30. The method of claim 28 wherein executable program code associated with the twitch video game is stored remote to the client devices.

31. The method of claim 28 wherein at least a portion of executable program code associated with the twitch video game is downloaded to, and run locally on, one or more of the client devices.