System and method for compressing streaming interactive video

• US 9,700,790 B2
• Filed: 09/12/2014
• Issued: 07/11/2017
• Est. Priority Date: 12/10/2002
• Status: Expired due to Term

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment America LLC (Sony Group Corp.)
• Inventors: Perlman, Stephen G., van der Laan, Roger
• Primary Examiner(s): VU, NGOC K
• Application Number:
  US14/484,641
• Publication Number:
  US 20140379866A1
• Time in Patent Office: 1,033 Days
• Field of Search: 375240, 37524001
• CPC Class Codes: A63F 13/12 involving interaction between a plurality of game devices, e.g. transmisison or distribution systems protocols for networked virtual reality, networked games H04L29/06034
• CPC Class Codes: A63F 13/30 Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
• CPC Class Codes: A63F 13/335 using Internet
• CPC Class Codes: A63F 13/355 Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• CPC Class Codes: A63F 13/358 Adapting the game course according to the network or server load, e.g. for reducing latency due to different connection speeds between clients
• CPC Class Codes: A63F 2300/402 Communication between platforms, i.e. physical link to protocol
• CPC Class Codes: A63F 2300/552 for downloading to client devices, e.g. using OS version, hardware or software profile of the client device
• CPC Class Codes: H04L 47/25 with rate being modified by the source upon detecting a change of network conditions
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/765 intermediate
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/169 characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/436 using parallelised computational arrangements
• CPC Class Codes: H04N 19/61 : in combination with predictive coding
• CPC Class Codes: H04N 19/85 : using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 21/226 : Characteristics of the server or Internal components of the server
• CPC Class Codes: H04N 21/2343 : involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/2381 : Adapting the multiplex stream to a specific network, e.g. an Internet Protocol [IP] network transmission of MPEG streams over ATM H04L12/5601
• CPC Class Codes: H04N 21/4781 : Games
• CPC Class Codes: H04N 21/6125 : involving transmission via Internet transmission by internet of broadcast information H04H60/82
• CPC Class Codes: H04N 21/6587 : Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 7/106 : for domestic distribution
• CPC Class Codes: H04W 28/06 : Optimizing the usage of the radio link, e.g. header compression, information sizing , discarding information system modifying transmission characteristic according to link quality by modifying frame length H04L1/0007; dynamic adaptation of the packet size for flow control or congestion control H04L47/365
• CPC Class Codes: H04W 4/00 : Services specially adapted for wireless communication networks; Facilities therefor

Abstract

A server center for hosting low-latency streaming interactive audio/video (A/V) includes a plurality of servers that run one or more applications and an inbound routing network that receives packet streams from users via a first network interface and routes the packets to one or more of the servers. The packet streams include user control input. One or more of the servers are operable to compute A/V data responsive to the user control input. A compression unit is coupled to receive the A/V data from the one or more of the servers and output compressed A/V data therefrom. An output routing network that routes the compressed A/V data to each of the users over a corresponding communication channel via a second interface, the compression unit is operable to modify a compression rate responsive to current characteristics of the corresponding communication channel for each user so as to optimize performance of the one or more applications.

Claims

1. A computer-implemented method comprising:

running a twitch video game or real-time application on a server unit, uncompressed streaming interactive video being produced therefrom; and

receiving the uncompressed streaming interactive video at a compression unit, the compression unit outputting compressed streaming interactive video therefrom, the server unit and the compression unit being located at a premises; and

transmitting the compressed streaming interactive video over a packetized network from the premises to one or more client devices associated with one or more users, each client device being located geographically remote to the premises, each client device not running the twitch video game or real-time application locally;

wherein the compressed streaming interactive video is compressed with a worst-case, round-trip latency of 90 milliseconds, from the user control input to display of a response to the user control input on a client device of a user, over a transmission distance of up to 1500 miles.

2. The computer-implemented method of claim 1 wherein the twitch video game or real-time application includes a television or movie application.

3. The computer-implemented method of claim 1 further comprising receiving, by the server unit, video control commands from at least one of the one or more client devices, the video control commands including at least one of pause, fast forward, rewind commands.

4. The computer-implemented method of claim 1 further comprising receiving, by the server unit, live broadcasts through a network connection, a satellite connection, a media player, or a cable television connection.

5. The computer-implemented method of claim 1 further comprising storing recorded media content in a storage unit.

6. The computer-implemented method of claim 5 wherein the recorded media content includes movies and/or programs.

7. The computer-implemented method of claim 5 wherein the recorded media content includes personal data of the users.

8. The computer-implemented method of claim 7 wherein the personal data includes photographs.

9. The computer-implemented method of claim 5 wherein the recorded media content includes video games.

10. The computer-implemented method of claim 1 further comprising an providing a connection to the packetized network through an interface.

11. The computer-implemented method of claim 1 wherein the packetized network is the Internet.

12. A non-transitory computer-readable storage medium encoded with computer instructions, which, when executed, operable to:

run a twitch video game or real-time application on a server unit, uncompressed streaming interactive video being produced therefrom; and

receive the uncompressed streaming interactive video at a compression unit, the compression unit outputting compressed streaming interactive video therefrom, the server unit and the compression unit being located at a premises; and

transmit the compressed streaming interactive video over a packetized network from the premises to one or more client devices associated with one or more users, each client device being located geographically remote to the premises, each client device not running the twitch video game or real-time application locally;

wherein the compressed streaming interactive video is compressed with a worst-case, round-trip latency of 90 milliseconds, from the user control input to display of a response to the user control input on a client device of a user, over a transmission distance of up to 1500 miles.

13. The non-transitory computer-readable storage medium of claim 12 wherein execution of the computer instructions is further operable to receive video control commands from at least one of the client devices, the video control commands including at least one of pause, fast forward, rewind commands.

14. The non-transitory computer-readable storage medium of claim 12 wherein execution of the computer instructions is further operable to receive live broadcasts through a network connection, a satellite connection, a media player, or a cable television connection.

15. The non-transitory computer-readable storage medium of claim 12 wherein execution of the computer instructions is further operable to store recorded media content in a storage unit.

16. The non-transitory computer-readable storage medium of claim 15 wherein the recorded media content includes movies and/or programs.

17. The non-transitory computer-readable storage medium of claim 15 wherein the recorded media content includes personal data of the users.

18. The non-transitory computer-readable storage medium of claim 15 wherein the personal data includes photographs.

19. The non-transitory computer-readable storage medium of claim 12 wherein the packetized network comprises the Internet.