System and Method for Multi-Stream Video Compression Using Multiple Encoding Formats

• US 20100166068A1
• Filed: 08/07/2009
• Published: 07/01/2010
• Est. Priority Date: 12/10/2002
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment America LLC (Sony Group Corp.)
• Inventors: Perlman, Stephen G., Cotter, Timothy, McCool, Robert, Buckley, Ian, Laan, Roger Van Der, Furman, Scott
• Granted Patent: US 8,964,830 B2
• Time in Patent Office: Days
• US Class Current: 375/240.120
• CPC Class Codes: A63F 13/335 using Internet
• CPC Class Codes: A63F 13/355 Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• CPC Class Codes: A63F 13/358 Adapting the game course according to the network or server load, e.g. for reducing latency due to different connection speeds between clients
• CPC Class Codes: A63F 13/497 Partially or entirely replaying previous game actions
• CPC Class Codes: A63F 13/86 Watching games played by other players
• CPC Class Codes: A63F 2300/407 Data transfer via internet
• CPC Class Codes: A63F 2300/534 for network load management, e.g. bandwidth optimization, latency reduction
• CPC Class Codes: A63F 2300/538 for performing operations on behalf of the game client, e.g. rendering
• CPC Class Codes: A63F 2300/5593 involving scheduling aspects
• CPC Class Codes: A63F 2300/577 for watching a game played by other players
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/765 intermediate
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/146 Data rate or code amount at the encoder output
• CPC Class Codes: H04N 19/169 characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/188 : the unit being a video data packet, e.g. a network abstraction layer [NAL] unit
• CPC Class Codes: H04N 19/436 : using parallelised computational arrangements
• CPC Class Codes: H04N 19/61 : in combination with predictive coding
• CPC Class Codes: H04N 21/231 : Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
• CPC Class Codes: H04N 21/2343 : involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/23439 : for generating different versions
• CPC Class Codes: H04N 21/2381 : Adapting the multiplex stream to a specific network, e.g. an Internet Protocol [IP] network transmission of MPEG streams over ATM H04L12/5601
• CPC Class Codes: H04N 21/4781 : Games
• CPC Class Codes: H04N 21/6377 : directed to server one-way streaming services wherein the source is controlled by the destination H04L65/613
• CPC Class Codes: H04N 21/658 : Transmission by the client directed to the server
• CPC Class Codes: H04N 21/6587 : Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 7/106 : for domestic distribution

Abstract

A computer-implemented system and method are described for performing video compression. For example, a method according to one embodiment comprises: encoding a plurality of video frames or portions thereof according to a first encoding format, the first encoding format being optimized for transmission to a client device over a current communication channel; transmitting the plurality of encoded video frames or portions to the client device over the communication channel; concurrently encoding the first plurality of video frames according to a second encoding format, the second encoding format having a relatively higher-quality compressed video and/or a lower compression ratio than the first encoding format; storing the first plurality of video frames encoded in the second encoding format on a storage device; and making the first plurality of video frames encoded in the second encoding format available to the client device and other client devices for playback of the video stream.

Claims

1. A computer-implemented method for performing video compression comprising:

encoding a plurality of video frames or portions thereof according to a first encoding format, the first encoding format being optimized for transmission to a client device over a current communication channel;

transmitting the plurality of encoded video frames or portions to the client device over the communication channel;

concurrently encoding the first plurality of video frames according to a second encoding format, the second encoding format having a relatively higher-quality compressed video and/or a lower compression ratio than the first encoding format;

storing the first plurality of video frames encoded in the second encoding format on a storage device; and

making the first plurality of video frames encoded in the second encoding format available to the client device and other client devices for playback of the video stream.

2. The method as in claim 1 wherein the first encoding format comprises frames which are dependent on other frames and the second encoding format comprises some frames which are not dependent on other frames.

3. The method as in claim 2 wherein the second encoding format includes I-Frames and wherein the first encoding format does not include I-Frames.

4. The method as in claim 3 wherein the first encoding format comprises only P-Frames.

5. The method as in claim 1 wherein the first plurality of video frames are frames of a video game, generated as a user of the client device plays an online video game and wherein the first encoding format is selected to reduce latency as the frames compressed according to the first encoding format are transmitted to the client device over the current communication channel.

6. The method as in claim 5 wherein the second encoding format is selected to archive the video frames of the online video game, the video frames to be made available for subsequent playback from the archive.

7. A computer system having a memory for storing program code and a processor for processing the program code to perform the operations of:

encoding a plurality of video frames or portions thereof according to a first encoding format, the first encoding format being optimized for transmission to a client device over a current communication channel;

transmitting the plurality of encoded video frames or portions to the client device over the communication channel;

concurrently encoding the first plurality of video frames according to a second encoding format, the second encoding format having a higher-quality compressed video and/or a relatively lower compression ratio than the first encoding format;

storing the first plurality of video frames encoded in the second encoding format on a storage device; and

making the first plurality of video frames encoded in the second encoding format available to the client device and other client devices for playback of the video stream.

8. The computer system as in claim 7 wherein the first encoding format comprises frames which are dependent on other frames and the second encoding format comprises some frames which are not dependent on other frames.

9. The computer system as in claim 8 wherein the second encoding format includes I-Frames and wherein the first encoding format does not include I-Frames.

10. The computer system as in claim 9 wherein the first encoding format comprises only P-Frames.

11. The computer system as in claim 7 wherein the first plurality of video frames are frames of a video game, generated as a user of the client device plays an online video game and wherein the first encoding format is selected to reduce latency as the frames compressed according to the first encoding format are transmitted to the client device over the current communication channel.

12. The computer system as in claim 11 wherein the second encoding format is selected to archive the video frames of the online video game, the video frames to be made available for subsequent playback from the archive.

13. A machine-readable medium having program code stored thereon which, when executed by a machine, causes the machine to perform the operations of:

encoding a plurality of video frames or portions thereof according to a first encoding format, the first encoding format being optimized for transmission to a client device over a current communication channel;

transmitting the plurality of encoded video frames or portions to the client device over the communication channel;

concurrently encoding the first plurality of video frames according to a second encoding format, the second encoding format having a higher-quality compressed video and/or a relatively lower compression ratio than the first encoding format;

storing the first plurality of video frames encoded in the second encoding format on a storage device; and

making the first plurality of video frames encoded in the second encoding format available to the client device and other client devices for playback of the video stream.

14. The machine-readable medium as in claim 13 wherein the first encoding format comprises frames which are dependent on other frames and the second encoding format comprises some frames which are not dependent on other frames.

15. The machine-readable medium as in claim 14 wherein the second encoding format includes I-Frames and wherein the first encoding format does not include I-Frames.

16. The machine-readable medium as in claim 15 wherein the first encoding format comprises only P-Frames.

17. The machine-readable medium as in claim 13 wherein the first plurality of video frames are frames of a video game, generated as a user of the client device plays an online video game and wherein the first encoding format is selected to reduce latency as the frames compressed according to the first encoding format are transmitted to the client device over the current communication channel.

18. The machine-readable medium as in claim 17 wherein the second encoding format is selected to archive the video frames of the online video game, the video frames to be made available for subsequent playback from the archive.