System and Method for Utilizig Forward Error Correction With Video Compression

• US 20160007045A1
• Filed: 07/06/2015
• Published: 01/07/2016
• Est. Priority Date: 12/05/2007
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment America LLC (Sony Group Corp.)
• Inventors: Perlman, Steve, van der Laan, Roger, Cotter, Timothy, Furman, Scott, McCool, Robert, Buckley, Ian
• Granted Patent: US 9,878,241 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A63F 13/335 using Internet
• CPC Class Codes: A63F 13/355 Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• CPC Class Codes: A63F 13/358 Adapting the game course according to the network or server load, e.g. for reducing latency due to different connection speeds between clients
• CPC Class Codes: A63F 13/77 involving data related to game devices or game servers, e.g. configuration data, software version or amount of memory
• CPC Class Codes: A63F 2300/407 Data transfer via internet
• CPC Class Codes: A63F 2300/535 for monitoring, e.g. of user parameters, terminal parameters, application parameters, network parameters
• CPC Class Codes: A63F 2300/538 for performing operations on behalf of the game client, e.g. rendering
• CPC Class Codes: A63F 2300/552 for downloading to client devices, e.g. using OS version, hardware or software profile of the client device
• CPC Class Codes: A63F 2300/577 for watching a game played by other players
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/765 intermediate
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/146 Data rate or code amount at the encoder output
• CPC Class Codes: H04N 19/164 Feedback from the receiver or from the transmission channel
• CPC Class Codes: H04N 19/169 characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/188 : the unit being a video data packet, e.g. a network abstraction layer [NAL] unit
• CPC Class Codes: H04N 19/436 : using parallelised computational arrangements
• CPC Class Codes: H04N 19/61 : in combination with predictive coding
• CPC Class Codes: H04N 19/65 : using error resilience
• CPC Class Codes: H04N 21/2343 : involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/23439 : for generating different versions
• CPC Class Codes: H04N 21/2381 : Adapting the multiplex stream to a specific network, e.g. an Internet Protocol [IP] network transmission of MPEG streams over ATM H04L12/5601
• CPC Class Codes: H04N 21/266 : Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
• CPC Class Codes: H04N 21/6377 : directed to server one-way streaming services wherein the source is controlled by the destination H04L65/613
• CPC Class Codes: H04N 21/658 : Transmission by the client directed to the server
• CPC Class Codes: H04N 21/6583 : Acknowledgement
• CPC Class Codes: H04N 21/6587 : Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 7/106 : for domestic distribution

Abstract

A computer-implemented system and method are described for performing video compression. For example, a method according to one embodiment comprises: encoding a plurality of video frames or portions thereof according to a first encoding format; transmitting the plurality of encoded video frames or portions to a client device; receiving feedback information from the client device, the feedback information usable to determine whether data contained in the video frames or portions has not been successfully received and/or decoded; in response to detecting that one or more video frames or portions thereof have not been successfully received and/or decoded, then either encoding a new video frame or portion thereof according to a second encoding format or encoding the new video frame to be dependent on the last video frame or portion thereof known to have been successfully received and/or decoded.

Claims

1. A computer-implemented method for performing video compression comprising:

executing a video game at a server;

encoding a plurality of video frames according to a first encoding format;

transmitting the plurality of encoded video frames to a client device;

receiving feedback information from the client device, the feedback information usable to determine whether data contained in the video frames has not been successfully received and/or decoded by the client

in response to detecting that one or more video frames have not been successfully received and/or decoded, then either encoding a new video frame according to a second encoding format or encoding the new video frame to be dependent on the last video frame known to have been successfully received and/or decoded;

if no forward error correction (FEC) coding is used to transmit the plurality of encoded video frames, then encoding a new video frame using FEC coding; and

transmitting the new video frame to the client device.

2. A computer-implemented method of claim 1, wherein the frames comprise P-frames when encoded according to the first encoding format and wherein the frames comprise I-frames when encoded according to the second encoding format.

4. A computer-implemented method of claim 2, wherein the frames comprise P-frames when encoded according to the first encoding format and wherein the frames comprise I-frames when encoded according to the second encoding format.

3. A computer-implemented method for performing video compression comprising:

executing a video game at a server;

encoding a plurality of video frames according to a first encoding format;

transmitting the plurality of encoded video frames to a client device;

receiving feedback information from the client device, the feedback information usable to determine whether data contained in the video frames has not been successfully received and/or decoded by the client

in response to detecting that one or more video frames have not been successfully received and/or decoded, then either encoding a new video frame according to a second encoding format or encoding the new video frame to be dependent on the last video frame known to have been successfully received and/or decoded;

if a first FEC coding is used to transmit the plurality of encoded video frames, then encoding a new video frame using a second FEC coding, the second FEC coding being a relatively stronger FEC coding than the first FEC coding; and

transmitting the new video frame to the client device.