System and Method for Video Compression Using Feedback Including Data Related to the Successful Receipt of Video Content

• US 20100166066A1
• Filed: 08/07/2009
• Published: 07/01/2010
• Est. Priority Date: 12/10/2002
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: OL2 Incorporated (Sony Group Corp.)
• Inventors: van der Laan, Roger, Cotter, Timothy, McCool, Robert, Buckley, Ian, Perlman, Steve, Furman, Scott
• Granted Patent: US 8,526,490 B2
• Time in Patent Office: Days
• US Class Current: 375/240.70
• CPC Class Codes: A63F 13/12 involving interaction between a plurality of game devices, e.g. transmisison or distribution systems protocols for networked virtual reality, networked games H04L29/06034
• CPC Class Codes: A63F 13/30 Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
• CPC Class Codes: A63F 13/355 Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• CPC Class Codes: A63F 13/358 Adapting the game course according to the network or server load, e.g. for reducing latency due to different connection speeds between clients
• CPC Class Codes: A63F 2300/402 Communication between platforms, i.e. physical link to protocol
• CPC Class Codes: A63F 2300/407 Data transfer via internet
• CPC Class Codes: A63F 2300/535 for monitoring, e.g. of user parameters, terminal parameters, application parameters, network parameters
• CPC Class Codes: A63F 2300/538 for performing operations on behalf of the game client, e.g. rendering
• CPC Class Codes: A63F 2300/552 for downloading to client devices, e.g. using OS version, hardware or software profile of the client device
• CPC Class Codes: A63F 2300/572 Communication between players during game play of non game information, e.g. e-mail, chat, file transfer, streaming of audio and streaming of video
• CPC Class Codes: A63F 2300/577 for watching a game played by other players
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/765 intermediate
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/146 Data rate or code amount at the encoder output
• CPC Class Codes: H04N 19/169 characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
• CPC Class Codes: H04N 19/172 : the region being a picture, frame or field
• CPC Class Codes: H04N 19/188 : the unit being a video data packet, e.g. a network abstraction layer [NAL] unit
• CPC Class Codes: H04N 19/436 : using parallelised computational arrangements
• CPC Class Codes: H04N 19/61 : in combination with predictive coding
• CPC Class Codes: H04N 21/2343 : involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/23439 : for generating different versions
• CPC Class Codes: H04N 21/2381 : Adapting the multiplex stream to a specific network, e.g. an Internet Protocol [IP] network transmission of MPEG streams over ATM H04L12/5601
• CPC Class Codes: H04N 21/266 : Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
• CPC Class Codes: H04N 21/6377 : directed to server one-way streaming services wherein the source is controlled by the destination H04L65/613
• CPC Class Codes: H04N 21/658 : Transmission by the client directed to the server
• CPC Class Codes: H04N 21/6583 : Acknowledgement
• CPC Class Codes: H04N 21/6587 : Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 7/106 : for domestic distribution

Abstract

A computer-implemented system and method for performing video compression are described. For example, a method according to one embodiment comprises: encoding a first plurality of video frames or portions thereof, wherein each encoded video frame or portion thereof is dependent on a previously-encoded video frame or portion thereof, respectively; transmitting the first plurality of encoded video frames or portions to a client device; receiving feedback information from the client device, the feedback information usable to determine whether data contained in the video frames or portions has not been successfully received and/or decoded; in response to detecting that a video frame or portion thereof has not been successfully received and/or decoded, encoding a current video frame or portion thereof to be dependent on a previously-encoded video frame or portion thereof known to have been successfully received and/or decoded on the client device; and transmitting the current video frame or portion thereof to the client device.

Claims

1. A computer-implemented method for performing video compression comprising:

encoding a first plurality of video frames or portions thereof, wherein each encoded video frame or portion thereof is dependent on a previously-encoded video frame or portion thereof, respectively;

transmitting the first plurality of encoded video frames or portions to a client device;

receiving feedback information from the client device, the feedback information usable to determine whether data contained in the video frames or portions has not been successfully received and/or decoded;

in response to detecting that a video frame or portion thereof has not been successfully received and/or decoded, encoding a current video frame or portion thereof to be dependent on a previously-encoded video frame or portion thereof known to have been successfully received and/or decoded on the client device; and

transmitting the current video frame or portion thereof to the client device.

2. The method as in claim 1 wherein the previously-encoded video frame or portion thereof known to have been successfully received and/or decoded comprises the last known video frame or portion thereof known to have been successfully received and/or decoded.

3. The method as in claim 1 wherein the video frames or portions thereof comprise P-frames or P-tiles, respectively.

4. The method as in claim 1 wherein the feedback information comprises an indication that the frames or portions thereof have been successfully received and/or decoded at the client device.

5. The method as in claim 1 wherein the feedback information comprises an indication that the frames or portions thereof have not been successfully received and/or decoded at the client device.

6. The method as in claim 1 wherein encoding a current video frame or portion thereof to be dependent on a previously-encoded video frame or portion thereof known to have been successfully received and/or decoded on the client device further comprises:

retrieving a prior state from a memory, the prior state comprising a state of an encoder after encoding the previously-encoded video frame or portion.

7. The method as in claim 6 further comprising:

decoding the current video frame or portion thereof, wherein decoding comprises retrieving a prior state from a memory, the prior state comprising a state of a decoder after decoding the previously-encoded video frame or portion.

8. The method as in claim 1 further comprising:

decoding the encoded video frames or portion thereof at the client device; and

displaying images associated with each of the video frames or portions thereof on a display on the client device.

9. A system comprising a memory for storing program code and a processor for processing the program code to perform the operations of:

encoding a first plurality of video frames or portions thereof, wherein each encoded video frame or portion thereof is dependent on a previously-encoded video frame or portion thereof, respectively;

transmitting the first plurality of encoded video frames or portions to a client device;

receiving feedback information from the client device, the feedback information usable to determine whether data contained in the video frames or portions has not been successfully received and/or decoded;

in response to detecting that a video frame or portion thereof has not been successfully received and/or decoded, encoding a current video frame or portion thereof to be dependent on a previously-encoded video frame or portion thereof known to have been successfully received and/or decoded on the client device; and

transmitting the current video frame or portion thereof to the client device.

10. The system as in claim 9 wherein the previously-encoded video frame or portion thereof known to have been successfully received and/or decoded comprises the last known video frame or portion thereof known to have been successfully received and/or decoded.

11. The system as in claim 9 wherein the video frames or portions thereof comprise P-frames or P-tiles, respectively.

12. The system as in claim 9 wherein the feedback information comprises an indication that the frames or portions thereof have been successfully received and/or decoded at the client device.

13. The system as in claim 9 wherein the feedback information comprises an indication that the frames or portions thereof have not been successfully received and/or decoded at the client device.

14. The system as in claim 9 wherein encoding a current video frame or portion thereof to be dependent on a previously-encoded video frame or portion thereof known to have been successfully received and/or decoded on the client device further comprises:

retrieving a prior state from a memory, the prior state comprising a state of an encoder after encoding the previously-encoded video frame or portion.

15. The system as in claim 14 comprising additional program code to cause the processor to perform the operations of:

decoding the current video frame or portion thereof, wherein decoding comprises retrieving a prior state from a memory, the prior state comprising a state of a decoder after decoding the previously-encoded video frame or portion.

16. The system as in claim 9 comprising additional program code to cause the processor to perform the operations of:

decoding the encoded video frames or portion thereof at the client device; and

displaying images associated with each of the video frames or portions thereof on a display on the client device.

17. A machine-readable medium having program code stored thereon which, when executed by a machine, causes the machine to perform the operations of:

encoding a first plurality of video frames or portions thereof, wherein each encoded video frame or portion thereof is dependent on a previously-encoded video frame or portion thereof, respectively;

transmitting the first plurality of encoded video frames or portions to a client device;

receiving feedback information from the client device, the feedback information usable to determine whether data contained in the video frames or portions has not been successfully received and/or decoded;

in response to detecting that a video frame or portion thereof has not been successfully received and/or decoded, encoding a current video frame or portion thereof to be dependent on a previously-encoded video frame or portion thereof known to have been successfully received and/or decoded on the client device; and

transmitting the current video frame or portion thereof to the client device.

18. The machine-readable medium as in claim 17 wherein the previously- encoded video frame or portion thereof known to have been successfully received and/or decoded comprises the last known video frame or portion thereof known to have been successfully received and/or decoded.

19. The machine-readable medium as in claim 17 wherein the video frames or portions thereof comprise P-frames or P-tiles, respectively.

20. The machine-readable medium as in claim 17 wherein the feedback information comprises an indication that the frames or portions thereof have been successfully received and/or decoded at the client device.

21. The machine-readable medium as in claim 17 wherein the feedback information comprises an indication that the frames or portions thereof have not been successfully received and/or decoded at the client device.

22. The machine-readable medium as in claim 17 wherein encoding a current video frame or portion thereof to be dependent on a previously-encoded video frame or portion thereof known to have been successfully received and/or decoded on the client device further comprises:

retrieving a prior state from a memory, the prior state comprising a state of an encoder after encoding the previously-encoded video frame or portion.

23. The machine-readable medium as in claim 22 comprising additional program code to cause the machine to perform the operations of:

decoding the current video frame or portion thereof, wherein decoding comprises retrieving a prior state from a memory, the prior state comprising a state of a decoder after decoding the previously-encoded video frame or portion.

24. The machine-readable medium as in claim 17 comprising additional program code to cause the processor to perform the operations of:

decoding the encoded video frames or portion thereof at the client device; and

displaying images associated with each of the video frames or portions thereof on a display on the client device.