Use of frame caching to improve packet loss recovery
First Claim
Patent Images
1. In a computing system that implements a video decoder, a method of decoding plural frames of a video sequence, the method comprising:
- decoding a first frame of the plural frames using encoded data for the first frame, wherein the encoded data for the first frame includes a frame header for the first frame;
storing, for use as a first reference frame, the first frame in a first portion of a decoder-side frame cache that corresponds to a first portion of an encoder-side frame cache at a video encoder, wherein the frame header for the first frame includes information indicating that the first frame is to be stored in the first portion of the decoder-side frame cache;
decoding a second frame of the plural frames using encoded data for the second frame, the second frame following the first frame in the video sequence;
storing, for use as a second reference frame, the second frame in a second portion of the decoder-side frame cache different than the first portion of the decoder-side frame cache;
detecting loss of encoded data for one of the plural frames;
in response to the detecting loss of encoded data, transmitting a loss detection signal in order to cause the video encoder to generate a new frame relative to the first reference frame stored in the first portion of the encoder-side frame cache at the video encoder, the new frame including different data than the one of the plural frames for which loss of encoded data is detected; and
decoding the new frame using encoded data for the new frame and using the first reference frame stored in the first portion of the decoder-side frame cache, the encoded data for the new frame having been generated by the video encoder, in response to the loss detection signal, during encoding of the new frame relative to the first reference frame stored in the first portion of the encoder-side frame cache at the video encoder.
1 Assignment
0 Petitions
Accused Products
Abstract
Various new and non-obvious apparatus and methods for using frame caching to improve packet loss recovery are disclosed. One of the disclosed embodiments is a method for using periodical and synchronized frame caching within an encoder and its corresponding decoder. When the decoder discovers packet loss, it informs the encoder which then generates a frame based on one of the shared frames stored at both the encoder and the decoder. When the decoder receives this generated frame it can decode it using its locally cached frame.
-
Citations
54 Claims
-
1. In a computing system that implements a video decoder, a method of decoding plural frames of a video sequence, the method comprising:
-
decoding a first frame of the plural frames using encoded data for the first frame, wherein the encoded data for the first frame includes a frame header for the first frame; storing, for use as a first reference frame, the first frame in a first portion of a decoder-side frame cache that corresponds to a first portion of an encoder-side frame cache at a video encoder, wherein the frame header for the first frame includes information indicating that the first frame is to be stored in the first portion of the decoder-side frame cache; decoding a second frame of the plural frames using encoded data for the second frame, the second frame following the first frame in the video sequence; storing, for use as a second reference frame, the second frame in a second portion of the decoder-side frame cache different than the first portion of the decoder-side frame cache; detecting loss of encoded data for one of the plural frames; in response to the detecting loss of encoded data, transmitting a loss detection signal in order to cause the video encoder to generate a new frame relative to the first reference frame stored in the first portion of the encoder-side frame cache at the video encoder, the new frame including different data than the one of the plural frames for which loss of encoded data is detected; and decoding the new frame using encoded data for the new frame and using the first reference frame stored in the first portion of the decoder-side frame cache, the encoded data for the new frame having been generated by the video encoder, in response to the loss detection signal, during encoding of the new frame relative to the first reference frame stored in the first portion of the encoder-side frame cache at the video encoder. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11)
-
-
12. In a computing system that implements a video encoder, a method of encoding plural frames of a video sequence, the method comprising:
-
encoding a first frame of the plural frames to produce encoded data for the first frame, wherein the encoded data for the first frame includes a frame header for the first frame; storing, for use as a first reference frame, the first frame in a first portion of an encoder-side frame cache that corresponds to a first portion of a decoder-side frame cache at a video decoder, wherein the frame header for the first frame includes information indicating that the first frame is to be stored in the first portion of the decoder-side frame cache; encoding a second frame of the plural frames to produce encoded data for the second frame, the second frame following the first frame in the video sequence; storing, for use as a second reference frame, the second frame in a second portion of the encoder-side frame cache different than the first portion of the encoder-side frame cache; receiving a loss detection signal that indicates loss of encoded data for one of the plural frames; and in response to the loss detection signal, encoding a new frame relative to the first reference frame stored in the first portion of the encoder-side frame cache to produce encoded data for the new frame, the new frame including different data than the one of the plural frames for which the loss detection signal indicates loss of encoded data, the new frame being decodable by the video decoder using the encoded data for the new frame and the first reference frame stored in the first portion of the decoder-side frame cache. - View Dependent Claims (13, 14, 15, 16, 17, 18, 19, 20, 21)
-
-
22. One or more computer-readable storage devices or memory having stored thereon computer-executable instructions for causing a computer system programmed thereby to perform operations for decoding plural frames of a video sequence in a video decoder, the operations comprising:
-
decoding a first frame of the plural frames using encoded data for the first frame, wherein the encoded data for the first frame includes a frame header for the first frame; storing, for use as a first reference frame, the first frame in a first portion of a decoder-side frame cache that corresponds to a first portion of an encoder-side frame cache at a video encoder, wherein the frame header for the first frame includes information indicating that the first frame is to be stored in the first portion of the decoder-side frame cache; decoding a second frame of the plural frames using encoded data for the second frame, the second frame following the first frame in the video sequence; storing, for use as a second reference frame, the second frame in a second portion of the decoder-side frame cache different than the first portion of the decoder-side frame cache; detecting loss of encoded data for one of the plural frames; in response to the detecting loss of encoded data, transmitting a loss detection signal in order to cause the video encoder to generate a new frame relative to the first reference frame stored in the first portion of the encoder-side frame cache at the video encoder; and decoding the new frame using encoded data for the new frame and using the first reference frame stored in the first portion of the decoder-side frame cache, the encoded data for the new frame including different data than the one of the plural frames for which loss is detected, the encoded data for the new frame having been generated by the video encoder, in response to the loss detection signal, during encoding of the new frame relative to the first reference frame stored in the first portion of the encoder-side frame cache at the video encoder. - View Dependent Claims (29, 30, 31, 40)
-
-
23. A computer system that implements a video decoder, the computer system comprising a processor and memory, wherein the video decoder includes:
-
a jitter buffer for storing encoded data for plural frames of a video sequence; a module configured for decoding the plural frames; a decoder-side frame cache, the decoder-side frame cache comprising; a first portion of the decoder-side frame cache for storing a first reference frame, wherein the first portion of the decoder-side frame cache corresponds to a first portion of an encoder-side frame cache at a video encoder, and wherein the first reference frame stored in the first portion of the decoder-side frame cache is updated according to information in frame headers for the plural frames that indicates which of the plural frames are to be cached in the first portion of the decoder-side frame cache after the decoding, and a second portion of the decoder-side frame cache for storing a second reference frame, wherein the second portion of the decoder-side frame cache is different than the first portion of the decoder-side frame cache, and wherein the second reference frame stored in the second portion of the decoder-side frame cache is updated to store a current frame of the plural frames after the decoding, for use in decoding a next frame of the plural frames; a module configured for detecting loss in the encoded data that affects decoding of a given frame of the plural frames, the given frame having been encoded relative to the second reference frame; and a module configured for transmitting a loss detection signal; the video decoder being configured to decode a new frame using the first reference frame in the first portion of the decoder-side frame cache, the new frame including different data than the encoded data for which loss is detected, the new frame having been encoded by the video encoder, in response to the loss detection signal, relative to the first reference frame stored in the first portion of the encoder-side frame cache at the video encoder. - View Dependent Claims (24, 25, 26, 41, 42, 43, 44)
-
-
27. A computer system that implements a video encoder, the computer system comprising a processor and memory, wherein the video encoder includes:
-
a module configured for encoding plural frames of a video sequence; an encoder-side frame cache, the encoder-side frame cache comprising; a first portion of the encoder-side frame cache for storing a first reference frame, wherein the first portion of the encoder-side frame cache corresponds to a first portion of a decoder-side frame cache at a video decoder, and wherein the first reference frame stored in the first portion of the encoder-side frame cache is updated according to information in frame headers for the plural frames that indicates which of the plural frames are to be cached in the first portion of the decoder-side frame cache after decoding, and a second portion of the encoder-side frame cache for storing a second reference frame, wherein the second portion of the encoder-side frame cache is different than the first portion of the encoder-side frame cache, and wherein the second reference frame stored in the second portion of the encoder-side frame cache is updated to store a current frame of the plural frames after encoding, for use in encoding a next frame of the plural frames; and a module configured for receiving a loss detection signal that indicates loss of encoded data that affects decoding of a given frame of the plural frames, the given frame having been encoded relative to the second reference frame; the video encoder being configured to encode, in response to the loss detection signal, a new frame relative to the first reference frame stored in the first portion of the encoder-side frame cache, the new frame including different data than the encoded data for which loss is detected, the new frame being decodable by the video decoder using encoded data for the new frame and the first reference frame stored in the first portion of the decoder-side frame cache. - View Dependent Claims (28, 45, 46, 47, 48, 49, 50, 51, 52, 53)
-
-
32. One or more computer-readable storage devices or memory having stored thereon computer-executable instructions for causing a computer system programmed thereby to perform operations for encoding plural frames of a video sequence, the operations comprising:
-
encoding a first frame of the plural frames to produce encoded data for the first frame, wherein the encoded data for the first frame includes a frame header for the first frame; storing, for use as a first reference frame, the first frame in a first portion of an encoder-side frame cache that corresponds to a first portion of a decoder-side frame cache at a video decoder, wherein the frame header for the first frame includes information indicating that the first frame is to be stored in the first portion of the decoder-side frame cache; encoding a second frame of the plural frames to produce encoded data for the second frame, the second frame following the first frame in the video sequence; storing, for use as a second reference frame, the second frame in a second portion of the encoder-side frame cache different than the first portion of the encoder-side frame cache; receiving a loss detection signal that indicates loss of encoded data for one of the plural frames; and in response to the loss detection signal, encoding a new frame relative to the first reference frame stored in the first portion of the encoder-side frame cache to produce encoded data for the new frame, the new frame including different data than the one of the plural frames for which loss is detected, the new frame being decodable by the video decoder using the encoded data for the new frame and the first reference frame stored in the first portion of the decoder-side frame cache. - View Dependent Claims (33, 34, 35, 36, 37, 38, 39, 54)
-
Specification