Keyframe mitigation for video streams with multiple receivers

• US 10,182,248 B2
• Filed: 11/21/2016
• Issued: 01/15/2019
• Est. Priority Date: 11/21/2016
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Bright-Thomas, Paul G.
• Primary Examiner(s): Dubasky, Gigi L
• Application Number:
  US15/357,066
• Publication Number:
  US 20180146221A1
• Time in Patent Office: 785 Days
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/65 using error resilience
• CPC Class Codes: H04N 19/895 in combination with error concealment
• CPC Class Codes: H04N 21/234 Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
• CPC Class Codes: H04N 21/2365 Multiplexing of several video streams
• CPC Class Codes: H04N 21/2407 Monitoring of transmitted content, e.g. distribution time, number of downloads arrangements for monitoring programmes for broadcast or for distribution combined with broadcast H04H20/14
• CPC Class Codes: H04N 21/6379 directed to encoder , e.g. for requesting a lower encoding rate

Abstract

In one embodiment, a video encoder generates an encoded bitstream representing a sequence of video frames including a keyframe. The encoder generates information for use by a decoder that receives the encoded bitstream to enable the decoder to generate display frames from a pre-keyframe video frame that is prior to the keyframe in the sequence of video frames. The encoded bitstream is sent to the decoder. In another embodiment, a video decoder receives from an encoder an encoded bitstream representing a sequence of video frames including a keyframe. The keyframe includes information to enable the decoder to generate display frames from a pre-keyframe video frame that was received prior to the keyframe in the sequence of video frames. The decoder generates display frames using the pre-keyframe video frame, information included with the keyframe and information included with an encoder-determined number of decoded frames subsequent to the keyframe.

Claims

1. A method comprising:

at a video decoder, receiving from an encoder an encoded bitstream representing a sequence of video frames, the sequence of video frames including a keyframe and at least one pre-keyframe video frame received prior to the keyframe in the sequence of video frames, wherein the keyframe includes supplemental syntax to enable the decoder to generate display frames based on the at least one pre-keyframe video frame that was received prior to the keyframe;

generating one or more improvement frames based on the at least one pre-keyframe video frame using the supplemental syntax included with the keyframe; and

generating a predetermined number of display frames subsequent to the keyframe, wherein each of the predetermined number of display frames is generated using a combination of an improvement frame from the one or more improvement frames and a decoded frame; and

upon completion of generating the predetermined number of display frames, generating additional subsequent display frames using a decoded frame without an improvement frame.

2. The method of claim 1, wherein the predetermined number of display frames is determined by the encoder.

3. The method of claim 1, wherein generating each of the predetermined number of display frames comprises:

generating a display frame based on an improvement frame generated for a position in the sequence of video frames and a decoded frame at a corresponding position in the sequence of video frames.

4. The method of claim 1, wherein generating each of the predetermined number of display frames is based on a weighted combination of an improvement frame and a decoded frame at a corresponding position in the sequence of video frames for each of the predetermined number of display frames.

5. The method of claim 1, wherein generating the one or more improvement frames is based on one or more motion vectors received in the encoded bitstream and applied to the pre-keyframe video frame.

6. The method of claim 1, wherein a sequence of the one or more improvement frames is generated for combination with the keyframe and one or more successive frames and applied to a point in the sequence that display frames derived from the encoded bitstream are no longer improved by reference to the pre-keyframe video frame.

7. The method of claim 4, wherein the supplemental syntax included with the keyframe indicates a weighting to be used for the weighted combination of the improvement frame and the decoded frame.

8. The method of claim 1, wherein a flag included with one or more blocks of the sequence of video frames indicates to the video decoder to use the supplemental syntax.

9. The method of claim 1, further comprising determining whether or not to generate and output display frames generated from the pre-keyframe video frame.

10. The method of claim 9, wherein determining comprises:

computing a checksum associated with the pre-keyframe video frame;

comparing the computed checksum with a checksum calculated at the encoder and included by the encoder in data sent by the encoder to the video decoder subsequent to the keyframe; and

wherein generating comprises generating display frames using information from the pre-keyframe video frame when the computed checksum matches the checksum sent by the encoder, and otherwise, not generating display frames from the pre-keyframe video frame.

11. A method comprising:

at a video encoder, generating an encoded bitstream representing a sequence of video frames including a keyframe, the sequence of video frames including a keyframe and at least one pre-keyframe video frame that is prior to the keyframe in the sequence of video frames;

generating supplemental syntax for use by a decoder that receives the encoded bitstream to enable the decoder to generate display frames based on the at least one pre-keyframe video frame that is prior to the keyframe in the sequence of video frames, wherein the supplemental syntax enables the decoder to generate one or more improvement frames based on the at least one pre-keyframe video frame and enables the decoder to generate a predetermined number of display frames subsequent to the keyframe, wherein each of the predetermined number of display frames is generated by the decoder using a combination of an improvement frame from the one or more improvement frames and a decoded frame; and

including the supplemental syntax with the keyframe and with an encoder-determined number of frames subsequent to the keyframe.

12. The method of claim 11, wherein the supplemental syntax indicates whether and how the decoder is to generate an improvement frame from the pre-keyframe video frame.

13. The method of claim 11, wherein the supplemental syntax indicates a weighting to be made by the decoder in combining the improvement frame, generated by the decoder using the pre-keyframe video frame, with the decoded frame derived from the encoded bitstream.

14. The method of claim 11, further comprising:

determining whether a scene associated with the encoded bitstream has changed; and

wherein including the supplemental syntax is not performed when it is determined that the scene associated with the encoded bitstream has changed.

15. The method of claim 11, further comprising:

determining that a part of the keyframe is visually continuous with prior video frames;

wherein the generating the supplemental syntax comprises generating the supplemental syntax to cause the decoder to use the pre-keyframe video frame to generate and apply improvement data for only a sub-frame region that is visually continuous with prior video frames.

16. The method of claim 11, wherein decoders that are capable of interpreting the supplemental syntax and decoders are not capable of interpreting the supplemental syntax can decode the keyframe and the encoder-determined number of decoded frames.

17. The method of claim 11, further comprising including in a video frame subsequent to the keyframe, information to indicate to the decoder whether or not to generate and output display frames generated from the pre-keyframe video frame.

18. The method of claim 17, wherein the information included in the video frame subsequent to the keyframe is a checksum computed by the encoder from the pre-keyframe video frame used for comparison with a checksum computed for the pre-keyframe video frame by the decoder.

19. An apparatus comprising:

a communication interface configured to enable communications over a network;

a processor coupled to the communication interface, and configured to;

receive, via the communication interface, from an encoder an encoded bitstream representing a sequence of video frames, the sequence of video frames including a keyframe and at least one pre-keyframe video frame received prior to the keyframe in the sequence of video frames, wherein the keyframe includes supplemental syntax to enable the processor to generate display frames based on the at least one pre-keyframe video frame that was received prior to the keyframe;

generate one or more improvement frames based on the at least one pre-keyframe video frame using the supplemental syntax included with the keyframe; and

generate a predetermined number of display frames subsequent to the keyframe, wherein each of the predetermined number of display frames is generated using a combination of an improvement frame from the one or more improvement frames and a decoded frame; and

upon completion of generating the predetermined number of display frames, generate additional subsequent display frames using a decoded frame without an improvement frame.

20. The apparatus of claim 19, wherein the processor is further configured to generate each of the predetermined number of display frames based on a weighted combination of an improvement frame and a decoded frame at a corresponding position in the sequence of video frames for each of the predetermined number of display frames.