Continuous generation of non-displayed reference frame in video encoding and decoding

• US 10,070,142 B2
• Filed: 11/11/2014
• Issued: 09/04/2018
• Est. Priority Date: 11/11/2014
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Davies, Thomas
• Primary Examiner(s): Holder, Anner N
• Application Number:
  US14/538,272
• Publication Number:
  US 20160134882A1
• Time in Patent Office: 1,393 Days
• CPC Class Codes: H04N 19/164 Feedback from the receiver or from the transmission channel
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/174 the region being a slice, e.g. a line of blocks or a group of blocks
• CPC Class Codes: H04N 19/23 with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/50 using predictive coding H04N19/61 takes precedence

Abstract

Methods and systems for the transmission of a high quality background reference frame (HQRF). Such a reference frame is sent from a video encoder to a video decoder in a number of portions, or slices. In the context of this description, a slice of a frame is a set of blocks in the frame, where the slice is coded separately from other slices of the frame. The slices are sent at respective intervals, so that the decoder receives the high quality background reference frame over time, in a piecemeal fashion. The slices can be buffered at the decoder, and the buffer may be continuously updated and used for reference at any point.

Claims

1. A method of creating a background reference frame at a video decoder comprising:

at each of a plurality of time intervals, wherein each time interval is 0.5 seconds to 5.0 seconds in length,

receiving one or more coded, high fidelity portions of a background reference frame from a video encoder;

decoding the one or more portions; and

storing the one or more decoded portions in a buffer such that, after the plurality of time intervals, some or all of a background reference frame is stored in the buffer.

2. The method of claim 1, wherein each portion represents 1 to 2% of the background reference frame.

3. The method of claim 1, further comprising:

receiving subsequent data that is predictively coded by reference to data in the background reference frame.

4. The method of claim 3, further comprising:

prediction that is performed on the subsequently received data using a partially complete background reference frame.

5. The method of claim 4, wherein the background reference frame comprises incomplete regions that contain default data, the default data comprising data from another reference frame or a constant tone color.

6. The method of claim 1, further comprising:

receiving an updated portion of the background reference frame;

decoding the updated portion to produce a decoded updated portion; and

as a result of an explicit instruction from the video encoder, overwriting a corresponding portion of the background reference frame in the buffer, using the decoded updated portion.

7. The method of claim 6, wherein the overwriting is initiated as a result of an explicit instruction from the video encoder.

8. The method of claim 6, wherein the overwriting is initiated as a result of a determination, at the video decoder, that the updated portion is of superior quality relative to the corresponding portion of the background reference frame.

9. A video decoder, comprising:

a processor to decode one or more coded high fidelity portions of a background reference frame that are received from a video encoder and each of a plurality of time intervals, wherein each time interval is 0.5 to 5.0 seconds in length; and

a buffer that stores some or all of a complete background reference frame comprising the decoded portions after the plurality of time intervals.

10. The method of claim 9, wherein each portion represents 1 to 2% of the entire background reference frame area.

11. The video decoder of claim 9, wherein the buffer is a buffer dedicated to storage of the entire background reference frame.

12. The video decoder of claim 9, wherein the processor decodes an updated portion of the background reference frame to produce a decoded updated portion, and overwrites a portion of the background reference frame in the buffer, using the decoded updated portion.

13. The video decoder of claim 12, wherein the processor overwrites the portion of the background reference frame as a result of a determination that the updated portion is of superior quality relative to the corresponding portion of the completed background reference frame.

14. The video decoder of claim 12, wherein the processor overwrites the portion of the background reference frame as a result of an explicit instruction from the video encoder.

15. One or more non-transitory computer readable storage media encoded with software comprising computer executable instructions that when executed by a processor, cause the processor to:

at each of a plurality of time intervals,

receive one or more coded, high fidelity portions of a background reference frame from a video encoder, wherein each portion represents 1 to 2% of the background reference frame;

decode the one or more portions; and

store the one or more decoded portions in a buffer, such that after the plurality of time intervals, some or all of a complete background reference frame is stored in the buffer.

16. The computer readable storage media of claim 15, wherein each time interval is 0.5 seconds to 5.0 seconds in length.

17. The computer readable storage media of claim 15, wherein the computer executable instructions further comprise instructions that cause the processor to receive a low fidelity background reference frame prior to receiving the high fidelity portions.

18. The computer readable storage media of claim 15, wherein the instructions that cause the processor to store the one or more decoded portions in a buffer comprise instructions that cause the processor to store the one or more decoded portions and a dedicated buffer.

19. The computer readable storage media of claim 15, wherein the computer executable instructions further comprise instructions that cause the processor to:

receive an updated portion of the background reference frame;

decode the updated portion to produce a decoded updated portion; and

overwrite a corresponding portion of the background reference frame in the buffer, using the decoded updated portion.

20. The computer readable storage media of claim 19, wherein the instructions that cause the processor to overwrite are executed as a result of a determination, at the video decoder, that the updated portion is of superior quality relative to the corresponding portion of the background reference frame.

21. The computer readable storage media of claim 19, wherein the instructions that cause the processor to overwrite are executed as a result of an explicit instruction from the video encoder.

22. The computer readable storage media of claim 15, further comprising instructions to cause the processor to receive subsequent data that is predictively coded by reference to data in the background reference frame.

23. The computer readable storage media of claim 22, further comprising instructions to cause the processor to perform prediction on the subsequently received data using a partially complete background reference frame.