Switching decode resolution during video decoding

• US 8,358,701 B2
• Filed: 04/15/2005
• Issued: 01/22/2013
• Est. Priority Date: 04/15/2005
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Chou, Peter Hungye, Agnoli, Giovanni, Guo, Haitao, Bhattacharya, Siladitya, Kasarabada, Vikrant
• Primary Examiner(s): Diep, Nhon
• Assistant Examiner(s): LI, TRACY Y
• Application Number:
  US11/107,987
• Publication Number:
  US 20060233259A1
• Time in Patent Office: 2,839 Days
• Field of Search: 37524001-24006, 37524012-24016, 37524025-24026
• US Class Current: 375/240.25
• CPC Class Codes: H04N 19/102 characterised by the element, parameter or selection affected or controlled by the adaptive coding
• CPC Class Codes: H04N 19/122 Selection of transform size, e.g. 8x8 or 2x4x8 DCT; Selection of sub-band transforms of varying structure or type
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/16 for a given display mode, e.g. for interlaced or progressive display mode
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/18 the unit being a set of transform coefficients
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/513 Processing of motion vectors
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution

Abstract

The decode resolution of a video signal can be varied during decoding by decoding a first portion of a video signal at a first resolution; determining a second resolution, different from the first resolution, at which a second portion of the video signal is to be decoded; adjusting one or more reference pictures associated with the video signal to correspond to the second resolution; and decoding the second portion of the video signal at the second resolution. The second resolution, at which the second portion of the video signal is decoded, can be selected from a plurality of predetermined decoding resolutions in response to the detection of a predetermined condition or in response to an input received from a user. Additionally, one or more reference pictures associated with the video signal can be up-sampled or down-sampled in at least one of the horizontal direction and the vertical direction.

Claims

1. A method of varying a decode resolution of a video signal during decoding, the method comprising:

decoding, using a processor, a first portion of a video signal at a first resolution to generate a first portion of an output signal;

caching, during said decoding the first portion of the video signal, reference pictures associated with the video signal that have been decoded at the first resolution, the cached reference pictures to be used for decoding subsequent pictures at the first resolution;

determining, during said decoding the first portion of the video signal, a second resolution, different from the first resolution, at which a second portion of the video signal is to be decoded;

identifying, using the processor in response to said determining the second resolution, a reference picture from the first portion of the video signal that is required to decode the second portion of the video signal;

instead of re-decoding the identified reference picture at the second resolution, generating, by the processor, an instance of the identified reference picture at the second resolution at least in part by adjusting a cached instance of the identified reference picture that has been decoded at the first resolution; and

decoding, using the processor and at least in part based on the instance of the identified reference picture that has been generated at the second resolution, the second portion of the video signal at the second resolution to generate a second portion of the output signal.

2. The method of claim 1, wherein said determining comprises selecting the second resolution from a plurality of predetermined decoding resolutions.

3. The method of claim 2, wherein said determining the second resolution comprises automatically selecting in response to detecting a predetermined condition.

4. The method of claim 3, wherein the predetermined condition is associated with a change in the availability of one or more system resources that occurred during said decoding the first portion of the video signal.

5. The method of claim 2, wherein said selecting the second resolution is performed in response to an input received from a user during said decoding the first portion of the video signal.

6. The method of claim 1, wherein said adjusting further comprises down-sampling the cached instance of the identified reference picture that has been decoded at the first resolution in at least one of a horizontal direction and a vertical direction.

7. The method of claim 1, wherein said adjusting further comprises up-sampling the cached instance of the identified reference picture that has been decoded at the first resolution in at least one of a horizontal direction and a vertical direction.

8. The method of claim 7, wherein said up-sampling further comprises:

duplicating image information representing a row or a column associated with the cached instance of the identified reference picture that has been decoded at the first resolution; and

inserting the duplicated image information into the cached instance of the identified reference picture that has been decoded at the first resolution to generate an up-sampled instance of the identified reference picture at the second resolution.

9. The method of claim 7, wherein said up-sampling further comprises:

interpolating image information representing a row or a column associated with the cached instance of the identified reference picture that has been decoded at the first resolution from a plurality of rows or columns of the cached instance of the identified reference picture; and

inserting the interpolated image information into the cached instance of the identified reference picture to generate an up-sampled instance of the identified reference picture at the second resolution.

10. The method of claim 1, wherein said adjusting further comprises:

identifying a resolution associated with the cached instance of the identified reference picture that has been decoded at the first resolution; and

determining a factor by which the cached instance of the identified reference picture that has been decoded at the first resolution can be adjusted to generate the instance of the identified reference picture at the second resolution.

24. The method of claim 1, further comprising caching, during said decoding the second portion of the video signal, other reference pictures associated with the video signal that have been decoded at the second resolution, the cached other reference pictures to be used for decoding subsequent pictures decoded at the second resolution.

11. A system for varying a decode resolution of a video signal during decoding, the system comprising processor electronics configured to perform operations comprising:

decoding a first portion of a video signal at a first resolution to generate a first portion of an output signal;

caching, during said decoding the first portion of the video signal, reference pictures associated with the video signal that have been decoded at the first resolution, the cached reference pictures to be used for decoding subsequent pictures at the first resolution;

determining, during said decoding the first portion of the video signal, a second resolution, different from the first resolution, at which a second portion of the video signal is to be decoded;

identifying, in response to said determining the second resolution, a reference picture from the first portion of the video signal that is required to decode the second portion of the video signal;

instead of re-decoding the identified reference picture at the second resolution, generating an instance of the identified reference picture at the second resolution at least in part by adjusting a cached instance of the identified reference picture that has been decoded at the first resolution; and

decoding, and at least in part based on the instance of the identified reference picture that has been generated at the second resolution, the second portion of the video signal at the second resolution to generate a second portion of the output signal.

12. The system of claim 11, wherein the processor electronics are configured to perform operations including selecting the second resolution from a plurality of predetermined decoding resolutions.

13. The system of claim 12, wherein the processor electronics are further configured to perform operations including automatically selecting the second resolution in response to detection of a predetermined condition.

14. The system of claim 13, wherein the predetermined condition is associated with a change in the availability of one or more system resources that occurred during said decoding the first portion of the video signal.

15. The system of claim 12, wherein the processor electronics are further configured to perform operations including selecting the second resolution in response to an input received from a user during said decoding the first portion of the video signal.

16. The system of claim 11, wherein the adjusting comprises down-sampling the cached instance of the identified reference picture that has been decoded at the first resolution in at least one of a horizontal direction and a vertical direction.

17. The system of claim 11, wherein the adjusting comprises up-sampling the cached instance of the identified reference picture that has been decoded at the first resolution in at least one of a horizontal direction and a vertical direction.

18. The system of claim 17, wherein the up-sampling comprises:

duplicating image information representing a row or a column associated with the cached instance of the identified reference picture that has been decoded at the first resolution; and

inserting the duplicated image information into the cached instance of the identified reference picture that has been decoded at the first resolution to generate an up-sampled instance of the identified reference picture at the second resolution.

19. The system of claim 17, wherein the up-sampling comprises:

interpolating image information representing a row or a column associated with the cached instance of the identified reference picture that has been decoded at the first resolution from a plurality of rows or columns of the cached instance of the identified reference picture; and

inserting the interpolated image information into the cached instance of the identified reference picture to generate an up-sampled instance of the identified reference picture at the second resolution.

20. The system of claim 11, wherein the adjusting comprises:

identifying a resolution associated with the cached instance of the identified reference picture that has been decoded at the first resolution; and

determining a factor by which the cached instance of the identified reference picture that has been decoded at the first resolution can be adjusted to generate the instance of the identified reference picture at the second resolution.

25. The system of claim 11, wherein the processor electronics are further configured to perform operations including caching, during said decoding the second portion of the video signal, other reference pictures associated with the video signal that have been decoded at the second resolution, the cached other reference pictures to be used for decoding subsequent pictures decoded at the second resolution.

21. A method of varying a decode resolution of a video signal during playback, the method comprising:

displaying a first portion of a video signal including pictures decoded in accordance with a first resolution;

caching, during said displaying the first portion of the video signal, reference pictures associated with the video signal that have been decoded at the first resolution, the cached reference pictures to be used for decoding subsequent pictures at the first resolution;

detecting, during said displaying the first portion of the video signal, an occurrence of a predetermined condition;

determining in response to the detected occurrence of the predetermined condition a second resolution, wherein the second resolution is different from the first resolution, at which a second portion of the video signal is to be decoded;

identifying, using a processor in response to said determining the second resolution, a reference picture from the first portion of the video signal that is required to decode the second portion of the video signal;

instead of re-decoding the identified reference picture at the second resolution, generating, by the processor, an instance of the identified reference picture at the second resolution at least in part by adjusting a cached instance of the identified reference picture that has been decoded at the first resolution; and

displaying the second portion of the video signal in accordance with the second resolution such that the second portion of the video signal includes pictures decoded at least in part based on the instance of the identified reference picture that has been generated the second resolution.

22. The method of claim 21, wherein said detecting the occurrence of the predetermined condition comprises determining that availability of a system resource has decreased below a level sufficient to decode the video signal at the first resolution in real time.

23. The method of claim 21, further comprising:

determining a third resolution, wherein the third resolution can differ from the first resolution and the second resolution; and

decoding and displaying a second video signal in accordance with the third resolution, wherein at least a portion of the first video signal and a portion of the second video signal are decoded and displayed simultaneously.