Power and computational load management techniques in video processing

• US 8,948,270 B2
• Filed: 12/16/2008
• Issued: 02/03/2015
• Est. Priority Date: 08/19/2008
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Raveendran, Vijayalakshmi
• Primary Examiner(s): PRINCE, JESSICA MARIE
• Application Number:
  US12/336,362
• Publication Number:
  US 20100046637A1
• Time in Patent Office: 2,240 Days
• Field of Search: 375/240.26, 725/150
• US Class Current: 375/240.26
• CPC Class Codes: H04N 19/127 Prioritisation of hardware or computational resources
• CPC Class Codes: H04N 19/174 the region being a slice, e.g. a line of blocks or a group of blocks
• CPC Class Codes: H04N 19/42 characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 21/4348 Demultiplexing of additional data and video streams
• CPC Class Codes: H04N 21/4382 Demodulation or channel decoding, e.g. QPSK demodulation analog front ends or means for connecting modulators, demodulators or transceivers to a transmission line H04L27/0002
• CPC Class Codes: H04N 21/4424 Monitoring of the internal components or processes of the client device, e.g. CPU or memory load, processing speed, timer, counter or percentage of the hard disk space used error monitoring in general G06F11/30; arrangements for monitoring conditions of receiving stations in broadcast systems H04H60/32; diagnosis, testing or measuring for television receivers H04N17/04
• CPC Class Codes: H04N 21/443 OS processes, e.g. booting an STB, implementing a Java virtual machine in an STB or power management in an STB
• CPC Class Codes: H04N 21/4432 Powering on the client, e.g. bootstrap loading using setup parameters being stored locally or received from the server resetting in general G06F1/14; program loading or initiating in general G06F9/445; bootstrapping in general G06F9/4401; secure boots of computer platforms G06F21/57
• CPC Class Codes: H04N 21/4436 Power management, e.g. shutting down unused components of the receiver power management in computer systems G06F1/3203; hibernate or awake process in computer systems G06F9/4418

Abstract

Techniques for power and computational load management in video processing and decoding are provided. In one configuration, an apparatus comprising a processor having a set of instructions operative to extract and compile information from a data stream having video is provided. The processor is operative to prioritize a set of parsing or decoding operations to process the data stream referred to as power management (PM) sequences based on the information and calculate projections of at least one of power and computational loading for each of the prioritized PM sequences.

Claims

1. A method of processing a data stream that includes video data, comprising the steps of:

extracting and compiling information from the data stream, the data stream comprising a transport layer and a video layer, wherein the compiling comprises computing information that is not directly extractable from the transport layer;

determining, based on the extracted and compiled information;

power management (PM) sequences of decodable units from the data stream, wherein a decodable unit comprises at least one of a packet, a picture, a slice, or a macroblock, and

a priority for each of the power management sequences to generate prioritized power management sequences;

selecting a low power mode in response to an amount of available electrical power or processing power; and

in response to the selected low power mode, selectively decoding at least one of the prioritized power management sequences based on the priority for each of the power management sequences, wherein the selective decoding is based on at least one of a power required by each of the prioritized power management (PM) sequences and a computational load required for each of the prioritized power management (PM) sequences.

2. The method of claim 1, wherein:

determining the priority is based on look-ahead processing of at least one of decoding and parsing operations.

3. The method of claim 1, further comprising the step of:

calculating at least one of a projection of a power required by each of the prioritized power management (PM) sequences and a projection of a computational load required for each of the prioritized power management (PM) sequences.

4. The method of claim 1, further comprising the step of:

prior to selecting the low power mode, generating a hierarchical list of low power modes based on the prioritization of the power management (PM) sequences.

5. The method of claim 1, wherein determining the priority of the power management (PM) sequences is based on at least one of visual quality of the video and granularity of the decoding and parsing operations.

6. A device for processing a data stream that includes video data, comprising:

a parser for extracting and compiling information from the data stream, the data stream comprising a transport layer and video layer, wherein the compiling comprises computing information that is not directly extractable from the transport layer; and

a processing unit for;

determining, based on the extracted and compiled information, power management (PM) sequences of decodable units from the data stream decoding operations, wherein a decodable unit comprises at least one of a packet, a picture, a slice, or a macroblock,

determining, based on the extracted and compiled information, a priority for each of the power management sequences to generate prioritized power management sequences,

selecting a low power mode in response to an amount of available electrical power or processing power; and

selectively decoding, in response to the selected low power mode, at least one of the prioritized power management sequences based on the priority for each of the power management sequences, wherein the selective decoding is based on at least one of a power required by each of the prioritized power management (PM) sequences and a computational load required for each of the prioritized power management (PM) sequences.

7. The device of claim 6, further comprising a projector for calculating at least one of a projection of a power required by each of the prioritized power management (PM) sequences and a projection of a computational load required for each of the prioritized power management (PM) sequences.

8. The device of claim 6, wherein determining the priority of the power management (PM) sequences is based on at least one of visual quality of the video and granularity of the decoding operations.

9. The device of claim 6, further comprising a generator for generating, prior to selecting the low power mode, a hierarchical list of low power modes based on the prioritization of the power management (PM) sequences.

10. An apparatus for processing a data stream that includes video data, comprising:

means for extracting and compiling information from the data stream, the data stream comprising a transport layer and a video layer, wherein the compiling comprises computing information that is not directly extractable from the transport layer;

means for determining based on the extracted and compiled information;

power management (PM) sequences of decodable units from the data stream, wherein a decodable unit comprises at least one of a packet, a picture, a slice, or a macroblock, and

a priority for each of the power management sequences to generate prioritized power management sequences;

means for selecting a low power mode in response to an amount of available electrical power or processing power; and

means for selectively decoding, in response to the selected low power mode, at least one of the prioritized power management sequences based on the priority for each of the power management sequences, wherein the selective decoding is based on at least one of a power required by each of the prioritized power management (PM) sequences and a computational load required for each of the prioritized power management (PM) sequences.

11. The apparatus of claim 10, wherein the means for determining the priority of the power management (PM) sequences prioritizes based on look-ahead processing of the decoding operations.

12. The apparatus of claim 10, further comprising:

means for calculating at least one of a projection of a power required by each of the prioritized power management (PM) sequences and a projection of a computational load required for each of the prioritized power management (PM) sequences.

13. The apparatus of claim 10, wherein determining the priority of the power management (PM) sequences is based on at least one of visual quality of the video and granularity of the decoding operations.

14. The apparatus of claim 10, further comprising:

means for generating, prior to selecting the low power mode, a hierarchical list of low power modes based on the prioritization of the power management (PM) sequences.

15. A computer program product for processing a data stream that includes video data, the computer program product including a non-transitory computer readable medium storing instructions that when executed cause a processor to:

extract and compile information from the data stream, the data stream comprising a transport layer and a video layer, wherein the compiling comprises computing information that is not directly extractable from the transport layer;

determine, based on the extracted and compiled information;

power management (PM) sequences of decodable units from the data stream, wherein a decodable unit comprises at least one of a packet, a picture, a slice, or a macroblock, and

a priority for each of the power management sequences to generate prioritized power management sequences;

select a low power mode in response to an amount of available electrical power or processing power; and

in response to the selected low power mode, selectively decode at least one of the prioritized power management sequences based on the priority for each of the power management sequences, wherein the selective decoding is based on at least one of a power required by each of the prioritized power management (PM) sequences and a computational load required for each of the prioritized power management (PM) sequences.

16. The computer program product of claim 15, wherein determining the priority of the power management (PM) sequences is based on look-ahead processing of decoding operations.

17. The computer program product of claim 15, wherein the computer readable medium further stores instructions that when executed cause a processor to:

calculate at least one of a projection of a power required by each of the prioritized power management (PM) sequences and a projection of a computational load required for each of the prioritized power management (PM) sequences.

18. The computer program product of claim 15, wherein determining the priority of the power management (PM) sequences is based on at least one of visual quality of the video and granularity of the decoding operations.

19. The computer program product of claim 15, wherein the computer readable medium further stores instructions that when executed cause a processor to:

generate a hierarchical list of low power modes, prior to selecting the low power mode, based on the prioritization of the power management (PM) sequences.