Power and computational load management techniques in video processing

• US 8,964,828 B2
• Filed: 12/16/2008
• Issued: 02/24/2015
• Est. Priority Date: 08/19/2008
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Raveendran, Vijayalakshmi
• Primary Examiner(s): BROWN, CHRISTOPHER J
• Application Number:
  US12/336,347
• Publication Number:
  US 20100046631A1
• Time in Patent Office: 2,261 Days
• Field of Search: 375/240, 725/150, 713/320
• US Class Current: 375/240
• CPC Class Codes: H04N 19/127 Prioritisation of hardware or computational resources
• CPC Class Codes: H04N 19/174 the region being a slice, e.g. a line of blocks or a group of blocks
• CPC Class Codes: H04N 19/42 characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 21/4348 Demultiplexing of additional data and video streams
• CPC Class Codes: H04N 21/4382 Demodulation or channel decoding, e.g. QPSK demodulation analog front ends or means for connecting modulators, demodulators or transceivers to a transmission line H04L27/0002
• CPC Class Codes: H04N 21/4424 Monitoring of the internal components or processes of the client device, e.g. CPU or memory load, processing speed, timer, counter or percentage of the hard disk space used error monitoring in general G06F11/30; arrangements for monitoring conditions of receiving stations in broadcast systems H04H60/32; diagnosis, testing or measuring for television receivers H04N17/04
• CPC Class Codes: H04N 21/4432 Powering on the client, e.g. bootstrap loading using setup parameters being stored locally or received from the server resetting in general G06F1/14; program loading or initiating in general G06F9/445; bootstrapping in general G06F9/4401; secure boots of computer platforms G06F21/57
• CPC Class Codes: H04N 21/4436 Power management, e.g. shutting down unused components of the receiver power management in computer systems G06F1/3203; hibernate or awake process in computer systems G06F9/4418

Abstract

Techniques for managing power consumption and computational load on a processor during video processing and decoding are provided. One representative embodiment discloses a method of processing a data stream that includes video data. According to the method, one or more protocols used to create the data stream are identified. The various parsing and decoding operations required by the protocol are then identified and managed based on the available electrical power or available processing power. Another representative embodiment discloses a method of processing a data stream that includes video data. According to the method, one or more protocols used to create the data stream are identified. The various parsing and decoding operations required by the protocol are then identified and managed based on a visual quality of the video or a quality of experience.

Claims

1. A method of processing a data stream, comprising the steps of:

receiving a data stream that includes video data;

identifying a transport protocol and a video coding protocol used to create the data stream;

identifying various parsing and decoding operations required by the transport protocol and the video coding protocol;

prioritizing a plurality of sequences of decodable units from the data stream for at least one of selective parsing or selective decoding based on the identified parsing and decoding operations;

evaluating an amount of available electrical power and an amount of available processing power; and

managing the various parsing operations and decoding operations for the data stream based on the amount of available electrical power and the amount of available processing power, wherein the managing of the various parsing operations and decoding operations comprises selectively carrying out the parsing and decoding operations for the data stream, and wherein selective parsing or decoding comprises parsing or decoding at least one higher priority sequence at a higher granularity or visual quality than a lower priority sequence to maximize the resulting visual quality corresponding to a power consumption target.

2. The method of claim 1, further comprising the step of:

associating the various parsing operations and decoding operations into groups wherein each group has at least one of a projected electrical power requirement and a projected processing power requirement.

3. The method of claim 2, further comprising the step of:

prioritizing the groups into a hierarchical list of low power modes based on at least one of the projected electrical power requirement and projected processing power requirement.

4. The method of claim 2, further comprising the step of:

selecting a group such that at least one of the selected group'"'"'s projected electrical power requirement and projected processing power requirement meets or does not exceed the available electrical power and available processing power, respectively.

5. The method of claim 3, further comprising the steps of:

presenting the hierarchical list of low power modes to a user;

receiving an input indicating the low power mode selected by the user; and

selecting a group of parsing operations and decoding operations in response to the input.

6. A method of processing a data stream, comprising the steps of:

receiving a data stream that includes video data;

identifying a transport protocol and a video coding protocol used to create the data stream;

identifying various parsing and decoding operations required by the transport protocol and the video coding protocol;

prioritizing a plurality of sequences of decodable units from the data stream for at least one of selective parsing or selective decoding based on the identified parsing and decoding operations; and

managing the various parsing operations and decoding operations for the data stream based on at least one of a visual quality of the video and a quality of experience (QoE), wherein the managing of the various parsing operations and decoding operations comprises selectively carrying out the parsing and decoding operations for the data stream, and wherein selective parsing or decoding comprises parsing or decoding at least one higher priority sequence at a higher granularity or visual quality than a lower priority sequence to maximize the resulting visual quality corresponding to a power consumption target.

7. The method of claim 6, further comprising the step of:

associating the various parsing operations and decoding operations into groups wherein each group corresponds to a degree of visual quality of the video or a degree of quality of experience.

8. The method of claim 7, further comprising the step of:

prioritizing the groups into a hierarchical list of quality modes based on at least one of the degree of visual quality of the video and degree of quality of experience.

9. The method of claim 7, further comprising the step of:

selecting a group such that at least one of the selected group'"'"'s visual quality of the video and quality of experience satisfies a desired degree of visual quality of the video and desired degree of quality of experience, respectively.

10. The method of claim 8, further comprising the steps of:

presenting the hierarchical list of quality modes to a user;

receiving an input indicating the quality mode selected by the user; and

selecting a group of parsing operations and decoding operations in response to the input.

11. A device for processing a data stream, comprising:

a processor configured to execute a set of instructions operable to;

receive a data stream that includes video data;

identify a transport protocol and a video coding protocol used to create the data stream;

identify various parsing and decoding operations required by the transport protocol and the video coding protocol;

prioritize a plurality of sequences of decodable units from the data stream for at least one of selective parsing or selective decoding based on the identified parsing and decoding operations;

evaluate an amount of available electrical power and an amount of available processing power; and

manage the various parsing operations and decoding operations for the data stream based on the amount of available electrical power and the amount of available processing power, wherein the managing of the various parsing operations and decoding operations comprises selectively carrying out the parsing and decoding operations for the data stream, and wherein selective parsing or decoding comprises parsing or decoding at least one higher priority sequence at a higher granularity or visual quality than a lower priority sequence to maximize the resulting visual quality corresponding to a power consumption target.

12. The device of claim 11, wherein the processor is further configured to execute instructions to:

associate the various parsing operations and decoding operations into groups wherein each group has at least one of a projected electrical power requirement and a projected processing power requirement.

13. The device of claim 12, wherein the processor is further configured to execute instructions to:

prioritize the groups into a hierarchical list of low power modes based on at least one of the projected electrical power requirement and projected processing power requirement.

14. The device of claim 12, wherein the processor is further configured to execute instructions to:

select a group such that at least one of the selected group'"'"'s projected electrical power requirement and projected processing power requirement meets or does not exceed the available electrical power and available processing power, respectively.

15. The device of claim 13, wherein the processor is further configured to execute instructions to:

present the hierarchical list of low power modes to a user;

receive an input indicating the low power mode selected by the user; and

select a group of parsing operations and decoding operations in response to the input.

16. A device for processing a data stream, comprising:

a processor configured to execute a set of instructions operable to;

receive a data stream that includes video data;

identify a transport protocol and a video coding protocol used to create the data stream;

identify various parsing and decoding operations required by the transport protocol and the video coding protocol;

prioritize a plurality of sequences of decodable units from the data stream for at least one of selective parsing or selective decoding based on the identified parsing and decoding operations; and

manage the various parsing operations and decoding operations for the data stream based on at least one of a visual quality of the video and a quality of experience (QoE), wherein the managing of the various parsing operations and decoding operations comprises selectively carrying out the parsing and decoding operations for the data stream, and wherein selective parsing or decoding comprises parsing or decoding at least one higher priority sequence at a higher granularity or visual quality than a lower priority sequence to maximize the resulting visual quality corresponding to a power consumption target.

17. The device of claim 16, wherein the processor is further configured to execute instructions to:

associate the various parsing operations and decoding operations into groups wherein each group corresponds to a degree of visual quality of the video or a degree of quality of experience.

18. The device of claim 17, wherein the processor is further configured to execute instructions to:

prioritize the groups into a hierarchical list of quality modes based on at least one of the degree of visual quality of the video and degree of quality of experience.

19. The device of claim 17, wherein the processor is further configured to execute instructions to:

select a group such that at least one of the selected group'"'"'s visual quality of the video and quality of experience satisfies a desired degree of visual quality of the video and desired degree of quality of experience, respectively.

20. The device of claim 18, wherein the processor is further configured to execute instructions to:

present the hierarchical list of quality modes to a user;

receive an input indicating the quality mode selected by the user; and

select a group of parsing operations and decoding operations in response to the input.

21. An apparatus for processing a data stream, comprising:

means for receiving a data stream that includes video data;

means for identifying a transport protocol and a video coding protocol used to create the data stream;

means for identifying various parsing and decoding operations required by the transport protocol and the video coding protocol;

means for prioritizing a plurality of sequences of decodable units from the data stream for at least one of selective parsing or selective decoding based on the identified parsing and decoding operations;

means for evaluating an amount of available electrical power and an amount of available processing power; and

means for managing the various parsing operations and decoding operations for the data stream based on the amount of available electrical power and the amount of available processing power, wherein the means for managing the various parsing operations and decoding operations selectively carries out the parsing and decoding operations for the data stream, and wherein selective parsing or decoding comprises parsing or decoding at least one higher priority sequence at a higher granularity or visual quality than a lower priority sequence to maximize the resulting visual quality corresponding to a power consumption target.

22. The apparatus of claim 21, further comprising:

means for associating the various parsing operations and decoding operations into groups wherein each group has at least one of a projected electrical power requirement and a projected processing power requirement.

23. The apparatus of claim 22, further comprising:

means for prioritizing the groups into a hierarchical list of low power modes based on at least one of the projected electrical power requirement and projected processing power requirement.

24. The apparatus of claim 22, further comprising:

means for selecting a group such that at least one of the selected group'"'"'s projected electrical power requirement and projected processing power requirement meets or does not exceed the available electrical power and available processing power, respectively.

25. The apparatus of claim 23, further comprising:

means for presenting the hierarchical list of low power modes to a user;

means for receiving an input indicating the low power mode selected by the user; and

means for selecting a group of parsing operations and decoding operations in response to the input.

26. An apparatus for processing a data stream, comprising:

means for receiving a data stream that includes video data;

means for identifying a transport protocol and a video coding protocol used to create the data stream;

means for identifying various parsing and decoding operations required by the transport protocol and the video coding protocol;

means for prioritizing a plurality of sequences of decodable units from the data stream for at least one of selective parsing or selective decoding based on the identified parsing and decoding operations; and

means for managing the various parsing operations and decoding operations for the data stream based on at least one of a visual quality of the video and a quality of experience (QoE), wherein the means for managing the various parsing operations and decoding operations selectively carries out the parsing and decoding operations for the data stream, and wherein selective parsing or decoding comprises parsing or decoding at least one higher priority sequence at a higher granularity or visual quality than a lower priority sequence to maximize the resulting visual quality corresponding to a power consumption target.

27. The apparatus of claim 26, further comprising:

means for associating the various parsing operations and decoding operations into groups wherein each group corresponds to a degree of visual quality of the video or a degree of quality of experience.

28. The apparatus of claim 27, further comprising:

means for prioritizing the groups into a hierarchical list of quality modes based on at least one of the degree of visual quality of the video and degree of quality of experience.

29. The apparatus of claim 27, further comprising:

means for selecting a group such that at least one of the selected group'"'"'s visual quality of the video and quality of experience satisfies a desired degree of visual quality of the video and desired degree of quality of user experience, respectively.

30. The apparatus of claim 28, further comprising:

means for presenting the hierarchical list of quality modes to a user;

means for receiving an input indicating the quality mode selected by the user; and

means for selecting a group of parsing operations and decoding operations in response to the input.

31. A computer program product including a non-transitory computer readable medium storing instructions which, when executed by a processor, cause the processor to:

receive a data stream that includes video data;

identify a transport protocol and a video coding protocol used to create the data stream;

identify various parsing and decoding operations required by the transport protocol and the video coding protocol;

prioritize a plurality of sequences of decodable units from the data stream for at least one of selective parsing or selective decoding based on the identified parsing and decoding operations;

evaluate an amount of available electrical power and an amount of available processing power; and

manage the various parsing operations and decoding operations for the data stream based on the amount of available electrical power and the amount of available processing power, wherein the managing of the various parsing operations and decoding operations comprises selectively carrying out the parsing and decoding operations for the data stream, and wherein selective parsing or decoding comprises parsing or decoding at least one higher priority sequence at a higher granularity or visual quality than a lower priority sequence to maximize the resulting visual quality corresponding to a power consumption target.

32. The computer program product of claim 31, further comprising instructions which, when executed by a processor, cause the processor to:

associate the various parsing operations and decoding operations into groups wherein each group has at least one of a projected electrical power requirement and a projected processing power requirement.

33. The computer program product of claim 32, further comprising instructions which, when executed by a processor, cause the processor to:

prioritize the groups into a hierarchical list of low power modes based on at least one of the projected electrical power requirement and projected processing power requirement.

34. The computer program product of claim 32, further comprising instructions which, when executed by a processor, cause the processor to:

select a group such that at least one of the selected group'"'"'s projected electrical power requirement and projected processing power requirement meets or does not exceed the available electrical power and available processing power, respectively.

35. The computer program product of claim 33, further comprising instructions which, when executed by a processor, cause the processor to:

present the hierarchical list of low power modes to a user;

receive an input indicating the low power mode selected by the user; and

select a group of parsing operations and decoding operations in response to the input.

36. A computer program product including a non-transitory computer readable medium storing instructions which, when executed by a processor, cause the processor to:

receive a data stream that includes video data;

identify a transport protocol and a video coding protocol used to create the data stream;

identify various parsing and decoding operations required by the transport protocol and the video coding protocol;

prioritize a plurality of sequences of decodable units from the data stream for at least one of selective parsing or selective decoding based on the identified parsing and decoding operations; and

manage the various parsing operations and decoding operations for the data stream based on at least one of a visual quality of the video and a quality of experience (QoE), wherein the managing of the various parsing operations and decoding operations comprises selectively carrying out the parsing and decoding operations for the data stream, and wherein selective parsing or decoding comprises parsing or decoding at least one higher priority sequence at a higher granularity or visual quality than a lower priority sequence to maximize the resulting visual quality corresponding to a power consumption target.

37. The computer program product of claim 36, further comprising instructions which, when executed by a processor, cause the processor to:

associate the various parsing operations and decoding operations into groups wherein each group corresponds to a degree of visual quality of the video or a degree of quality of experience.

38. The computer program product of claim 37, further comprising instructions which, when executed by a processor, cause the processor to:

prioritize the groups into a hierarchical list of quality modes based on at least one of the degree of visual quality of the video and degree of quality of experience.

39. The computer program product of claim 37, further comprising instructions which, when executed by a processor, cause the processor to:

select a group such that at least one of the selected group'"'"'s visual quality of the video and quality of experience satisfies a desired degree of visual quality of the video and desired degree of quality of experience, respectively.

40. The computer program product of claim 37, further comprising instructions which, when executed by a processor, cause the processor to:

present the hierarchical list of quality modes to a user;

receive an input indicating the quality mode selected by the user; and

select a group of parsing operations and decoding operations in response to the input.