Display mode-based video encoding in wireless display devices

• US 9,277,230 B2
• Filed: 11/23/2011
• Issued: 03/01/2016
• Est. Priority Date: 11/23/2011
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Rabii, Khosro M., Sethi, Sumeet Singh
• Primary Examiner(s): Vaughn, Jr., William C
• Assistant Examiner(s): Fuentes, Luis Perez
• Application Number:
  US13/303,931
• Publication Number:
  US 20130128948A1
• Time in Patent Office: 1,560 Days
• Field of Search: 375/240.01
• US Class Current: 1/1
• CPC Class Codes: H04L 65/764 at the destination reformatting of additional data in video clients H04N21/4355
• CPC Class Codes: H04N 19/114 Adapting the group of pictures [GOP] structure, e.g. number of B-frames between two anchor frames H04N19/107 takes precedence
• CPC Class Codes: H04N 19/12 Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/179 the unit being a scene or a shot
• CPC Class Codes: H04N 21/4122 additional display device, e.g. video projector digital output for controlling a plurality of local displays G06F3/1423
• CPC Class Codes: H04N 21/41265 having a remote control device for bidirectional communication between the remote control device and client device unidirectional remote control device H04N21/42204

Abstract

This disclosure relates to techniques for performing display mode based video encoding to reduce power consumption in a source device of a Wireless Display (WD) system. More specifically, the techniques enable the source device to select an encoding process based on whether the video data to be encoded is in a static display mode or a dynamic display mode. For example, when the video data is in the static display mode, the source device selects a static encoding process that reduces an amount of video data processing and compression, which in turn reduces power consumption at the source device. When the video data is in the dynamic mode, the source device may select a conventional dynamic encoding process. The source device encodes the video data according to the selected encoding process, and transmits the encoded video data to one or more sink devices in the WD system for display.

Claims

1. A method of encoding video data comprising:

processing, with a source device in a Wireless Display (WD) system, video data for display at the source device and display at one or more sink devices in the WD system;

determining, with the source device, a display mode of the video data to be encoded based on content information of the video data;

when the display mode of the video data is determined to be a dynamic mode based on the content information indicating that the video data includes dynamic content over a period of time, selecting a first encoding process for the video data;

when the display mode of the video data is determined to be a static mode based on the content information indicating that the video data includes static content over a period of time, selecting a second encoding process for the video data different from the first encoding process,

wherein the second encoding process reduces an amount of video data processing and compression performed by the source device compared to the first encoding process;

adjusting, with the source device, the selected encoding process for the video data based on status information received from the one or more sink devices via an independent channel,

wherein the status information includes at least one of display capability profiles, display related adjustments, or display use status of the sink devices; and

encoding the video data according to the selected encoding process for transmission to the one or more sink devices in the WD system,

wherein encoding the video data according to the second encoding process comprises inter-encoding video frames of the video data as skip-mode frames without performing motion estimation; and

wherein encoding the video data according to the first encoding process comprises;

intra-encoding one or more video frames of the video data according to video hierarchy values specified by a video coding standard used by the source device and the one or more sink devices,

wherein the video hierarchy values-comprise one or more of a group of pictures (GOP) size, a video sequence start position, a picture type, or a slice type for the video data; and

inter-encoding other video frames of the video data using motion estimation according to the video coding standard.

2. The method of claim 1, further comprising transmitting the encoded video data to the one or more sink devices in the WD system for display.

3. The method of claim 1, wherein determining a display mode of the video data comprises determining a display mode for the video data in each of two or more display segments that correspond to different areas of a display device at one or more of the sink devices.

4. The method of claim 3, further comprising, when the display mode of the video data in at least one of the display segments of the display device is determined to be the static mode, encoding the video data in the at least one of the display segments according to the second encoding process, and when the display mode of the video data in another one of the display segments of the display device is determined to be the dynamic mode, encoding the video data in the another one of the display segments according to the first encoding process.

5. The method of claim 1, further comprising signaling the selected encoding process for the video data to the sink devices.

6. The method of claim 1, wherein the static content comprises content that is displayed for a defined period of time without changing.

7. The method of claim 1, wherein encoding the video data according to the second encoding process comprises:

adjusting one or more video hierarchy values to reduce a number of intra-encoded video frames for the video data and reduce motion estimation performance for inter-encoded video frames of the video data compared to the first encoding process, wherein the video hierarchy values comprise one or more of a group of pictures (GOP) size, a video sequence start position, a picture type, or a slice type for the video data;

intra-encoding one or more video frames of the video data according to the adjusted video hierarchy values; and

inter-encoding other video frames of the video data as skip-mode frames without performing motion estimation.

8. The method of claim 7, further comprising:

transmitting the skip-mode frames to the one or more sink devices in the WD system, wherein transmitting the skip-mode frames comprises transmitting an indication to reuse content of a previous frame for the skip-mode frames; and

eliminating retransmission of the skip-mode frames when an acknowledgement of receipt of the transmitted skip-mode frames is not received from the sink devices.

9. A source device in a Wireless Display (WD) system comprising:

a memory to store video data; and

a processor configured to;

process, with a source device in a Wireless Display (WD) system, video data for display at the source device and display at one or more sink devices in the WD system;

determine, with the source device, a display mode of the video data to be encoded based on content information of the video data;

when the display mode of the video data is determined to be a dynamic mode based on the content information indicating that the video data includes dynamic content over a period of time, select a first encoding process for the video data;

when the display mode of the video data is determined to be a static mode based on the content information indicating that the video data includes static content over a period of time, select a second encoding process for the video data different from the first encoding process,

wherein the second encoding process reduces an amount of video data processing and compression performed by the source device compared to the first encoding process;

adjust, with the source device, the selected encoding process for the video data based on status information received from the one or more sink devices via an independent channel,

wherein the status information includes at least one of display capability profiles, display related adjustments, or display use status of the sink devices; and

encode the video data according to the selected encoding process for transmission to the one or more sink devices in the WD system,

wherein encoding the video data according to the second encoding process comprises inter-encoding video frames of the video data as skip-mode frames without performing motion estimation; and

wherein, when the first encoding process is selected, the processor;

intra-encodes one or more video frames of the video data according to video hierarchy values specified by a video coding standard used by the source device and the one or more sink devices,

wherein the video hierarchy values comprise one or more of a group of pictures (GOP) size, a video sequence start position, a picture type, or a slice type for the video data; and

inter-encodes other video frames of the video data using motion estimation according to the video coding standard.

10. The source device of claim 9, further comprising a transmitter that transmits the encoded video data to the one or more sink devices in the WD system for display.

11. The source device of claim 9, wherein the processor determines a display mode for the video data in each of two or more display segments that correspond to different areas of a display device at one or more of the sink devices.

12. The source device of claim 11, wherein, when the display mode of the video data in at least one of the display segments of the display device is determined to be the static mode, the processor encodes the video data in the at least one of the display segments according to the second encoding process, and when the display mode of the video data in another one of the display segments of the display device is determined to be the dynamic mode, the processor encodes the video data in the another one of the display segments according to the first encoding process.

13. The source device of claim 9, wherein the processor signals the selected encoding process for the video data to the sink devices.

14. The source device of claim 9, wherein, when the second encoding process is selected, the processor:

adjusts one or more video hierarchy values to reduce a number of intra-encoded video frames for the video data and reduce motion estimation performance for inter-encoded video frames of the video data compared to the first encoding process, wherein the video hierarchy values comprise one or more of a group of pictures (GOP) size, a video sequence start position, a picture type, or a slice type for the video data;

intra-encodes one or more video frames of the video data according to the adjusted video hierarchy values; and

inter-encodes other video frames of the video data as skip-mode frames without performing motion estimation.

15. The source device of claim 14, wherein the processor transmits the skip-mode frames to the one or more sink devices in the WD system with an indication to reuse content of a previous frame for the skip-mode frames, and eliminates retransmission of the skip-mode frames when an acknowledgement of receipt of the transmitted skip-mode frames is not received from the sink devices.

16. A non-transitory, computer-readable medium comprising instructions for encoding video data that cause one or more processor to:

process, with a source device in a Wireless Display (WD) system, video data for display at the source device and display at one or more sink devices in the WD system;

determine, with the source device, a display mode of the video data to be encoded based on content information of the video data;

when the display mode of the video data is determined to be a dynamic mode based on the content information indicating that the video data includes dynamic content over a period of time, select a first encoding process for the video data;

when the display mode of the video data is determined to be a static mode based on the content information indicating that the video data includes static content over a period of time, select a second encoding process for the video data different from the first encoding process,

wherein the second encoding process reduces an amount of video data processing and compression performed by the source device compared to the first encoding process;

adjust, with the source device, the selected encoding process for the video data based on status information received from the one or more sink devices via an independent channel,

wherein the status information includes at least one of display capability profiles, display related adjustments, and display use status of the sink devices; and

encode the video data according to the selected encoding process for transmission to the one or more sink devices in the WD system,

wherein when the second encoding process is selected, the instructions cause the processors to inter-encode video frames of the video data as skip-mode frames without performing motion estimation; and

further comprising, when the first encoding process is selected, instructions that cause the processors to;

intra-encode one or more video frames of the video data according to video hierarchy values specified by a video coding standard used by the source device and the one or more sink devices,

wherein the video hierarchy values comprise one or more of a group of pictures (GOP) size, a video sequence start position, a picture type, or a slice type for the video data; and

inter-encode other video frames of the video data using motion estimation according to the video coding standard.

17. The non-transitory, computer-readable medium of claim 16, further comprising instructions that cause the processors to transmit the encoded video data to the one or more sink devices in the WD system for display.

18. The non-transitory, computer-readable medium of claim 16, wherein the instructions cause the processor to determine a display mode for the video data in each of two or more display segments that correspond to different areas of a display device at one or more of the sink devices.

19. The non-transitory, computer-readable medium of claim 16, further comprising instructions that cause the processors to signal the selected encoding process for the video data to the sink devices.

20. The computer-readable medium of claim 16, wherein, when the second encoding process is selected, the instructions cause the processors to:

adjust one or more video hierarchy values to reduce a number of intra-encoded video frames for the video data and reduce motion estimation performance for inter-encoded video frames of the video data compared to the first encoding process, wherein the video hierarchy values comprise one or more of a group of pictures (GOP) size, a video sequence start position, a picture type, or a slice type for the video data;

intra-encode one or more video frames of the video data according to the adjusted video hierarchy values; and

inter-encode other video frames of the video data as skip-mode frames without performing motion estimation.