Adjusting sampling rate for encoding

• US 7,653,250 B2
• Filed: 04/28/2005
• Issued: 01/26/2010
• Est. Priority Date: 04/28/2005
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Normile, Jim, Tung, Berkat Shing, Jeong, Hyeonkuk
• Primary Examiner(s): Carter; Aaron W
• Application Number:
  US11/118,554
• Publication Number:
  US 20060244816A1
• Time in Patent Office: 1,734 Days
• Field of Search: 382/232, 348 1401- 1416, 718/100, 718/102, 709/231, 709/203, 370260-271, 715719-726
• US Class Current: 382/232
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/6377 directed to server one-way streaming services wherein the source is controlled by the destination H04L65/613
• CPC Class Codes: H04N 21/6379 directed to encoder , e.g. for requesting a lower encoding rate
• CPC Class Codes: H04N 21/658 Transmission by the client directed to the server
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

Some embodiments provide a method for adjusting a video sampling rate during an video encoding operation. The method receives a metric that quantifies the usage of a computer that performs the video encoding operation. The method computes an adjustment factor based on the metric. In some embodiments the metric is a system idle time. The method defines the video sampling rate based on the adjustment factor. Some embodiments compute the adjustment factor by performing a non-linear operation based on the metric. In some embodiments, the non-linear operation includes performing an integral operation based on the metric. In some embodiments, the non-linear operation includes performing a derivative operation based on the metric. Some embodiments perform more than one operation to compute the adjustment factor. In some embodiments, each operation is assigned a weight.

Claims

1. A method for adjusting a video sampling rate that is used during a video encoding operation of a multi-participant video conference, the method comprising:

at a computing device of a first participant of said video conference;

defining an initial video sampling rate for determining the rate at which a composite video of said video conference is generated;

receiving video content of at least said first participant and a second participant of said conference;

receiving a metric that quantifies the usage of the computing device that performs the video encoding operation;

computing an adjustment factor based on the metric;

redefining the video sampling rate based on the adjustment factor; and

based on the redefined video sampling rate, encoding said composite video comprising said video content of said first and second participants.

2. The method of claim 1, wherein the computing of the adjustment factor comprises using a proportional integral derivative (“

PID”

) operation to compute the adjustment factor based on the received metric.

3. The method of claim 2, wherein the PID operation performs a proportional calculation, an integral calculation, and a derivative calculation, and combines results of said calculations through a set of weighting factors.

4. The method of claim 3, wherein at least one weighting factor in the set of weighting factors is set to zero.

5. The method of claim 1, wherein the computing of the adjustment factor comprises using a non-linear calculation to compute the adjustment factor based on the received metric.

6. The method of claim 1, wherein the metric comprises a system idle of the computing device.

7. The method of claim 6, wherein the system idle is an idle percentage of a processor of the computing device.

8. The method of claim 6, wherein the video sampling rate is not redefined when the system idle is above a certain threshold value.

9. The method of claim 6, wherein the video sampling rate is redefined when the system idle is above a certain threshold value.

10. The method of claim 6, wherein the video sampling rate is redefined when the system idle is below a certain threshold value.

21. The method of claim 1 further comprising encoding the video content at the redefined sampling rate.

22. The method of claim 1, further comprising distributing the video content to a set of non-focus devices in the video conference, said metric quantifying the usage of the computing device.

23. The method of claim 1, wherein said metric comprises the usage of a processor of said computing device that performs the video encoding operation.

11. A computer readable storage medium storing a computer program which when executed by one or more processors adjusts a video sampling rate that is used during a video encoding operation of a multi-participant video conference, the computer program comprising sets of instructions for:

at a computing device of a first participant of said video conference;

defining an initial video sampling rate for determining the rate at which a composite video is generated;

receiving video content of at least said first participant and a second participant of said conference;

receiving a metric that quantifies the usage of the computing device that performs the video encoding operation;

computing an adjustment factor based on the metric;

redefining the video sampling rate based on the adjustment factor; and

based on the redefined video sampling rate, encoding said composite video comprising said video content of said first and second participants.

12. The computer readable storage medium of claim 11, wherein the set of instructions for computing of the adjustment factor comprises a set of instructions for using a proportional integral derivative (“

PID”

) operation to compute the adjustment factor based on the received metric.

13. The computer readable storage medium of claim 12, wherein the PID operation performs a proportional calculation, an integral calculation, and a derivative calculation, and combines results of said calculations through a set of weighting factors.

14. The computer readable storage medium of claim 13, wherein one or more of weighting factors in the set of weighting factors is set to zero.

15. The computer readable storage medium of claim 11, wherein the set of instructions for computing of the adjustment factor comprises a set of instructions for using a non-linear calculation to compute the adjustment factor based on the received metric.

16. The computer readable storage medium of claim 11, wherein the metric comprises a system idle of the computing device.

17. The computer readable storage medium of claim 16, wherein the system idle is an idle percentage of a processor of the computing device.

18. The computer readable storage medium of claim 16, wherein the video sampling rate is not redefined when the system idle is above a certain threshold value.

19. The computer readable storage medium of claim 16, wherein the video sampling rate is redefined when the system idle is above a certain threshold value.

20. The computer readable storage medium of claim 16, wherein the video sampling rate is redefined when the system idle is below a certain threshold value.

24. A method for adjusting a video sampling rate that is used during a video encoding operation of a multi-participant video conference, the method comprising:

at a computing device of a participant, which serves as one of a central distributor and a non-central distributor of video content during the multi-participant video conference;

determining a metric that quantifies processor usage of said computing device that performs the encoding during the video conference;

computing an adjustment factor based on the metric;

adjusting the video sampling rate based on the computed adjustment factor;

when the computing device serves as the central distributor, encoding composite video content for other non-central distributors based on the adjusted sampling rate; and

when the computing device serves as the non-central distributor, encoding video content for another central distributor based on the adjusted sampling rate.

25. The method of claim 24, wherein the computing of the adjustment factor comprises using a proportional integral derivative (“

PID”

) operation to compute the adjustment factor based on the determined metric.

26. A computer-implemented method for encoding frames of a video conference, said method comprising:

at a computing device of a participant, which serves as one of a central distributor and a non-central distributor of video content during the multi-participant video conference;

providing a control system for receiving a usage metric that quantifies the usage of the computing device that performs the encoding during the video conference and computing an adjustment factor based on the usage metric;

providing a frame rate adjuster for receiving the adjustment factor and generating a frame sampling rate; and

providing a video encoding module for receiving the generated frame sampling rate and encoding video of the video conference according to the generated frame sampling rate, wherein the video encoding module encodes the video for other non-central distributors when the computing device serves as the central distributor, and encodes the video for another central distributor when the computing device serves as the non-central distributor.

27. The computer-implemented method of claim 26, wherein said frame rate adjuster is further for receiving a reference input that represents a minimum threshold for said usage metric.

28. The computer-implemented method of claim 26, wherein said usage metric comprises at least one of system idle time and system processing time.

29. The computer-implemented method of claim 26, wherein said control system is farther for computing an error value by calculating the difference between the usage metric and a reference input that represents a minimum threshold for said usage metric.

30. The computer-implemented method of claim 29, wherein said control system is further for computing from said error value at least one of a proportional output value, an integral output value, and a derivative output value.

31. The computer-implemented method of claim 30, wherein said control system computes said adjustment factor by summing the proportional, integral, and derivative output values.

32. The computer-implemented method of claim 30, wherein said usage metric comprises at least one of system idle time and system processing time.

33. The computer-implemented method of claim 26, wherein the computing of the adjustment factor comprises using a proportional integral derivative (“

PID”

) operation to compute the adjustment factor based on said usage metric.