Encoding an audio stream

• US 10,332,534 B2
• Filed: 01/07/2016
• Issued: 06/25/2019
• Est. Priority Date: 01/07/2016
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Srinivasan, Sriram, Lee, Ming-Chieh
• Primary Examiner(s): Adesanya, Olujimi A
• Application Number:
  US14/990,658
• Publication Number:
  US 20170200454A1
• Time in Patent Office: 1,265 Days
• Field of Search: 704500-504
• CPC Class Codes: G06F 9/5038 considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration scheduling strategies G06F9/4881 and subgroups
• CPC Class Codes: G10L 19/002 Dynamic bit allocation for perceptual audio coders G10L19/032
• CPC Class Codes: G10L 19/167 Audio streaming, i.e. formatting and decoding of an encoded audio signal representation into a data stream for transmission or storage purposes
• CPC Class Codes: G10L 19/22 Mode decision, i.e. based on audio signal content versus external parameters
• CPC Class Codes: G10L 19/24 Variable rate codecs, e.g. for generating different qualities using a scalable representation such as hierarchical encoding or layered encoding
• CPC Class Codes: H04L 43/0888 Throughput
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 65/61 for supporting one-way streaming services, e.g. Internet radio
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 65/80 Responding to QoS

Abstract

An audio stream is encoded for transmission to a receiving device via a communications channel. The to-be transmitted audio stream is received at an audio encoder executed on a processor. The processor has an amount of available processing resources. An available bandwidth of the communications channel is determined. Based on the determined bandwidth, a portion of the available processing resources is allocated to the audio encoder. The allocated portion is greater if the determined bandwidth is below a bandwidth threshold. The audio encoder encodes the audio stream using the allocated portion of processing resources, and transmits the encoded audio stream to the receiving device via the communications channel.

Claims

1. A method of encoding an audio stream comprising:

receiving, at an audio encoder executed on a processor, an audio stream to be transmitted to a receiving device via a communications channel, the processor having an amount of available processing resources;

determining an available bandwidth of the communications channel;

comparing the determined bandwidth to a bandwidth threshold;

determining a measure of processor load that corresponds to the amount of available processing resources;

allocating a portion of the available processing resources to the audio encoder based on the determined bandwidth of the communications channel and the determined processor load,

wherein the allocated portion of the available processing resources is greater if the determined bandwidth is below the bandwidth threshold than if the determined bandwidth is above the bandwidth threshold, and

wherein the portion of the available processing resources allocated to the audio encoder is greater if the determined bandwidth is below the bandwidth threshold and the processor load is below a processor load threshold than if the determined bandwidth is below the bandwidth threshold and the processor load is above the processor load threshold;

encoding, by the audio encoder, the audio stream using the allocated portion of processing resources; and

transmitting the encoded audio stream for receipt by the receiving device via the communications channel;

wherein multiple audio encoders are executed on the processor, wherein each of the audio encoders is allocated a respective portion of the available processing resources which that audio encoder uses to encode a respective audio stream for transmission for receipt by a respective receiving device via a respective communications channel; and

wherein the allocating includes, in response to the processor load increasing to the processor load threshold, identifying a subset of the audio encoders whose respective communications channels have the lowest available bandwidths, wherein a greater portion of the available processing resources is allocated to each of the identified audio encoders than is allocated to any of the other audio encoders.

2. A method according to claim 1, wherein the allocating step is performed by selecting based on the determined bandwidth one of a plurality of encoding modes of the audio encoder having different complexities, and configuring the encoder to operate in the selected encoding mode to encode the audio stream;

wherein a higher complexity one of the encoding modes requiring the greater portion of the available processing resources is selected if the determined bandwidth is below the bandwidth threshold and the processor load is below the processor load threshold.

3. A method according to claim 1, wherein in response to an increase in the determined bandwidth occurring after said allocation, the allocated portion of the processing resources is reduced.

4. A method according to claim 2, wherein:

in response to an increase in the determined bandwidth occurring after said allocation, the allocated portion of the processing resources is reduced; and

the portion of the available processing resources allocated to the audio encoder is reduced by switching, in response to the increase in the determined bandwidth, the audio encoder to a lower complexity one of the encoding modes requiring only the reduced portion of the available processing resources.

5. A method according to claim 3, wherein if the determined bandwidth subsequently increases from a value below the bandwidth threshold, the portion of the processing resources allocated to the audio encoder is reduced when the increasing bandwidth reaches a different bandwidth threshold.

6. A method according to claim 5, wherein the different bandwidth threshold is higher than the bandwidth threshold.

7. A method according to claim 3, wherein the audio encoder is part of a communication client executed on the processor and at least some of the available processing resources rendered available by said reduction in the allocated portion are reallocated to another software module of the communications client executed on the processor.

8. A method according to claim 1, wherein the portion of the processing resources allocated to the audio encoder is changed at a rate no greater than a maximum allocation change rate that is independent of the available bandwidth.

9. A method according to claim 1, wherein the portion of the available processing resources is initially allocated to the audio encoder independently of the available bandwidth when the audio stream is initially received at the audio encoder, and then increased in response to a subsequent determination that the available bandwidth is below the bandwidth threshold.

10. A method according to claim 1, further comprising:

encoding the audio stream at an output bitrate that is adapted in response to changes in the available bandwidth of the communications channel.

11. A method according to claim 10, wherein the changed bandwidth is used to adapt the output bitrate.

12. A method according to claim 10, wherein in response to an increase in the available bandwidth, the output bitrate is increased and the portion of the available processing resources allocated to the audio encoder is reduced.

13. A method according to claim 1, wherein the audio encoders implement a plurality of encoding modes comprising a higher complexity encoding mode that requires the greater portion of the available processing resources and a lower complexity encoding mode that requires a lesser portion of the available processing resources, further comprising:

for one or more initial audio streams being encoded;

determining that the determined bandwidth is below the bandwidth threshold and the processor load is below the processor load threshold; and

encoding the one or more initial audio streams using the high complexity encoding mode; and

for a subsequent audio stream being encoded;

determining that the determined bandwidth is below the bandwidth threshold and the processor load is above the processor load threshold; and

encoding the subsequent audio stream using the low complexity encoding mode.

14. A method according to claim 13, wherein the one or more initial audio streams and the subsequent audio stream are part of a same conference call.

15. A computer system comprising:

a network interface for communicating with a receiving device via a communications channel;

a bandwidth determiner configured to;

determine an available bandwidth of the communications channel; and

determining a measure of processor load that corresponds to an amount of available processing resources;

an audio encoder executed on a processor of the computer system and configured to receive an audio stream to be transmitted to the receiving device via the communications channel, the processor having the amount of available processing resources; and

a resource allocator configured to compare the determined bandwidth to a bandwidth threshold, and to allocate, to the audio encoder based on the determined bandwidth of the communications channel and the determined processor load, a portion of the available processing resources, wherein the allocated portion of the available processing resources is greater if the determined bandwidth is below the bandwidth threshold than if the determined bandwidth is above the bandwidth threshold, wherein the portion of the available processing resources allocated to the audio encoder is greater if the determined bandwidth is below the bandwidth threshold and the processor load is below a processor load threshold than if the determined bandwidth is below the bandwidth threshold and the processor load is above the processor load threshold, wherein the audio encoder is configured to encode the audio stream using the allocated portion of processing resources, and wherein the network interface is configured to transmit the encoded audio stream for receipt by the receiving device via the communications channel;

wherein multiple audio encoders are executed on the processor, wherein each of the audio encoders is allocated a respective portion of the available processing resources which that audio encoder uses to encode a respective audio stream for transmission for receipt by a respective receiving device via a respective communications channel; and

wherein the resource allocator is further configured to, in response to the processor load increasing to the processor load threshold, identifying a subset of the audio encoders whose respective communications channels have the lowest available bandwidths, wherein a greater portion of the available processing resources is allocated to each of the identified audio encoders than is allocated to any of the other audio encoders.

16. A computer system according to claim 15, wherein the computing system comprises one or more of a server or a user device.

17. A computer readable storage device storing instructions that are executable by a processing system to perform operations including:

determining an available bandwidth of a communications channel via which an audio stream is to be transmitted to a receiving device;

comparing the determined bandwidth to a bandwidth threshold;

determining a measure of processor load that corresponds to an amount of available processing resources; and

allocating a portion of the available processing resources to an audio encoder based on the determined bandwidth of the communications channel and the determined processor load, thereby causing the audio encoder to encode the audio stream using the allocated portion of the processing resources for transmission to the receiving device via the communications channel,

wherein the allocated portion is greater if the determined bandwidth is below the bandwidth threshold than if the determined bandwidth is above the bandwidth threshold, and

wherein the portion of the available processing resources allocated to the audio encoder is greater if the determined bandwidth is below the bandwidth threshold and the processor load is below a processor load threshold than if the determined bandwidth is below the bandwidth threshold and the processor load is above the processor load threshold;

wherein multiple audio encoders are executed on the processor, wherein each of the audio encoders is allocated a respective portion of the available processing resources which that audio encoder uses to encode a respective audio stream for transmission for receipt by a respective receiving device via a respective communications channel; and

wherein the allocating includes, in response to the processor load increasing to the processor load threshold, identifying a subset of the audio encoders whose respective communications channels have the lowest available bandwidths, wherein a greater portion of the available processing resources is allocated to each of the identified audio encoders than is allocated to any of the other audio encoders.