System for real time audio processing driven by interrupt and timer event

• US 9,396,735 B2
• Filed: 01/03/2013
• Issued: 07/19/2016
• Est. Priority Date: 01/03/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Wang, Zhonglei
• Primary Examiner(s): Saint Cyr, Leonard
• Application Number:
  US13/733,728
• Publication Number:
  US 20150199971A1
• Time in Patent Office: 1,293 Days
• Field of Search: 704500-504
• US Class Current: 1/1
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 9/46 Multiprogramming arrangements
• CPC Class Codes: G10L 19/16 Vocoder architecture

Abstract

Provided are methods and systems for processing audio using a combination of interrupts and timer events. Buffer queues and timer events are configured and utilized between audio capture and encoding processes, and also between audio decoding and playout processes to deliver a fixed-rate audio packet and higher quality audio output. Thread execution priorities may be assigned to the capture and/or playout timer events, and such priorities may be automatically adjusted to conserve processing resources. By configuring timer events to drive audio encoding and decoding processes, and utilizing buffer queues to absorb variances in the timing of audio capture and playout interrupts, the methods and systems provided are able to deliver improved audio quality while conserving processing resources.

Claims

1. A computer-implemented method for processing audio data, the method comprising:

generating a capture interrupt event;

in response to the capture interrupt event, copying audio data input from a capture device into an audio buffer configured to store the audio data according to one or more specified parameters;

in response to a capture timer event, calculating an amount of audio data stored in the audio buffer, wherein the capture timer event is associated with a thread execution priority for encoding the audio data stored in the audio buffer;

comparing the amount of audio data stored in the audio buffer with a detected latency of an operating system associated with the capture device;

in response to the comparison indicating that the amount of audio data stored in the audio buffer satisfies a threshold, adjusting the thread execution priority for encoding the audio data stored in the audio buffer; and

providing the audio data stored in the audio buffer to an encoder based on the thread execution priority or the adjusted thread execution priority.

2. The method of claim 1, further comprising:

encoding the audio data stored in the audio buffer; and

transmitting the encoded audio data over a network to a receiver.

3. The computer-implemented method of claim 1, further comprising, in response to the capture timer event, detecting the latency of the operating system.

4. The method of claim 1, further comprising:

creating the capture timer event; and

assigning the thread execution priority to the capture timer event.

5. The method of claim 1, wherein the capture timer event is executed at 10 millisecond intervals.

6. A computer-implemented method for processing audio data, the method comprising:

in response to a playout timer event, calculating an amount of audio data stored in an audio buffer configured for storing audio data for payout according to one or more specified parameters, wherein the playout timer event is associated with a thread execution priority for decoding audio data received over a network;

comparing the amount of audio data stored in the audio buffer with a detected latency of an operating system associated with the audio buffer;

in response to the comparison indicating that the amount of audio data stored in the audio buffer satisfies a threshold, adjusting the thread execution priority for decoding audio data received over the network; and

transmitting to a decoder for decoding the audio data received over the network according to the thread execution priority or the adjusted thread execution priority.

7. The method of claim 6, further comprising:

storing the decoded audio data in the audio buffer; and

in response to the playout interrupt event, sending the audio data stored in the audio buffer to a playout device.

8. The method of claim 6, further comprising:

creating the playout timer event; and

assigning the thread execution priority to the playout timer event.

9. The computer-implemented method of claim 6, further comprising, in response to the playout timer event, detecting the latency of the operating system.

10. The method of claim 6, wherein the playout timer event is executed at 10 millisecond intervals.

11. A system for processing audio data, the system comprising:

at least one processor; and

a non-transitory computer-readable medium coupled to the at least one processor having instructions stored thereon that, when executed by the at least one processor, causes the at least one processor to;

generate a capture interrupt event;

in response to the capture interrupt event, copy audio data input from a capture device into an audio buffer configured to store the audio data according to one or more specified parameters;

in response to a capture timer event, calculate an amount of audio data stored in the audio buffer, wherein the capture timer event is associated with a thread execution priority for encoding the audio data stored in the audio buffer;

compare the amount of audio data stored in the audio buffer with a detected latency of an operating system associated with the capture device;

in response to the comparison indicating that the amount of audio data stored in the audio buffer satisfies a threshold, adjust the thread execution priority for encoding the audio data stored in the audio buffer; and

transmit to an encoder for encoding the audio data stored in the audio buffer according to the thread execution priority or the adjusted thread execution priority.

12. The system of claim 11, wherein the at least one processor is further caused to:

transmit the encoded audio data over a network to a receiver.

13. The system of claim 11, wherein the at least one processor is further caused to:

in response to the capture timer event, detect the latency of the operating system.

14. The system of claim 11, wherein the at least one processor is further caused to:

create the capture timer event; and

assign the thread execution priority to the capture timer event.

15. The system of claim 11, wherein the capture timer event is executed at 10 millisecond intervals.

16. A system for processing audio data, the system comprising:

at least one processor; and

a non-transitory computer-readable medium coupled to the at least one processor having instructions stored thereon that, when executed by the at least one processor, causes the at least one processor to;

in response to a payout timer event, calculate an amount of audio data stored in an audio buffer configured for storing audio data for playout according to one or more specified parameters, wherein the playout timer event is associated with a thread execution priority for decoding audio data received over a network;

compare the amount of audio data stored in the audio buffer with a detected latency of an operating system associated with the audio buffer;

in response to the comparison indicating that the amount of audio data stored in the audio buffer satisfies a threshold, adjust the thread execution priority for decoding audio data received over the network; and

transmit to a decoder for decoding the audio data received over the network according to the thread execution priority or the adjusted thread execution priority.

17. The system of claim 16, wherein the at least one processor is further caused to:

store the decoded audio data in the audio buffer; and

in response to a playout interrupt event, send the audio data stored in the audio buffer to a playout device.

18. The system of claim 16, wherein the at least one processor is further caused to:

create the playout timer event; and

assign the thread execution priority to the playout timer event.

19. The system of claim 16, wherein the at least one processor is further caused to:

in response to the playout timer event, detect the latency of the operating system.

20. The system of claim 16, wherein the playout timer event is executed at 10 millisecond intervals.