Techniques for providing audio for synchronized playback by multiple devices

• US 8,037,220 B2
• Filed: 10/23/2009
• Issued: 10/11/2011
• Est. Priority Date: 06/25/2004
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Moore, Jeffrey C., Lengeling, Gerhard H., Stewart, William G.
• Primary Examiner(s): PEYTON, TAMMARA R
• Application Number:
  US12/605,131
• Publication Number:
  US 20100042238A1
• Time in Patent Office: 718 Days
• Field of Search: 710 8- 10, 710 52- 54
• US Class Current: 710/52
• CPC Class Codes: G06F 3/00 Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
• CPC Class Codes: G06F 3/162 Interface to dedicated audio devices, e.g. audio drivers, interface to CODECs
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G11B 20/10527 Audio or video recording; Data buffering arrangements G11B20/12 - G11B20/18 take precedence
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04N 21/4143 embedded in a Personal Computer [PC]
• CPC Class Codes: H04N 21/43076 of the same content streams on multiple devices, e.g. when family members are watching the same movie on different devices
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 21/8106 involving special audio data, e.g. different tracks for different languages

Abstract

An audio system communicates with an aggregate device that includes multiple audio devices. When providing audio data for playback, the system compensates for presentation latency differences between the various audio devices. In addition, the system adjusts for device clock drift by selecting a master device and resampling the audio data provided to the other devices based on the difference between the device clock of the master device and the device clocks of the other devices.

Claims

1. A method for providing audio data for synchronized playback by a plurality of audio devices, the method comprising:

writing audio data into each of a plurality of buffers;

wherein each buffer of the plurality of buffers is associated with one audio device of the plurality of audio devices; and

determining where to write audio data within one buffer of the plurality of buffers based, at least in part, on presentation latencies of two or more of the plurality of audio devices;

wherein the method is performed by one or more computing devices.

2. The method of claim 1, wherein said plurality of buffers includes a first ring buffer and a second ring buffer, the first ring buffer associated with a first audio device of the plurality of audio devices, the second ring buffer associated with a second audio device of the plurality of audio devices, the method further comprising:

upon reaching the end of said first ring buffer while writing audio data to the first ring buffer, performing a wrap-around operation to continue writing audio data beginning at the start of the first ring buffer; and

determining where to write audio data within said first ring buffer based, at least in part, on (a) a presentation latency between when said first audio device reads audio data from the first ring buffer and when said first audio device plays the audio data read from the first ring buffer and (b) a presentation latency between when said second audio device reads audio data from the second ring buffer and when said second audio device plays the audio data read from the second ring buffer.

3. The method of claim 2, further comprising:

upon the first audio device reaching the end of said first ring buffer while reading audio data from the first ring buffer, the first audio device performing a wrap-around operation to continue reading audio data beginning at the start of the first ring buffer;

upon the second audio device reaching the end of said second ring buffer while reading audio data from the second ring buffer, the second audio device performing a wrap-around operation to continue reading audio data beginning at the start of the second ring buffer;

storing first timestamp information when the first audio device performs a wrap-around operation while reading audio data from the first ring buffer; and

storing second timestamp information when the second audio device performs a wrap-around operation while reading audio data from the second ring buffer.

4. The method of claim 2,

wherein the first audio device is currently reading audio data from a first current position in the first ring buffer;

wherein determining where to write audio data within said first ring buffer includes selecting how far ahead of said first current position to write audio data in said first ring buffer.

5. The method of claim 4, wherein selecting how far ahead of said first current position to write audio data in said first ring buffer includes:

receiving first presentation latency information from said first audio device; and

receiving second presentation latency information from said second audio device.

6. The method of claim 2,

wherein the first audio device is currently reading audio data from a first current position in the first ring buffer;

wherein the second audio device is currently reading audio data from a second current position in the second ring buffer;

receiving first safety offset information from said first audio device, wherein said first safety offset information identifies a minimum amount ahead of said first current position to write audio data in said first ring buffer;

receiving second safety offset information from said second audio device, wherein said second safety offset information identifies a minimum amount ahead of said second current position to write audio data in said second ring buffer; and

based on said first safety offset information and said second safety offset information, selecting how far ahead of said first current position to write audio data in said first ring buffer based on a presentation latency between when the second audio device reads audio data from the second ring buffer and when the second audio device plays the audio data.

7. The method of claim 1,

wherein said plurality of audio devices consists of a first audio speaker device and a second audio speaker device;

and

wherein determining where to write audio data within one of the plurality of buffers comprises determining where to write audio data within a buffer associated with the first audio speaker device based, at least in part, on a presentation latency of said first audio speaker device and a presentation latency of said second audio speaker device.

8. A non-transitory computer-readable medium storing instructions which, when executed by one or more computing devices, cause the one or more computing devices to perform:

writing audio data into each of a plurality of buffers;

wherein each buffer of the plurality of buffers is associated with one audio device of a plurality of audio devices; and

determining where to write audio data within one buffer of the plurality of buffers based, at least in part, on presentation latencies of two or more of the plurality of audio devices.

9. The non-transitory computer-readable medium of claim 8, wherein said plurality of buffers includes a first ring buffer and a second ring buffer, the first ring buffer associated with a first audio device of the plurality of audio devices, the second ring buffer associated with a second audio device of the plurality of audio devices, and wherein the instructions, when executed by the one or more computing devices, cause the one or more computing devices to further perform:

upon reaching the end of said first ring buffer while writing audio data to the first ring buffer, performing a wrap-around operation to continue writing audio data beginning at the start of the first ring buffer; and

determining where to write audio data within said first ring buffer based, at least in part, on (a) a presentation latency between when said first audio device reads audio data from the first ring buffer and when said first audio device plays the audio data read from the first ring buffer and (b) a presentation latency between when said second audio device reads audio data from the second ring buffer and when said second audio device plays the audio data read from the second ring buffer.

10. The non-transitory computer-readable medium of claim 9, wherein the instructions, when executed by the one or more computing devices, cause the one or more computing devices to further perform:

upon the first audio device reaching the end of said first ring buffer while reading audio data from the first ring buffer, the first audio device performing a wrap-around operation to continue reading audio data beginning at the start of the first ring buffer;

upon the second audio device reaching the end of said second ring buffer while reading audio data from the second ring buffer, the second audio device performing a wrap-around operation to continue reading audio data beginning at the start of the second ring buffer;

storing first timestamp information when the first audio device performs a wrap-around operation while reading audio data from the first ring buffer; and

storing second timestamp information when the second audio device performs a wrap-around operation while reading audio data from the second ring buffer.

11. The non-transitory computer-readable medium of claim 9,

wherein the first audio device is currently reading audio data from a first current position in the first ring buffer;

wherein determining where to write audio data within said first ring buffer includes selecting how far ahead of said first current position to write audio data in said first ring buffer.

12. The non-transitory computer-readable medium of claim 11, wherein selecting how far ahead of said first current position to write audio data in said first ring buffer includes:

receiving first presentation latency information from said first audio device; and

receiving second presentation latency information from said second audio device.

13. The non-transitory computer-readable medium of claim 9,

wherein the first audio device is currently reading audio data from a first current position in the first ring buffer;

wherein the second audio device is currently reading audio data from a second current position in the second ring buffer; and

wherein the instructions, when executed by the one or more computing devices, cause the one or more computing devices to further perform;

receiving first safety offset information from said first audio device, wherein said first safety offset information identifies a minimum amount ahead of said first current position to write audio data in said first ring buffer; and

receiving second safety offset information from said second audio device, wherein said second safety offset information identifies a minimum amount ahead of said second current position to write audio data in said second ring buffer

based on said first safety offset information and said second safety offset information, selecting how far ahead of said first current position to write audio data in said first ring buffer based on a presentation latency between when the second audio device reads audio data from the second ring buffer and when the second audio device plays the audio data.

14. The non-transitory computer-readable medium of claim 8,

wherein said plurality of audio devices consists of a first audio speaker device and a second audio speaker device;

and

wherein determining where to write audio data within one of the plurality of buffers comprises determining where to write audio data within a buffer associated with the first audio speaker device based, at least in part, on a presentation latency of said first audio speaker device and a presentation latency of said second audio speaker device.

15. A computer system comprising:

one or more processors;

logic encoded in one or more computer-readable storage media wherein execution by the one or more processors causes;

writing audio data into each of a plurality of buffers;

wherein each buffer of the plurality of buffers is associated with one audio device of the plurality of audio devices; and

determining where to write audio data within one buffer of the plurality of buffers based, at least in part, on presentation latencies of two or more of the plurality of audio devices.

16. The computer system of claim 15, wherein said plurality of buffers includes a first ring buffer and a second ring buffer, the first ring buffer associated with a first audio device of the plurality of audio devices, the second ring buffer associated with a second audio device of the plurality of audio devices, and wherein the logic, when executed by the one or more processors, further causes:

upon reaching the end of said first ring buffer while writing audio data to the first ring buffer, performing a wrap-around operation to continue writing audio data beginning at the start of the first ring buffer; and

determining where to write audio data within said first ring buffer based, at least in part, on (a) a presentation latency between when said first audio device reads audio data from the first ring buffer and when said first audio device plays the audio data read from the first ring buffer and (b) a presentation latency between when said second audio device reads audio data from the second ring buffer and when said second audio device plays the audio data read from the second ring buffer.

17. The computer system of claim 16, wherein the logic, when executed by the one or more processors, further causes:

upon the first audio device reaching the end of said first ring buffer while reading audio data from the first ring buffer, the first audio device performing a wrap-around operation to continue reading audio data beginning at the start of the first ring buffer;

upon the second audio device reaching the end of said second ring buffer while reading audio data from the second ring buffer, the second audio device performing a wrap-around operation to continue reading audio data beginning at the start of the second ring buffer;

storing first timestamp information when the first audio device performs a wrap-around operation while reading audio data from the first ring buffer; and

storing second timestamp information when the second audio device performs a wrap-around operation while reading audio data from the second ring buffer.

18. The computer system of claim 16,

wherein the first audio device is currently reading audio data from a first current position in the first ring buffer;

wherein determining where to write audio data within said first ring buffer includes selecting how far ahead of said first current position to write audio data in said first ring buffer.

19. The computer system of claim 18, wherein selecting how far ahead of said first current position to write audio data in said first ring buffer includes:

receiving first presentation latency information from said first audio device; and

receiving second presentation latency information from said second audio device.

20. The computer system of claim 16,

wherein the first audio device is currently reading audio data from a first current position in the first ring buffer;

wherein the second audio device is currently reading audio data from a second current position in the second ring buffer; and

wherein the logic, when executed by the one or more processors, further causes;

receiving first safety offset information from said first audio device, wherein said first safety offset information identifies a minimum amount ahead of said first current position to write audio data in said first ring buffer; and

receiving second safety offset information from said second audio device, wherein said second safety offset information identifies a minimum amount ahead of said second current position to write audio data in said second ring buffer

based on said first safety offset information and said second safety offset information, selecting how far ahead of said first current position to write audio data in said first ring buffer based on a presentation latency between when the second audio device reads audio data from the second ring buffer and when the second audio device plays the audio data.

21. The computer system of claim 15,

wherein said plurality of audio devices consists of a first audio speaker device and a second audio speaker device;

and

wherein determining where to write audio data within one of the plurality of buffers comprises determining where to write audio data within a buffer associated with the first audio speaker device based, at least in part, on a presentation latency of said first audio speaker device and a presentation latency of said second audio speaker device.

22. A method for providing audio data for synchronized playback by a plurality of audio devices, the method comprising:

writing audio data into a first buffer of a plurality of buffers, said first buffer associated with a first audio device of said plurality of audio devices;

writing audio data into a second buffer of the plurality of buffers, said second buffer associated with a second audio device of said plurality of audio devices;

wherein said first audio device and said second audio device are different audio devices;

wherein said first buffer and said second buffer are different buffers;

determining where to write audio data within said first buffer based, at least in part, on a presentation latency of said first audio device and a presentation latency of said second audio device; and

determining where to write audio data within said second buffer based, at least in part, on a presentation latency of said first audio device and a presentation latency of said second audio device.