System having a plurality of buffers for providing audio for synchronized playback to multiple audio devices

• US 8,185,674 B2
• Filed: 10/23/2009
• Issued: 05/22/2012
• Est. Priority Date: 06/25/2004
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Moore, Jeffrey C., Stewart, William G., Lengeling, Gerhard H.
• Primary Examiner(s): PEYTON, TAMMARA R
• Application Number:
  US12/605,137
• Publication Number:
  US 20100042239A1
• Time in Patent Office: 942 Days
• Field of Search: 710 8- 10, 710 52- 54, 348/515, 713/400, 713/500
• US Class Current: 710/52
• CPC Class Codes: G06F 3/00 Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
• CPC Class Codes: G06F 3/162 Interface to dedicated audio devices, e.g. audio drivers, interface to CODECs
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G11B 20/10527 Audio or video recording; Data buffering arrangements G11B20/12 - G11B20/18 take precedence
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04N 21/4143 embedded in a Personal Computer [PC]
• CPC Class Codes: H04N 21/43076 of the same content streams on multiple devices, e.g. when family members are watching the same movie on different devices
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 21/8106 involving special audio data, e.g. different tracks for different languages

Abstract

An audio system communicates with an aggregate device that includes multiple audio devices. When providing audio data for playback, the system compensates for presentation latency differences between the various audio devices. In addition, the system adjusts for device clock drift by selecting a master device and resampling the audio data provided to the other devices based on the difference between the device clock of the master device and the device clocks of the other devices.

Claims

1. A method for providing audio for playback by multiple devices, the method comprising:

writing audio data into each of a plurality of buffers for consumption by a plurality of audio devices;

wherein each device of the plurality of devices consumes audio data from a corresponding buffer of the plurality of buffers;

resampling audio data to be written into a buffer of the plurality of buffers based, at least in part, on a difference between a device clock rate of the device that corresponds to the buffer and a device clock rate of another device of the plurality of devices;

wherein the method is performed by one or more computing devices.

2. The method of claim 1, wherein said plurality of buffers is a plurality of ring buffers including a first ring buffer for a first device and a second ring buffer for a second device, the method further comprising:

writing audio data into each of the plurality of ring buffers;

upon reaching the end of any ring buffer while writing audio data to the ring buffer, performing a wrap-around operation to continue writing audio data beginning at the start of the ring buffer;

upon reaching the end of any ring buffer while reading audio data from the ring buffer, the corresponding device performing a wrap-around operation to continue reading audio data beginning at the start of the ring buffer;

the first device currently reading audio data from a first current position in the first ring buffer;

the second device currently reading audio data from a second current position in the second ring buffer;

storing first timestamp information when the first device performs a wrap-around operation while reading audio data from the first buffer;

storing second timestamp information when the second device performs a wrap-around operation while reading audio data from the second buffer;

determining a timing difference between the first device and the second device based, at least in part, on the first timestamp information and the second timestamp information; and

resampling audio data to be written to the second ring buffer based on the timing difference.

3. The method of claim 1, further comprising:

selecting a device from the plurality of devices as a master device based, at least in part, on presentation latency information received from said plurality of devices;

determining a timing difference between said master device and any device of said plurality of devices; and

resampling audio data to be written to any buffer of said plurality of buffers based on said timing difference.

4. The method of claim 1, wherein resampling audio data to be written to said buffer further comprises increasing the rate at which audio data is written to said buffer.

5. The method of claim 1, wherein resampling audio data to be written to said buffer further comprises decreasing the rate at which audio data is written to said buffer.

6. A non-transitory computer-readable medium storing instructions which, when executed by one or more computing devices, cause the one or more computing devices to perform:

writing audio data into each of a plurality of buffers for consumption by a plurality of audio devices;

wherein each device of the plurality of devices consumes audio data from a corresponding buffer of the plurality of buffers;

resampling audio data to be written into a buffer of the plurality of buffers based, at least in part, on a difference between a device clock rate of the device that corresponds to the buffer and a device clock rate of another device of the plurality of devices.

7. The non-transitory computer-readable medium of claim 6, wherein said plurality of buffers is a plurality of ring buffers including a first ring buffer for a first device and a second ring buffer for a second device, wherein the instructions, when executed by the one or more computing devices, cause the one or more computing devices to further perform:

writing audio data into each of the plurality of ring buffers;

upon reaching the end of any ring buffer while writing audio data to the ring buffer, performing a wrap-around operation to continue writing audio data beginning at the start of the ring buffer;

upon reaching the end of any ring buffer while reading audio data from the ring buffer, the corresponding device performing a wrap-around operation to continue reading audio data beginning at the start of the ring buffer;

the first device currently reading audio data from a first current position in the first ring buffer;

the second device currently reading audio data from a second current position in the second ring buffer;

storing first timestamp information when the first device performs a wrap-around operation while reading audio data from the first buffer;

storing second timestamp information when the second device performs a wrap-around operation while reading audio data from the second buffer;

determining a timing difference between the first device and the second device based, at least in part, on the first timestamp information and the second timestamp information; and

resampling audio data to be written to the second ring buffer based on the timing difference.

8. The non-transitory computer-readable medium of claim 6, wherein the instructions, when executed by the one or more computing devices, cause the one or more computing devices to further perform:

selecting a device from the plurality of devices as a master device based, at least in part, on presentation latency information received from said plurality of devices;

determining a timing difference between said master device and any device of said plurality of devices; and

resampling audio data to be written to any buffer of said plurality of buffers based on said timing difference.

9. The non-transitory computer-readable medium of claim 6, wherein resampling audio data to be written to said buffer further comprises increasing the rate at which audio data is written to said buffer.

10. The non-transitory computer-readable medium of claim 6, wherein resampling audio data to be written to said buffer further comprises decreasing the rate at which audio data is written to said buffer.

11. A computer system comprising:

one or more processors;

logic encoded in one or more computer-readable storage media wherein execution by the one or more processors causes;

writing audio data into each of a plurality of buffers for consumption by a plurality of audio devices;

wherein each device of the plurality of devices consumes audio data from a corresponding buffer of the plurality of buffers;

resampling audio data to be written into a buffer of the plurality of buffers based, at least in part, on a difference between a device clock rate of the device that corresponds to the buffer and a device clock rate of another device of the plurality of devices.

12. The computer system of claim 11, wherein said plurality of buffers is a plurality of ring buffers including a first ring buffer for a first device and a second ring buffer for a second device, wherein the logic, when executed by the one or more processors, further causes:

writing audio data into each of the plurality of ring buffers;

upon reaching the end of any ring buffer while writing audio data to the ring buffer, performing a wrap-around operation to continue writing audio data beginning at the start of the ring buffer;

upon reaching the end of any ring buffer while reading audio data from the ring buffer, the corresponding device performing a wrap-around operation to continue reading audio data beginning at the start of the ring buffer;

the first device currently reading audio data from a first current position in the first ring buffer;

the second device currently reading audio data from a second current position in the second ring buffer;

storing first timestamp information when the first device performs a wrap-around operation while reading audio data from the first buffer;

storing second timestamp information when the second device performs a wrap-around operation while reading audio data from the second buffer;

determining a timing difference between the first device and the second device based, at least in part, on the first timestamp information and the second timestamp information; and

resampling audio data to be written to the second ring buffer based on the timing difference.

13. The computer system of claim 11, wherein the logic, when executed by the one or more processors, further causes:

selecting a device from the plurality of devices as a master device based, at least in part, on presentation latency information received from said plurality of devices;

determining a timing difference between said master device and any device of said plurality of devices; and

resampling audio data to be written to any buffer of said plurality of buffers based on said timing difference.

14. The computer system of claim 11, wherein resampling audio data to be written to said buffer further comprises increasing the rate at which audio data is written to said buffer.

15. The computer system of claim 11, wherein resampling audio data to be written to said buffer further comprises decreasing the rate at which audio data is written to said buffer.

16. A system comprising:

a plurality of buffers;

a plurality of audio devices, wherein each audio device of the plurality of audio devices is operatively coupled to one buffer of the plurality of buffers, wherein each audio device of the plurality of audio devices is associated with a device clock;

a mechanism operatively coupled to said plurality of buffers and configured to resample audio data to be written into a buffer of the plurality of buffers based, at least in part, on a difference between a device clock rate of the audio device that corresponds to the buffer and a device clock rate of another device of the plurality of audio devices.

17. The system of claim 16,

wherein said plurality of buffers is a plurality of ring buffers;

wherein a first audio device of said plurality of audio devices is operatively coupled to a first ring buffer of the plurality of ring buffers;

wherein said first audio device is configured to;

upon reaching the end of the first ring buffer while reading audio data from the first ring buffer, perform a wrap-around operation to continue reading audio data beginning at the start of the first ring buffer; and

store first timestamp information when the first audio device performs a wrap-around operation while reading audio data from the first ring buffer;

wherein a second audio device of said plurality of audio devices is operatively coupled to a second ring buffer of the plurality of ring buffers;

wherein said second audio device is configured to;

upon reaching the end of the second ring buffer while reading audio data from the second ring buffer, perform a wrap-around operation to continue reading audio data beginning at the start of the second ring buffer; and

store second timestamp information when the second audio device performs a wrap-around operation while reading audio data from the second buffer;

wherein said mechanism is further configured to;

write audio data into each of the plurality of ring buffers;

upon reaching the end of any ring buffer while writing audio data to the ring buffer, perform a wrap-around operation to continue writing audio data beginning at the start of the ring buffer;

determine a timing difference between the first audio device and the second audio device based, at least in part, on the first timestamp information and the second timestamp information; and

resample audio data to be written to the second ring buffer based on the timing difference.

18. The system of claim 16, wherein said mechanism is further configured to:

select a device from the plurality of audio devices as a master device based, at least in part, on presentation latency information received from said plurality of audio devices;

determine a timing difference between said master device and any audio device of said plurality of audio devices; and

resample audio data to be written to any buffer of said plurality of buffers based on said timing difference.

19. The system of claim 16, wherein the mechanism is further configured to resample audio data to be written to said buffer by increasing the rate at which audio data is written to said buffer.

20. The system of claim 16, wherein the mechanism is further configured to resample audio data to be written to said buffer by decreasing the rate at which audio data is written to said buffer.