Using a plurality of buffers to provide audio for synchronized playback to multiple audio devices having separate device clocks

• US 10,558,422 B2
• Filed: 04/12/2018
• Issued: 02/11/2020
• Est. Priority Date: 06/25/2004
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Moore, Jeffrey C., Stewart, William George, Lengeling, Gerhard
• Primary Examiner(s): Peyton, Tammara R
• Application Number:
  US15/952,118
• Publication Number:
  US 20180336003A1
• Time in Patent Office: 670 Days
• CPC Class Codes: G06F 3/00 Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
• CPC Class Codes: G06F 3/162 Interface to dedicated audio devices, e.g. audio drivers, interface to CODECs
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G11B 20/10527 Audio or video recording; Data buffering arrangements G11B20/12 - G11B20/18 take precedence
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04N 21/4143 embedded in a Personal Computer [PC]
• CPC Class Codes: H04N 21/43076 of the same content streams on multiple devices, e.g. when family members are watching the same movie on different devices
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 21/8106 involving special audio data, e.g. different tracks for different languages

Abstract

According to one embodiment, a media system communicates with an aggregate device that includes multiple media output devices. When providing media data for presentation, the system adjusts for device clock drift by resampling the media data provided to a media output device based at least in part on a device clock rate difference between a device clock of one of the media output devices and a device clock of another of the media output devices.

Claims

1. A method for writing data for synchronized playback by a first playback device and a second playback device, the method comprising:

writing, by a computing device, data into a first buffer and a second buffer, wherein the first buffer is associated with the first playback device, and wherein the second buffer is associated with the second playback device; and

determining, by the computing device, where to write the data within each of the buffers based, at least in part, on presentation latencies of the first and second playback devices.

2. The method as recited in claim 1, wherein determining where to write the data within each of the buffers further comprises:

determining, by the computing device, a first sum equal to a first safety offset added to a first presentation latency for the first playback device;

determining, by the computing device, a second sum equal to a second safety offset added to a second presentation latency for the second playback device;

determining, by the computing device, a maximum value from amongst the first sum and the second sum;

determining, by the computing device, a first latency offset for the first playback device based on a difference between the first sum and the maximum value; and

determining, by the computing device, a second latency offset for the second playback device based on a difference between the second sum and the maximum value.

3. The method as recited in claim 2, wherein the first safety offset indicates a minimum read ahead amount for writing the data for the first playback device, and wherein the second safety offset indicates a minimum read ahead amount for writing the data for the second playback device.

4. The method as recited in claim 1, wherein the computing device is selected from a group consisting of:

the first playback device and the second playback device.

5. The method as recited in claim 1, wherein each playback device comprises a separate device clock, the method further comprising:

determining, by the computing device, a difference between a first device clock rate of the first playback device and a second device clock rate of the second playback device; and

resampling, by the computing device, the data to be written into the first buffer based at least in part on the difference.

6. The method as recited in claim 5, wherein determining the difference between the first device clock rate and the second device clock rate is based on a first timestamp information generated by the first playback device and a second timestamp information generated by the second playback device.

7. The method as recited in claim 5, wherein resampling the data to be written into the first buffer based at least in part on the difference comprises:

in response to a determination that the first playback device is slower than the second playback device, reducing, by the computing device, a size of the data for at least a segment thereof to produce size-adjusted data;

in response to a determination that the first playback device is faster than the second playback device, increasing, by the computing device, a size of the data for at least a segment thereof to produce the size-adjusted data; and

writing, by the computing device, the size-adjusted data to the first buffer instead of the data.

8. The method as recited in claim 1, further comprising:

upon reaching an end of the first buffer while writing the data to the first buffer, performing, by the computing device, a wrap-around operation to continue writing the data beginning at a start of the first buffer, wherein the first buffer comprises a ring buffer.

9. The method as recited in claim 1, further comprising:

acquiring, by the computing device, the data as a digital data stream via a wireless connection to a network device.

10. A system for writing data for synchronized playback by a first playback device and a second playback device, the system comprising:

at least one processor; and

a non-transitory computer-readable medium storing instructions that cause the at least one processor to perform operations comprising;

writing, by the at least one processor, data into a first buffer and a second buffer, wherein the first buffer is associated with the first playback device, and wherein the second buffer is associated with the second playback device; and

determining, by the at least one processor, where to write the data within each of the buffers based, at least in part, on presentation latencies of the first and second playback devices.

11. The system as recited in claim 10, wherein the instructions that determine where to write the data within each of the buffers further causes the at least one processor to perform operations comprising:

determining, by the at least one processor, a first sum equal to a first safety offset added to a first presentation latency for the first playback device;

determining, by the at least one processor, a second sum equal to a second safety offset added to a second presentation latency for the second playback device;

determining, by the at least one processor, a maximum value from amongst the first sum and the second sum;

determining, by the at least one processor, a first latency offset for the first playback device based on a difference between the first sum and the maximum value; and

determining, by the at least one processor, a second latency offset for the second playback device based on a difference between the second sum and the maximum value.

12. The system as recited in claim 11, wherein the first safety offset indicates a minimum read ahead amount for writing the data for the first playback device, and wherein the second safety offset indicates a minimum read ahead amount for writing the data for the second playback device.

13. The system as recited in claim 10, wherein either the first playback device or the second playback device comprises the at least one processor.

14. The system as recited in claim 10, wherein each playback device comprises a separate device clock, the operations further comprising:

determining, by the at least one processor, a difference between a first device clock rate of the first playback device and a second device clock rate of the second playback device; and

resampling, by the at least one processor, the data to be written into the first buffer based at least in part on the difference.

15. The system as recited in claim 14, wherein the difference between the first device clock rate and the second device clock rate is based on a first timestamp information generated by the first playback device and a second timestamp information generated by the second playback device.

16. The system as recited in claim 14, wherein the instructions that resample the data to be written into the first buffer based at least in part on the difference further causes the at least one processor to perform operations comprising:

in response to a determination that the first playback device is slower than the second playback device, reducing, by the at least one processor, a size of the data for at least a segment thereof to produce size-adjusted data;

in response to a determination that the first playback device is faster than the second playback device, increasing, by the at least one processor, a size of the data for at least a segment thereof to produce the size-adjusted data; and

writing, by the at least one processor, the size-adjusted data to the first buffer instead of the data.

17. The system as recited in claim 10, wherein the operations further comprise:

upon reaching an end of the first buffer while writing the data to the first buffer, performing, by the at least one processor, a wrap-around operation to continue writing the data beginning at a start of the first buffer, wherein the first buffer comprises a ring buffer.

18. The system as recited in claim 10, wherein the operations further comprise:

acquiring, by the at least one processor, the data as a digital data stream via a wireless connection to a network device.

19. A non-transitory computer-readable medium for writing data for synchronized playback by a first playback device and a second playback device, the non-transitory computer-readable medium storing instructions that cause at least one processor to perform operations comprising:

writing, by the at least one processor, data into a first buffer and a second buffer, wherein the first buffer is associated with the first playback device, and wherein the second buffer is associated with the second playback device; and

determining, by the at least one processor, where to write the data within each of the buffers based, at least in part, on presentation latencies of the first and second playback devices.

20. The non-transitory computer-readable medium as recited in claim 19, wherein the instructions that determine where to write the data within each of the buffers further causes the at least one processor to perform operations comprising:

determining, by the at least one processor, a first sum equal to a first safety offset added to a first presentation latency for the first playback device;

determining, by the at least one processor, a second sum equal to a second safety offset added to a second presentation latency for the second playback device;

determining, by the at least one processor, a maximum value from amongst the first sum and the second sum;

determining, by the at least one processor, a first latency offset for the first playback device based on a difference between the first sum and the maximum value; and

determining, by the at least one processor, a second latency offset for the second playback device based on a difference between the second sum and the maximum value.

21. The non-transitory computer-readable medium as recited in claim 20, wherein the first safety offset indicates a minimum read ahead amount for writing the data for the first playback device, and wherein the second safety offset indicates a minimum read ahead amount for writing the data for the second playback device.

22. The non-transitory computer-readable medium as recited in claim 19, wherein either the first playback device or the second playback device comprises the at least one processor.

23. The non-transitory computer-readable medium as recited in claim 19, wherein each playback device comprises a separate device clock, the operations further comprising:

determining, by the at least one processor, a difference between a first device clock rate of the first playback device and a second device clock rate of the second playback device; and

resampling, by the at least one processor, the data to be written into the first buffer based at least in part on the difference.

24. The non-transitory computer-readable medium as recited in claim 23, wherein the difference between the first device clock rate and the second device clock rate is based on a first timestamp information generated by the first playback device and a second timestamp information generated by the second playback device.

25. The non-transitory computer-readable medium as recited in claim 23, wherein the instructions that resample the data to be written into the first buffer based at least in part on the difference further causes the at least one processor to perform operations comprising:

in response to a determination that the first playback device is slower than the second playback device, reducing, by the at least one processor, a size of the data for at least a segment thereof to produce size-adjusted data;

in response to a determination that the first playback device is faster than the second playback device, increasing, by the at least one processor, a size of the data for at least a segment thereof to produce the size-adjusted data; and

writing, by the at least one processor, the size-adjusted data to the first buffer instead of the data.

26. The non-transitory computer-readable medium as recited in claim 19, wherein the operations further comprise:

upon reaching an end of the first buffer while writing the data to the first buffer, performing, by the at least one processor, a wrap-around operation to continue writing the data beginning at a start of the first buffer, wherein the first buffer comprises a ring buffer.

27. The non-transitory computer-readable medium as recited in claim 19, wherein the operations further comprise:

acquiring, by the at least one processor, the data as a digital data stream via a wireless connection to a network device.