Using a plurality of buffers to provide audio for synchronized playback to multiple audio devices having separate device clocks

• US 9,977,648 B2
• Filed: 01/03/2017
• Issued: 05/22/2018
• Est. Priority Date: 06/25/2004
• Status: Expired due to Term

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Moore, Jeffrey C., Stewart, William George, Lengeling, Gerhard
• Primary Examiner(s): Peyton, Tammara
• Application Number:
  US15/397,558
• Publication Number:
  US 20170185374A1
• Time in Patent Office: 504 Days
• CPC Class Codes: G06F 3/00 Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
• CPC Class Codes: G06F 3/162 Interface to dedicated audio devices, e.g. audio drivers, interface to CODECs
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G11B 20/10527 Audio or video recording; Data buffering arrangements G11B20/12 - G11B20/18 take precedence
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04N 21/4143 embedded in a Personal Computer [PC]
• CPC Class Codes: H04N 21/43076 of the same content streams on multiple devices, e.g. when family members are watching the same movie on different devices
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 21/8106 involving special audio data, e.g. different tracks for different languages

Abstract

According to one embodiment, a media system communicates with an aggregate device that includes multiple media output devices. When providing media data for presentation, the system adjusts for device clock drift by resampling the media data provided to a media output device based at least in part on a device clock rate difference between a device clock of one of the media output devices and a device clock of another of the media output devices.

Claims

1. A method comprising:

receiving, by a computing device, presentation latency data of a plurality of playback devices, the presentation latency data indicating a respective amount of presentation latency at each playback device when playing content;

writing, by the computing device, content data into a plurality of buffers, each buffer corresponding to a respective playback device, wherein writing the content data into the buffers comprises;

determining a respective offset for each playback device based on the presentation latency data, the offset compensating for the respective amount of presentation latency at the corresponding playback device; and

writing the content data into each buffer at a respective position corresponding to the respective offset; and

causing, by the computing device, the playback devices to play the content in synchronization by reading the content data from the respective buffers.

2. The method of claim 1, wherein:

the content includes audio,

each playback device includes at least one speaker,

each buffer is a ring buffer, and

each presentation latency is a difference between a time a corresponding playback device reads the ring buffer and a time the corresponding playback device plays the audio.

3. The method of claim 2, wherein each offset is an offset from a respective read position of each buffer, the read position being a position in a corresponding buffer at which a respective playback device reads data.

4. The method of claim 3, wherein each read position is estimated by the computing device based on a respective timestamp generated by a corresponding playback device when the playback device reaches an end of a corresponding ring buffer during reading.

5. The method of claim 1, wherein:

each offset includes a respective safety offset and a respective latency offset,

the respective safety offset corresponds to characteristics of a respective playback device, and

the respective latency offset corresponds to a difference between a composite latency including sum of the respective safety offset and the respective presentation latency and a maximum composite latency among the playback devices.

6. The method of claim 5, wherein:

the computing device designates a particular playback device of the playback devices that has the maximum composite latency as a master device, and

the computing device changes master devices dynamically during playback and, upon changing the master device to a new master device, determines new offsets based on the new master device.

7. The method of claim 1, wherein:

the computing device includes a system-side audio handling input/output unit, and

each playback device includes a respective device-side audio handling input/output unit.

8. The method of claim 1, wherein each offset is a temporal offset or a size offset.

9. A system comprising:

a computing device; and

a non-transitory computer-readable medium storing instructions operable to cause the computing device to perform operations comprising;

receiving, by the computing device, presentation latency data of a plurality of playback devices, the presentation latency data indicating a respective amount of presentation latency at each playback device when playing content;

writing, by the computing device, content data into a plurality of buffers, each buffer corresponding to a respective playback device, wherein writing the content data into the buffers comprises;

determining a respective offset for each playback device based on the presentation latency data, the offset compensating for the respective amount of presentation latency at the corresponding playback device; and

writing the content data into each buffer at a respective position corresponding to the respective offset; and

causing, by the computing device, the playback devices to play the content in synchronization by reading the content data from the respective buffers.

10. The system of claim 9, wherein:

the content includes audio,

each playback device includes at least one speaker,

each buffer is a ring buffer, and

each presentation latency is a difference between a time a corresponding playback device reads the ring buffer and a time the corresponding playback device plays the audio.

11. The system of claim 10, wherein each offset is an offset from a respective read position of each buffer, the read position being a position in a corresponding buffer at which a respective playback device reads data.

12. The system of claim 11, wherein each read position is estimated by the computing device based on a respective timestamp generated by a corresponding playback device when the playback device reaches an end of a corresponding ring buffer during reading.

13. The system of claim 9, wherein:

each offset includes a respective safety offset and a respective latency offset,

the respective safety offset corresponds to characteristics of a respective playback device, and

the respective latency offset corresponds to a difference between a composite latency including sum of the respective safety offset and the respective presentation latency and a maximum composite latency among the playback devices.

14. The system of claim 13, wherein:

the computing device designates a particular playback device of the playback devices that has the maximum composite latency as a master device, and

the computing device changes master devices dynamically during playback and, upon changing the master device to a new master device, determines new offsets based on the new master device.

15. A non-transitory computer-readable medium storing instructions operable to cause a computing device to perform operations comprising:

receiving, by the computing device, presentation latency data of a plurality of playback devices, the presentation latency data indicating a respective amount of presentation latency at each playback device when playing content;

writing, by the computing device, content data into a plurality of buffers, each buffer corresponding to a respective playback device, wherein writing the content data into the buffers comprises;

determining a respective offset for each playback device based on the presentation latency data, the offset compensating for the respective amount of presentation latency at the corresponding playback device; and

writing the content data into each buffer at a respective position corresponding to the respective offset; and

causing, by the computing device, the playback devices to play the content in synchronization by reading the content data from the respective buffers.

16. The non-transitory computer-readable medium of claim 15, wherein:

the content includes audio,

each playback device includes at least one speaker,

each buffer is a ring buffer, and

each presentation latency is a difference between a time a corresponding playback device reads the ring buffer and a time the corresponding playback device plays the audio.

17. The non-transitory computer-readable medium of claim 16, wherein each offset is an offset from a respective read position of each buffer, the read position being a position in a corresponding buffer at which a respective playback device reads data.

18. The non-transitory computer-readable medium of claim 17, wherein each read position is estimated by the computing device based on a respective timestamp generated by a corresponding playback device when the playback device reaches an end of a corresponding ring buffer during reading.

19. The non-transitory computer-readable medium of claim 15, wherein:

each offset includes a respective safety offset and a respective latency offset,

the respective safety offset corresponds to characteristics of a respective playback device, and

the respective latency offset corresponds to a difference between a composite latency including sum of the respective safety offset and the respective presentation latency and a maximum composite latency among the playback devices.

20. The non-transitory computer-readable medium of claim 19, wherein:

the computing device designates a particular playback device of the playback devices that has the maximum composite latency as a master device, and

the computing device changes master devices dynamically during playback and, upon changing the master device to a new master device, determines new offsets based on the new master device.