Encoded audio synchronization

• US 10,454,604 B2
• Filed: 10/02/2015
• Issued: 10/22/2019
• Est. Priority Date: 10/02/2015
• Status: Active Grant

• Current Assignee: Bose Corporation
• Original Assignee: Bose Corporation
• Inventors: Elliot, Michael W., Banerjee, Debasmit, Sundaresan, Rameshwar
• Primary Examiner(s): Nguyen, Duc
• Assistant Examiner(s): Eljaiek, Alexander L
• Application Number:
  US14/874,124
• Publication Number:
  US 20170098466A1
• Time in Patent Office: 1,481 Days
• Field of Search: 370328, 370329, 370338, 370350, 370471, 370503, 370508, 370509, 370510, 370512, 370514, 381 2, 381 59, 381 77, 381 79, 381 80, 455 412, 455 413, 455 306, 700 94, 704E19048, 709208, 710106
• CPC Class Codes: H04J 3/0641 Change of the master or reference, e.g. take-over or failure of the master
• CPC Class Codes: H04J 3/0661 using timestamps
• CPC Class Codes: H04R 2227/003 Digital PA systems using, e.g. LAN or internet
• CPC Class Codes: H04R 2227/005 Audio distribution systems for home, i.e. multi-room use
• CPC Class Codes: H04R 2420/07 Applications of wireless loudspeakers or wireless microphones
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F

Abstract

A method of synchronizing playback of audio over a plurality of wireless speaker packages that are each adapted to play audio, where the audio comprises a plurality of frames of encoded audio data. The method includes determining an offset time of a frame of encoded audio data, where the offset time corresponds to the time when the frame playback is to begin, and inserting the determined offset time into the frame of encoded audio data.

Claims

1. A method of synchronizing playback of audio over a plurality of wireless speaker packages that are each adapted to play audio, where the audio comprises an audio track or stream that comprises a plurality of frames of encoded audio data, wherein one of the plurality of wireless speaker packages is a master playback device that receives encoded audio data from an audio source and distributes the encoded audio data to the rest of the plurality of wireless speaker packages, which function as slave playback devices, wherein each of the wireless speaker packages comprises an encoded data buffer that stores frames of encoded audio data, and wherein each of the wireless speaker packages comprises a decoder that decodes encoded data that is stored in the encoded data buffer of the respective wireless speaker package and a decoded data buffer that stores frames of decoded audio data, and wherein the master playback device and the slave playback devices have clocks that are synchronized, the method comprising:

the master playback device determining and providing to the slave playback devices a play at time for audio received from an audio source, wherein the play at time is provided to the slave playback devices separately from the audio data and is provided only once per track or stream;

using the decoder of the master playback device to determine an offset time for each of a plurality of frames of encoded audio data, where the offset time defines the time after the play at time when the playback of the respective frame is to begin;

using the decoder of the master playback device to determine a checksum on the decoded audio samples or the quantity of audio samples in the frame for each frame of encoded audio data stored in the encoded data buffer of the master playback device;

inserting the determined offset time into the respective frame of encoded audio data;

storing in the encoded data buffer of the master playback device the encoded frames along with the inserted determined offset times and the determined checksum on the decoded audio samples or the quantity of audio samples for the respective frames;

distributing frames from the master encoded data buffer to each of the slave playback devices;

wherein the slave playback devices also determine the checksum on the decoded audio samples or the quantity of audio samples in the frame, for each frame of encoded audio data they receive from the master playback device;

wherein the slave playback devices synchronize playback of audio with the master playback device based at least in part on a match between the determined checksum on the decoded audio samples or the quantity of audio samples in the frame determined by the master and that was stored in the master encoded data buffer along with the respective frame, and the checksum on the decoded audio samples or the quantity of audio samples in the frame as determined by the slave playback device; and

wherein the slave playback devices synchronize playback of audio with the master playback device beginning by playing a first frame when the play at time plus the offset for that first frame equals the current synchronized clock time;

wherein a joining slave playback device is configured to join an ongoing playback of a track or stream by the master playback device, by;

the joining slave playback device pulling from the encoded data buffer of the master playback device a plurality of encoded frames along with the inserted determined offset times and the determined checksums or the quantity of audio samples for each of the plurality of pulled encoded frames; and

placing the decoders of the joining slave playback device and the master playback device in the same state, wherein the state is based on a match between the master playback device and the joining slave playback device on the checksum on the audio samples or the quantity of audio samples in the plurality of pulled encoded frames.

2. The method of claim 1 wherein at least some of the frames stored in the encoded data buffer of the master playback device comprise frame headers, and the determined offset times of the respective frames are inserted into the frame headers before the frames are distributed to the slave playback devices.

3. The method of claim 2 wherein after synchronizing the playback of audio, the frames that are distributed by the master playback device to the synchronized slave playback device do not include offset times.

4. The method of claim 2 wherein the decoded data buffers of each of the wireless speaker packages store offset times along with the frames of decoded audio data.

6. The method of claim 1, wherein the respective clocks on the plurality of wireless speaker packages are all synchronized to a common reference clock.

7. The method of claim 1, wherein each wireless speaker package further comprises an asynchronous sample rate converter that is adapted to manipulate a duration of played audio data to maintain synchronization among all of the playback devices by ensuring that each playback device outputs audio at the same rate.

8. The method of claim 7, wherein the asynchronous sample rate converter provides audio data to audio hardware at a constant sample rate.

9. The method of claim 8, wherein decoded audio samples are fed to the asynchronous sample rate converter and wherein the offset time in a frame is used by the slave audio playback devices to determine when the corresponding frame is fed to the asynchronous sample rate converter.

13. The method of claim 1, wherein each frame that is received from the audio source by the master playback device comprises a source time that indicates when the frame was sent by the source, and wherein the master playback device adds to each frame before it is stored in a data buffer an arrival time when the frame arrived at the master playback device.

14. The method of claim 13, wherein the offset time for each frame determined by the master playback device is based on a comparison of the arrival time and the source time.

5. A method of synchronizing playback of audio over a master wireless speaker package and a slave wireless speaker package that are each adapted to play audio, where the audio comprises an audio track or stream that comprises a plurality of frames of encoded audio data, where the master wireless speaker package plays frames of decoded audio data that each comprise a frame header that includes a checksum on decoded audio samples or the quantity of audio samples in the frame, and where the slave wireless speaker package receives from the master wireless speaker package encoded frames that include in the frame header the checksum on decoded audio samples or the quantity of audio samples in the frame, and wherein the master wireless speaker package and the slave wireless speaker package have clocks that are synchronized to a common reference clock, the method comprising:

the slave wireless speaker package receiving from the master wireless speaker package a play at time for audio received from an audio source, wherein the play at time is provided to the slave wireless speaker package separately from the audio data and is provided only once per track or stream;

the slave wireless speaker package receiving from the master wireless speaker package frames of audio data that each comprise an offset time that defines the time after the play at time when the playback of the respective frame is to begin;

the slave wireless speaker package determining the checksum on decoded audio samples or the quantity of audio samples in a frame that it received from the master wireless speaker package;

the slave wireless speaker package beginning playback based at least in part on the slave wireless speaker package determining a match between the checksum on decoded audio samples or the quantity of audio samples in the frame determined by the master, and the checksum on decoded audio samples or the quantity of audio samples in the frame as determined by the slave; and

wherein the slave wireless speaker package synchronizes playback of audio beginning by playing a first frame when the play at time plus the offset for that first frame equals the current synchronized clock time;

wherein a joining slave playback device is configured to join an ongoing playback of a track or stream by the master playback device, by;

the joining slave playback device pulling from the encoded data buffer of the master playback device a plurality of encoded frames along with the inserted determined offset times and the determined checksums or the quantity of audio samples for each of the plurality of pulled encoded frames; and

placing the decoders of the joining slave playback device and the master playback device in the same state, wherein the state is based on a match between the master playback device and the joining slave playback device on the checksum on the audio samples or the quantity of audio samples in the plurality of pulled encoded frames.

10. The method of claim 5, wherein each wireless speaker package further comprises an asynchronous sample rate converter that is adapted to manipulate a duration of played audio data to maintain synchronization among all of the wireless speaker packages by ensuring that each wireless speaker package outputs audio at the same rate.

11. The method of claim 10, wherein the asynchronous sample rate converter provides audio data to audio hardware at a constant sample rate.

12. The method of claim 11, wherein decoded audio samples are fed to the asynchronous sample rate converter and wherein the offset time in a frame is used by the slave wireless speaker package to determine when the corresponding frame is fed to the asynchronous sample rate converter.