Schedule-based coordination of audio sources

• US 10,241,748 B2
• Filed: 08/15/2017
• Issued: 03/26/2019
• Est. Priority Date: 12/13/2016
• Status: Active Grant

• Current Assignee: B & W Group Ltd (Masimo Corporation)
• Original Assignee: EVA Automation, Inc.
• Inventors: Yu, Gaylord, Stupp, Steven
• Primary Examiner(s): Elbin, Jesse A
• Application Number:
  US15/678,048
• Publication Number:
  US 20180165055A1
• Time in Patent Office: 588 Days
• Field of Search: 700 94
• CPC Class Codes: G06F 1/12 Synchronisation of different clock signals provided by a plurality of clock generators
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/60 Network streaming of media packets
• CPC Class Codes: H04L 7/02 Speed or phase control by the received code signals, the signals containing no special synchronisation information H04L7/0075 takes precedence
• CPC Class Codes: H04N 21/4305 Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/8106 involving special audio data, e.g. different tracks for different languages
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content
• CPC Class Codes: H04R 2227/005 Audio distribution systems for home, i.e. multi-room use
• CPC Class Codes: H04R 2420/07 Applications of wireless loudspeakers or wireless microphones
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F
• CPC Class Codes: H04R 3/12 for distributing signals to two or more loudspeakers specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04S 7/301 Automatic calibration of stereophonic sound system, e.g. with test microphone
• CPC Class Codes: H04S 7/307 Frequency adjustment, e.g. tone control H04S7/301 takes precedence
• CPC Class Codes: H04W 4/023 using mutual or relative location information between multiple location based services [LBS] targets or of distance thresholds
• CPC Class Codes: H04W 56/005 compensating for timing error by adjustment in the receiver

Abstract

An audio/video (A/V) hub that coordinates playback of audio content is described. In particular, the A/V hub may calculate current time offsets between clocks in electronic devices and a clock in the A/V hub based on differences between receive times when frames are received from electronic devices and expected transmit times of the frames. For example, the expected transmit times may be based on coordination of clocks in the electronic devices and a clock in the A/V hub at a previous time and a predefined transmit schedule of the frames. Then, the A/V hub may transmit, to the electronic devices, one or more frames that include audio content and playback timing information, which may specify playback times when the electronic devices are to playback the audio content based on the current time offsets.

Claims

1. A coordination device, comprising:

one or more nodes configured to communicatively couple to one or more antennas;

an interface circuit communicatively coupled to the one or more nodes, wherein the coordination device is configured to;

receive, from the one or more nodes, input frames associated with electronic devices;

store receive times when the input frames were received, wherein the receive times are based on a clock in the coordination device;

calculate current time offsets between clocks in the electronic devices and the clock in the coordination device based on the receive times and expected transmit times of the input frames, wherein the expected transmit times are based on coordination of the clocks in the electronic devices and the clock in the coordination device at a previous time and a predefined transmit schedule of the input frames, and wherein the calculation of a given current time offset is based on unilateral communication of timing information between a given electronic device and the coordination device; and

transmit, via the one or more nodes, one or more output frames that comprise audio content and playback timing information intended for the electronic devices, wherein the playback timing information specifies playback times when the electronic devices are to concurrently playback the audio content based on the current time offsets, and

wherein the playback times of the electronic devices have a temporal relationship so that the playback of the audio content by the electronic devices is coordinated.

2. The coordination device of claim 1, wherein the temporal relationship has a non-zero value, so that at least some of the electronic devices are instructed to playback the audio content with a phase relative to each other by using different values of the playback times.

3. The coordination device of claim 2, wherein the different playback times are based on acoustic characterization of an environment.

4. The coordination device of claim 2, wherein the different playback times are based on a desired acoustic characteristic in an environment.

5. The coordination device of claim 2, wherein the electronic devices are located at vector distances from the coordination device;

wherein the interface circuit is configured to determine magnitudes of the vector distances based on the expected transmit times of the input frames and the receive times using wireless ranging, and is configured to determine angles of the vector distances based on the angle of arrival of wireless signals associated with the input frames; and

wherein the different playback times are based on the determined vector distances.

6. The coordination device of claim 2, wherein the different playback times are based on an estimated location of a listener relative to the electronic devices.

7. The coordination device of claim 6, wherein the interface circuit is configured to:

communicate, to or from the one or more nodes, second frames associated with another electronic device; and

calculate the estimated location of the listener based on the communication with the other electronic device.

8. The coordination device of claim 6, wherein the coordination device further comprises an acoustic transducer configured to perform sound measurements of an environment;

and

wherein the coordination device is configured to calculate the estimated location of the listener based on the sound measurements.

9. The coordination device of claim 6, wherein the interface circuit is configured to communicate, to or from the one or more nodes, third frames associated with other electronic devices in the environment and to receive, from the one or more nodes, additional sound measurements, associated with the other electronic devices, of the environment;

and

wherein the coordination device is configured to calculate the estimated location of the listener based on the additional sound measurements.

10. The coordination device of claim 6, wherein the interface circuit is configured to:

perform time-of-flight measurements; and

calculate the estimated location of the listener based on the time-of-flight measurements.

11. The coordination device of claim 1, wherein the coordination of the clocks in the electronic devices and the clock in the coordination device occurred during an initialization mode of operation.

12. The coordination device of claim 1, wherein the current time offsets are further based on models of clock drift in the electronic devices.

13. A non-transitory computer-readable storage medium for use with a coordination device, the computer-readable storage medium storing a program module that, when executed by the coordination device, causes the coordination device to coordinate playback of audio content by carrying out one or more operations that comprise:

receiving, from one or more nodes in the coordination device that are communicatively coupled to one or more antennas, input frames associated with electronic devices;

storing receive times when the input frames were received, wherein the receive times are based on a clock in the coordination device;

calculating current time offsets between clocks in the electronic devices and the clock in the coordination device based on the receive times and expected transmit times of the input frames, wherein the expected transmit times are based on coordination of the clocks in the electronic devices and the clock in the coordination device at a previous time and a predefined transmit schedule of the input frames, and wherein the calculation of a given current time offset is based on unilateral communication of timing information between a given electronic device and the coordination device; and

transmitting, via the one or more nodes, one or more output frames that comprise the audio content and playback timing information intended for the electronic devices, wherein the playback timing information specifies playback times when the electronic devices are to concurrently playback the audio content based on the current time offsets, and

wherein the playback times of the electronic devices have a temporal relationship so that the playback of the audio content by the electronic devices is coordinated.

14. The non-transitory computer-readable storage medium of claim 13, wherein the temporal relationship has a non-zero value, so that at least some of the electronic devices are instructed to playback the audio content with a phase relative to each other by using different values of the playback times;

and

wherein the different playback times are based on acoustic characterization of an environment.

15. The non-transitory computer-readable storage medium of claim 14, wherein the temporal relationship has a non-zero value, so that at least some of the electronic devices are instructed to playback the audio content with a phase relative to each other by using different values of the playback times;

and

wherein the different playback times are based on a desired acoustic characteristic in the environment.

16. The non-transitory computer-readable storage medium of claim 14, wherein the temporal relationship has a non-zero value, so that at least some of the electronic devices are instructed to playback the audio content with a phase relative to each other by using different values of the playback times;

wherein the electronic devices are located at vector distances from the coordination device;

wherein the one or more operations comprise determining magnitudes of the vector distances based on the expected transmit times of the input frames and the receive times using wireless ranging, and determining angles of the vector distances based on the angle of arrival of wireless signals associated with the input frames; and

wherein the different playback times are based on the determined vector distances.

17. The non-transitory computer-readable storage medium of claim 14, wherein the temporal relationship has a non-zero value, so that at least some of the electronic devices are instructed to playback the audio content with a phase relative to each other by using different values of the playback times;

and

wherein the different playback times are based on an estimated location of a listener relative to the electronic devices.

18. The non-transitory computer-readable storage medium of claim 17, wherein the one or more operations comprise:

performing time-of-flight measurements; and

calculating the estimated location of the listener based on the time-of-flight measurements.

19. The non-transitory computer-readable storage medium of claim 13, wherein the current time offsets are further based on models of clock drift in the electronic devices.

20. A method for coordinating playback of audio content, wherein the method comprises:

by a coordination device;

receiving, from one or more nodes in the coordination device that are communicatively coupled to one or more antennas, input frames associated with electronic devices;

storing receive times when the input frames were received, wherein the receive times are based on a clock in the coordination device;

calculating current time offsets between clocks in the electronic devices and the clock in the coordination device based on the receive times and expected transmit times of the input frames, wherein the expected transmit times are based on coordination of the clocks in the electronic devices and the clock in the coordination device at a previous time and a predefined transmit schedule of the input frames, and wherein the calculation of a given current time offset is based on unilateral communication of timing information between a given electronic device and the coordination device; and

transmitting, via the one or more nodes, one or more output frames that comprise audio content and playback timing information intended for the electronic devices, wherein the playback timing information specifies playback times when the electronic devices are to concurrently playback the audio content based on the current time offsets, and

wherein the playback times of the electronic devices have a temporal relationship so that the playback of the audio content by the electronic devices is coordinated.