Monitoring audio fidelity and audio-video synchronization

• US 10,305,763 B2
• Filed: 07/21/2014
• Issued: 05/28/2019
• Est. Priority Date: 11/09/2010
• Status: Active Grant

• Current Assignee: Vmware LLC (Broadcom, Inc.)
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: Spracklen, Lawrence Andrew
• Primary Examiner(s): Nguyen, Duc
• Assistant Examiner(s): Eljaiek, Alexander L
• Application Number:
  US14/336,835
• Publication Number:
  US 20140328203A1
• Time in Patent Office: 1,772 Days
• Field of Search: 709224, 709231, 709223, 709203, 709217, 709219, 709230, 709232, 713400, 700 94
• CPC Class Codes: H04L 43/08 Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
• CPC Class Codes: H04L 7/043 Pseudo-noise [PN] codes variable during transmission synchronisation of spread spectrum receivers H04B1/69
• CPC Class Codes: H04N 17/004 for digital television systems
• CPC Class Codes: H04R 2227/003 Digital PA systems using, e.g. LAN or internet
• CPC Class Codes: H04R 2227/005 Audio distribution systems for home, i.e. multi-room use
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F

Abstract

Methods, systems, and computer programs for monitoring quality of audio delivered over a communications channel are presented. One method includes an operation for defining timestamps. The timestamps are associated with a measure of time while delivering audio to a client computer, where each timestamp includes a plurality of timestamp bits. Further, the method includes an operation for modulating an audio signal with pseudo noise (PN) codes when a timestamp bit has a first logical value, and modulating the audio signal with a negative of the PN codes when the timestamp bit has a second logical value. After transmitting the modulated audio signal to the client computer, the timestamp bits are extracted from a received modulated audio signal to obtain received timestamps. The quality of the audio is assessed based on the received timestamps, and the quality of the audio is stored in computer memory.

Claims

1. A method comprising:

determining, by one or more computers, a sequence of two or more timestamps, the timestamps being associated with a measure of time progress while delivering audio to a client computer, each timestamp in the sequence of two or more timestamps including a plurality of timestamp bits;

modulating an audio signal with the sequence of two or more timestamps, wherein modulating the audio signal for each timestamp comprises;

modulating a carrier signal for each timestamp bit of the timestamp with an instance of a k-bit encoding of length k that spreads the encoding over a plurality of audio samples, wherein each instance of the k-bit encoding is a positive version that positively correlates with the k-bit encoding or negative version that negatively correlates with the k-bit encoding based on the binary value of that particular timestamp bit, and

adding the modulated carrier signal to the audio signal;

transmitting the modulated audio signal to a client computer; and

determining a quality value of the audio based on the two or more timestamps received by the client computer, wherein the two or more timestamps are obtained using the timestamp bits extracted by the client computer from the received modulated audio signal.

2. The method of claim 1, wherein the modulation of the audio signal is a positive value of pseudo noise (PN) codes when a timestamp bit has a first logical value and is a negative value of the PN codes when the timestamp bit has a second logical value.

3. The method of claim 2, wherein modulating the audio signal comprises:

modulating a digital carrier with the PN codes or with the negative of the PN codes to obtain a digital modulated PN signal; and

combining the digital modulated PN signal with a digital form of the audio signal to obtain the modulated audio signal.

4. The method of claim 1, wherein each timestamp includes a counter and a marker, the marker indicative of a location of the timestamp, the counter indicative of a place of each timestamp within the sequence of two or more timestamps.

5. The method of claim 4, wherein the extraction of the timestamp bits by the client computer includes:

detecting the marker; and

determining a value of the counter based on the detecting of the marker.

6. The method of claim 1, comprising:

inserting video markers in a video signal associated with the audio signal;

transmitting the video signal with the modulated audio signal;

receiving information associated with the reception of the video markers;

correlating a reception of timestamps in the modulated audio signal and a reception of video markers in the transmitted video signal; and

determining a synchronization between the audio signal and the video signal based on the correlating.

7. The method of claim 1, wherein determining the quality value further includes:

comparing time progress of the received modulated audio signal with progress of the received timestamps; and

determining the quality value based on a synchronism between the received modulated audio signal and the progress of the received timestamps.

8. A method comprising:

receiving a modulated audio signal at a client computer, wherein the modulated audio signal is modulated with a sequence of two or more timestamps, the timestamps being associated with a measure of time progress while delivering audio to the client computer, each timestamp including a plurality of timestamp bits, wherein the audio signal is modulated for each timestamp bit with an instance of a k-bit encoding of length k that spreads the encoding over a plurality of audio samples, wherein each instance of the k-bit encoding is a positive version that positively correlates with the k-bit encoding or negative version that negatively correlates with the k-bit encoding based on the binary value of that particular timestamp bit;

extracting the timestamp bits from the received modulated audio signal to obtain two or more received timestamps; and

determining a quality value of the audio based on the two or more received timestamps.

9. The method of claim 8, wherein the modulated audio signal is a positive value of pseudo noise (PN) codes when a timestamp bit has a first logical value and is a negative value of the PN codes when the timestamp bit has a second logical value.

10. The method of claim 9, wherein extracting the timestamp bits for each timestamp comprises:

detecting a correlation on the received modulated audio signal to determine if the positive value of the PN code or the negative value of the PN code is received, wherein detecting a positive correlation with the PN code is interpreted as a timestamp bit with the first logical value and detecting a negative correlation with the PN code is interpreted as a timestamp bit with the second logical value.

11. The method of claim 9, wherein determining the quality value of the audio comprises determining the quality value of the audio based on the received timestamps and a quality of a received PN code in the modulated audio signal.

12. The method of claim 8, comprising:

receiving a video signal associated with the received modulated audio signal, the video signal including a plurality of video markers; and

determining a synchronization between the received audio signal and the video signal based on a correlation between the two or more received timestamps and the plurality of video markers.

13. The method of claim 8, further comprising:

providing the quality value to a server.

14. The method of claim 8, wherein each timestamp includes a counter and a marker, the marker indicative of a location of the timestamp, the counter indicative of a place of each timestamp within the sequence of two or more timestamps, and wherein extracting the timestamp bits further includes:

detecting the marker; and

determining a value of the counter based on the detecting of the marker.

15. The method of claim 14, wherein the marker is a logical value including a plurality of logical 1 bits, wherein the counter begins and ends with a logical 0 bit.

16. The method of claim 8, wherein determining the quality value further includes:

comparing time progress of the received modulated audio signal with progress of the received timestamps; and

determining the quality value based on a synchronism between the received modulated audio signal and the progress of the received timestamps.

17. The method of claim 8, wherein determining the quality value further includes measuring a longest sequence of consecutive timestamps extracted during a period of time.

18. A method comprising:

receiving a modulated audio signal at a client computer, wherein the modulated audio signal is modulated with a sequence of two or more timestamps, the timestamps being associated with a measure of time progress while delivering audio to the client computer, each timestamp including a plurality of timestamp bits, wherein the audio signal is modulated for each timestamp bit with an instance of a k-bit encoding of length k that spreads the encoding over a plurality of audio samples, wherein each instance of the k-bit encoding is a positive version that positively correlates with the k-bit encoding or negative version that negatively correlates with the k-bit encoding based on the binary value of that particular timestamp bit;

receiving a video signal associated with the received modulated audio signal, the video signal including a plurality of video markers;

extracting the timestamp bits from the received modulated audio signal to obtain two or more received timestamps, wherein extracting the timestamp bits for each timestamp include;

detecting a correlation on the received modulated audio signal to determine if the positive value of the k-bit encoding or the negative value of the k-bit encoding is received, wherein determining a positive correlation with the k-bit encoding is interpreted as a timestamp bit with the first logical value and determining a negative correlation with the k-bit encoding is interpreted as a timestamp bit with the second logical value; and

determining a synchronization between the received audio signal and the video signal based on a correlation between the two or more received timestamps and the plurality of video markers.

19. One or more non-transitory computer-readable storage media encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:

determining, by one or more computers, a sequence of two or more timestamps, the timestamps being associated with a measure of time progress while delivering audio to a client computer, each timestamp in the sequence of two or more timestamps including a plurality of timestamp bits;

modulating an audio signal with the sequence of two or more timestamps, wherein modulating the audio signal for each timestamp comprises;

modulating a carrier signal for each timestamp bit of the timestamp with an instance of a k-bit encoding of length k that spreads the encoding over a plurality of audio samples, wherein each instance of the k-bit encoding is a positive version that positively correlates with the k-bit encoding or negative version that negatively correlates with the k-bit encoding based on the binary value of that particular timestamp bit, and

adding the modulated carrier signal to the audio signal;

transmitting the modulated audio signal to a client computer; and

determining a quality value of the audio based on the two or more timestamps received by the client computer, wherein the two or more timestamps are obtained using the timestamp bits extracted by the client computer from the received modulated audio signal.

20. One or more non-transitory computer-readable storage media encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:

receiving a modulated audio signal at a client computer, wherein the modulated audio signal is modulated with a sequence of two or more timestamps, the timestamps being associated with a measure of time progress while delivering audio to the client computer, each timestamp including a plurality of timestamp bits, wherein the audio signal is modulated for each timestamp bit with an instance of a k-bit encoding of length k that spreads the encoding over a plurality of audio samples, wherein each instance of the k-bit encoding is a positive version that positively correlates with the k-bit encoding or negative version that negatively correlates with the k-bit encoding based on the binary value of that particular timestamp bit;

extracting the timestamp bits from the received modulated audio signal to obtain two or more received timestamps; and

determining a quality value of the audio based on the two or more received timestamps.

21. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

determining, by one or more computers, a sequence of two or more timestamps, the timestamps being associated with a measure of time progress while delivering audio to a client computer, each timestamp in the sequence of two or more timestamps including a plurality of timestamp bits;

modulating an audio signal with the sequence of two or more timestamps, wherein modulating the audio signal for each timestamp comprises;

modulating a carrier signal for each timestamp bit of the timestamp with an instance of a k-bit encoding of length k that spreads the encoding over a plurality of audio samples, wherein each instance of the k-bit encoding is a positive version that positively correlates with the k-bit encoding or negative version that negatively correlates with the k-bit encoding based on the binary value of that particular timestamp bit, and

adding the modulated carrier signal to the audio signal;

transmitting the modulated audio signal to a client computer; and

determining a quality value of the audio based on the two or more timestamps received by the client computer, wherein the two or more timestamps are obtained using the timestamp bits extracted by the client computer from the received modulated audio signal.

22. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving a modulated audio signal at a client computer, wherein the modulated audio signal is modulated with a sequence of two or more timestamps, the timestamps being associated with a measure of time progress while delivering audio to the client computer, each timestamp including a plurality of timestamp bits, wherein the audio signal is modulated for each timestamp bit with an instance of a k-bit encoding of length k that spreads the encoding over a plurality of audio samples, wherein each instance of the k-bit encoding is a positive version that positively correlates with the k-bit encoding or negative version that negatively correlates with the k-bit encoding based on the binary value of that particular timestamp bit;

extracting the timestamp bits from the received modulated audio signal to obtain two or more received timestamps; and

determining a quality value of the audio based on the two or more received timestamps.