Synchronized audio/video decoding for network devices

• US 8,238,376 B2
• Filed: 03/20/2006
• Issued: 08/07/2012
• Est. Priority Date: 04/13/2005
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Iwamura, Ryuichi
• Primary Examiner(s): Ly, Anh-Vu H
• Assistant Examiner(s): KHIRODHAR, MAHARISHI V
• Application Number:
  US11/386,526
• Publication Number:
  US 20060233203A1
• Time in Patent Office: 2,332 Days
• Field of Search: 370/503, 370/509, 370/512
• US Class Current: 370/503
• CPC Class Codes: H04N 21/2368 Multiplexing of audio and video streams
• CPC Class Codes: H04N 21/4305 Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/4341 Demultiplexing of audio and video streams
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805

Abstract

A method of synchronizing decoders within a network to a server includes receiving a set of timestamps and local clock signals upon receiving the beacon interrupt signal, computing differential timestamp and local clock values based on values of timestamp and local clock signals, respectively, within the sets of timestamp and local clock signals, determining whether the differential local clock value has a predetermined relationship with the differential timestamp value, and transmitting a clock rate adjustment command signal to the decoder when differential local clock value does not have the predetermined relationship with the differential timestamp value. The clock rate adjustment command signal adjusts the local system time clock of the decoder such that a subsequent differential clock value will have the predetermined relationship with the differential timestamp value. When this method is performed for each decoder within the network, the decoders are substantially synchronized and the decoding delay can be kept below humanly perceptible levels.

Claims

1. A method of synchronizing a plurality of decoders within a network to a server by way of a processor coupled to a network interface and to the plurality of decoders, comprising:

providing the processor coupled to the network interface and to the plurality of decoders;

receiving a set of timestamp signals, each timestamp signal within the set of timestamp signals having a value corresponding to a timestamped beacon transmitted by a network server, the network server for streaming encoded data over a network through the network interface;

receiving a set of local clock signals, each local clock signal within the set of local clock signals having a value generated by a local system time clock associated with a plurality of decoders of a network client, the plurality of decoders for decoding the encoded data;

computing a differential timestamp value based on values of timestamp signals within the set of timestamp signals by the processor;

computing a differential local clock value based on values of local clock signals within the set of local clock signals;

determining whether the differential local clock value has a predetermined relationship with respect to the differential timestamp value, wherein the predetermined relationship exists when the differential local clock value and the differential timestamp value are equal or within a predefined tolerance;

transmitting a clock rate adjustment command signal to the plurality of decoders when it is determined that the differential local clock value does not have the predetermined relationship with the differential timestamp value, the clock rate adjustment command signal for adjusting the local system time clock such that a subsequent differential local clock value will have the predetermined relationship with the differential timestamp value;

when the predetermined relationship exists, preventing sending unneeded clock rate adjustment command signals to the plurality of decoders; and

adjusting a buffer occupancy of the plurality of decoders, wherein adjusting the buffer occupancy comprises;

comparing a current buffer occupancy at each of the plurality of decoders to a predefined threshold to determine whether a relationship exists between the current buffer occupancy and the predefined threshold; and

adjusting the buffer occupancy when it is determined that the relationship exists;

thereby contemporaneously performing a clock rate adjustment while the plurality of decoders is decoding encoded data, and

wherein a decoding delay between any two decoders of the plurality of decoders is maintained in a range of less than 10 milliseconds.

2. The method of claim 1, further comprising:

receiving a beacon interrupt signal from a network interface associated with the network client, the beacon interrupt signal generated by the network interface upon receipt of a beacon transmitted by the network server, wherein

each timestamp signal of the set of timestamp signals is received from the network interface and has a value corresponding to a beacon interrupt signal.

3. The method of claim 1, further comprising:

receiving a beacon interrupt signal from a network interface associated with the network client, the beacon interrupt signal generated by the network interface upon receipt of a beacon transmitted by the network server, wherein

each local clock signal of the set of local clock signals is received from the local system time clock and has a value corresponding to a beacon interrupt signal.

4. The method of claim 1, further comprising:

receiving the sets of timestamp and local clock signals within a period during which a predetermined number of beacons are output by the network server; and

transmitting the clock rate adjustment command signal to the decoder before a beacon is output subsequent to the period.

5. The method of claim 1, wherein the clock rate adjustment command signal increases the frequency of the local system time clock when the differential local clock value is less than the differential timestamp value.

6. The method of claim 1, wherein the clock rate adjustment command signal decreases the frequency of the local system time clock when the differential local clock value is greater than the differential timestamp value.

7. The method of claim 1, further comprising:

receiving a system time clock preset command signal; and

transmitting the system time clock preset command signal to the local system time clock, the system time clock preset command signal setting the local system time clock to a specific preset value upon transmission of a beacon by the network server.

8. The method of claim 1, further comprising:

receiving a decode start signal after transmitting the clock rate adjustment command signal; and

transmitting the decode start signal to the decoder, wherein decode start signal causes the decoder to decode the encoded data upon transmission of a beacon by the network server.

9. The method of claim 1, wherein:

determining whether a relationship exists between the current buffer occupancy and a predefined threshold comprises determining whether a buffer occupancy of a local buffer of each decoder of the plurality of decoders is within a predetermined range of a total buffer occupancy as each decoder of the plurality of decoders decodes encoded data within the local buffer; and

adjusting the buffer occupancy when it is determined that the relationship exists comprises transmitting a data stream control command signal to the network server when it is determined that

the buffer occupancy is outside the predetermined range, the data stream control command signal altering a then-current data streaming characteristic of the network server.

10. The method of claim 9, wherein the data stream control command signal causes the network server to stop streaming data when it is determined that the buffer occupancy exceeds an upper threshold of the predetermined range.

11. The method of claim 9, wherein the data stream control command signal causes the network server to resume streaming data when it is determined that the buffer occupancy exceeds a lower threshold of the predetermined range.

12. The method of claim 1, wherein:

determining whether a relationship exists between the current buffer occupancy and a predefined threshold comprises;

determining whether a buffer occupancy of a local buffer of each decoder of the plurality of decoders is within a predetermined range of a total buffer occupancy as each decoder of the plurality of decoders decodes encoded data within the local buffer; and

wherein adjusting the buffer occupancy when it is determined that the relationship exists comprises;

transmitting a clock rate control command signal to the network server when it is determined that the buffer occupancy is outside the predetermined range;

receiving a clock ratio adjustment signal corresponding to the clock rate control command signal, the clock ratio adjustment signal containing a clock adjustment ratio;

computing the clock rate adjustment command signal based on the clock adjustment ratio; and

transmitting the computed clock rate adjustment command signal to each decoder of the plurality of decoders to adjust the local system time clock such that a subsequent differential local clock value is within the predetermined range of the total buffer occupancy.

13. The method of claim 12, wherein the clock rate adjustment

command signal increases the frequency of the local system time clock when it is determined that the buffer occupancy exceeds an upper threshold of the predetermined range.

14. The method of claim 12, wherein the clock rate adjustment command signal decreases the frequency of the local system time clock when it is determined that the buffer occupancy exceeds a lower threshold of the predetermined range.

19. The method of claim 1, wherein the beacon transmission is approximately 40 milliseconds before a second beacon transmission.

21. The method of claim 1, wherein the range of decoding delay comprises less than approximately 1 millisecond.

15. A network device, comprising:

a network interface for receiving encoded data transmitted over a network;

a plurality of decoders, coupled to the network interface, for decoding the received encoded data; and

a processor coupled to the network interface and the plurality of decoders, the processor containing circuitry for;

receiving a set of timestamp signals via the network interface, wherein each timestamp signal within the set of timestamp signals has a value corresponding to a timestamped beacon transmitted by a network server;

receiving a set of local clock signals, wherein each local clock signal within the set of local clock signals has a value generated by a local system time clock associated with the plurality of decoders;

computing a differential timestamp value based on values of timestamp signals within the set of timestamp signals;

computing a differential local clock value based on values of local clock signals within the set of local clock signals;

determining whether the differential local clock value has a predetermined relationship with respect to the differential timestamp value, wherein the predetermined relationship exists when the differential local clock value and the differential timestamp value are equal or within a predefined tolerance;

transmitting a clock rate adjustment command signal to the plurality of decoders when it is determined that the differential local clock value does not have the predetermined relationship with the differential timestamp value, wherein the clock rate adjustment command signal adjusts the local

system time clock such that a subsequent differential local clock value will have the predetermined relationship with the differential timestamp value;

when the predetermined relationship exists, preventing sending unneeded clock rate adjustment command signals to the plurality of decoders; and

adjusting a buffer occupancy of the plurality of decoders, wherein adjusting the buffer occupancy comprises;

determining whether a relationship exists between the current buffer occupancy and a predefined threshold; and

adjusting the buffer occupancy when it is determined that the relationship exists;

whereby a clock rate adjustment is contemporaneously performable while the plurality of decoders is decoding encoded data, and

wherein a decoding delay between any two decoders of the plurality of decoders is maintained in a range of less than 10 milliseconds.

16. The network device of claim 15, wherein the clock rate adjustment command signal increases the frequency of the local system time clock when the differential local clock value is less than the differential timestamp value and decreases the frequency of the local system time clock when the differential local clock value is greater than the differential timestamp value.

17. The network device of claim 15, wherein

each decoder of the plurality of decoders includes a local buffer for storing encoded data;

each decoder of the plurality of decoders decodes the encoded data stored within the local buffer; and

wherein determining whether a relationship exists between the current buffer occupancy and a predefined threshold comprises;

determining whether a buffer occupancy of the local buffer is within a predetermined range of a total buffer occupancy; and

wherein adjusting the buffer occupancy when it is determined that the relationship exists comprises;

transmitting a data stream control command signal to the network server when it is determined that the buffer occupancy is outside the predetermined range, and

wherein the data stream control command signal alters a then-current data streaming characteristic of the network server.

18. The network device of claim 15, wherein

each decoder of the plurality of decoders includes a local buffer for storing encoded data;

each decoder of the plurality of decoders decodes the encoded data stored within the local buffer; and

wherein determining whether a relationship exists between the current buffer occupancy and a predefined threshold comprises;

determining whether a buffer occupancy of the local buffer is within a predetermined range of a total buffer occupancy; and

wherein adjusting the buffer occupancy when it is determined that the relationship exists comprises;

transmitting a clock rate control command signal to the network server when it is determined that the buffer occupancy is outside the predetermined range;

receiving a clock ratio adjustment signal corresponding to the clock rate control command signal, the clock ratio adjustment signal containing a clock adjustment ratio;

computing the clock rate adjustment command signal based on the clock adjustment ratio; and

transmitting the computed clock rate adjustment command signal to each decoder of the plurality of decoders to adjust the local system time clock such that a subsequent differential local clock value is within the predetermined range of the total buffer occupancy.

22. The network device of claim 15, wherein the range of decoding delay comprises less than approximately 1 millisecond.

20. A method of fabricating a network device, comprising providing a processor coupled to a network interface and to the plurality of decoders, the processor

providing step comprising:

providing a network interface for receiving encoded data transmitted over a network;

coupling a plurality of decoders to the network interface for decoding the received encoded data; and

coupling a processor to the network interface and the plurality of decoders, the processor coupling step comprising providing the processor containing circuitry for;

receiving a set of timestamp signals via the network interface, wherein each timestamp signal within the set of timestamp signals has a value corresponding to a timestamped beacon transmitted by a network server;

receiving a set of local clock signals, wherein each local clock signal within the set of local clock signals has a value generated by a local system time clock associated with the decoder;

computing a differential timestamp value based on values of timestamp signals within the set of timestamp signals;

computing a differential local clock value based on values of local clock signals within the set of local clock signals;

determining whether the differential local clock value has a predetermined relationship with respect to the differential timestamp value, wherein the predetermined relationship exists when the differential local clock value and the differential timestamp value are equal or within a predefined tolerance;

transmitting a clock rate adjustment command signal to the plurality of decoders when it is determined that the differential local clock value does not have the predetermined relationship with the differential timestamp value, wherein the clock rate adjustment command signal adjusts the local system time clock such that a subsequent differential local clock value will have the predetermined relationship with the differential timestamp value;

when the predetermined relationship exists, preventing sending unneeded clock rate adjustment command signals to the plurality of decoders; and

adjusting a buffer occupancy of the plurality of decoders, wherein adjusting the buffer occupancy comprises;

comparing a current buffer occupancy at each of the plurality of decoders to a predefined threshold to determine whether a relationship exists between the current buffer occupancy and the predefined threshold; and

adjusting the buffer occupancy when it is determined that the relationship exists;

whereby a clock rate adjustment is contemporaneously performable while the plurality of decoders is decoding encoded data, and

wherein a decoding delay between any two decoders of the plurality of decoders is maintained in a range of less than 10 milliseconds.

23. The method of claim 20, wherein the range of decoding delay comprises less than approximately 1 millisecond.

24. A method of synchronizing a plurality of decoders within a network to a server by way of a processor coupled to a network interface and to the plurality of decoders, comprising:

providing the processor coupled to the network interface and to the plurality of decoders;

receiving a set of timestamp signals, each timestamp signal within the set of timestamp signals having a value corresponding to a timestamped beacon transmitted by a network server, the network server for streaming encoded data over a network through the network interface;

receiving a set of local clock signals, each local clock signal within the set of local clock signals having a value generated by a local system time clock associated with a plurality of decoders of a network client, the plurality of decoders for decoding the encoded data;

computing a differential timestamp value based on values of timestamp signals within the set of timestamp signals by the processor;

computing a differential local clock value based on values of local clock signals within the set of local clock signals;

determining whether the differential local clock value has a predetermined relationship with respect to the differential timestamp value, wherein the predetermined relationship exists when the differential local clock value and the differential timestamp value are equal or within a predefined tolerance;

transmitting a clock rate adjustment command signal to the plurality of decoders when it is determined that the differential local clock value does not have the predetermined relationship with the differential timestamp value, the clock rate adjustment command signal for adjusting the local system time clock such that a subsequent differential local clock value will have the predetermined relationship with the differential timestamp value;

when the predetermined relationship exists, preventing sending unneeded clock rate adjustment command signals to the plurality of decoders; and

adjusting a buffer occupancy of the plurality of decoders,

thereby contemporaneously performing a clock rate adjustment while the plurality of decoders is decoding encoded data, and

wherein a decoding delay between any two decoders of the plurality of decoders is maintained in a range of less than 10 milliseconds;

wherein adjusting the buffer occupancy of the plurality of decoders comprises;

determining whether a buffer occupancy of a local buffer of each decoder of the plurality of decoders is within a predetermined range of a total buffer occupancy as each decoder of the plurality of decoders decodes encoded data within the local buffer;

transmitting a clock rate control command signal to the network server when it is determined that the buffer occupancy is outside the predetermined range;

receiving a clock ratio adjustment signal corresponding to the clock rate control command signal, the clock ratio adjustment signal containing a clock adjustment ratio;

computing the clock rate adjustment command signal based on the clock adjustment ratio by the processor to compute the differential timestamp value, wherein the differential timestamp value is defined by a parameter selected from a group consisting essentially of;

a difference in values between a pair of timestamps having been consecutively transmitted by the powerline communication interface;

an average difference in values between a plurality of pairs of consecutively output timestamps; and

transmitting the computed clock rate adjustment command signal to each decoder of the plurality of decoders to adjust the local system time clock such that a subsequent differential local clock value is within the predetermined range of the total buffer occupancy.