Lip sync error detection and correction
First Claim
1. A multimedia synchronization method, comprising:
- identifying, by a server, a packet pair, the packet comprising a video stream packet comprising a video packet in a video stream associated with a multimedia program and an audio stream packet comprising an audio packet in an audio stream associated with the multimedia program;
receiving the video stream packet at a first monitoring point in a multimedia network;
decoding a timestamp associated with the video stream packet;
determining a first inter-stream synchronization offset, the first inter-stream synchronization offset indicative of a time difference between arrival of the video stream packet at the first monitoring point and arrival of the audio stream packet at the first monitoring point;
responsive to detecting the video stream packet and the audio stream packet at a second monitoring point of the multimedia network, determining a second inter-stream synchronization offset, the second inter-stream synchronization offset indicative of a time difference between arrival of the video stream packet at the second monitoring point and arrival of the audio stream packet at the second monitoring point, wherein the video stream packet traverses a first network path, comprising a first set of network switches, between the first monitoring point and the second monitoring point and the audio stream packet traverses a second network path, comprising a second set of network switches, different than the first set of network switches, between the first monitoring point and the second monitoring point;
determining, by the server, a synchronization delta indicative of a difference between the first inter-stream synchronization offset and the second inter-stream synchronization offset, wherein the synchronization delta indicates a variation in inter-stream synchronization offset associated with the packet pair between two points in the multimedia network; and
responsive to determining that the synchronization delta exceeds a predetermined threshold;
logging, by the server, synchronization error information; and
initiating, at a monitoring point, a corrective action procedure.
1 Assignment
0 Petitions
Accused Products
Abstract
A method of managing lip synchronization error in a multimedia content delivery network includes identifying a video packet and an audio packet associated with the video packet and determining a synchronization offset between the audio and video packets at a first monitoring point in the network. The audio and video packets are then detected at a second monitoring point in the network and a second synchronization offset is determined. When a delta between the first synchronization offset and the second synchronization offset exceeds a threshold, lip synchronization error information may be automatically recorded and/or reported to a service provider and corrective action may be taken if potential sources of the lip synchronization error are within the domain of the service provider. The video packet may be identified by a timestamp within the packet and the audio packet may be identified by audio data within the audio packet.
-
Citations
20 Claims
-
1. A multimedia synchronization method, comprising:
-
identifying, by a server, a packet pair, the packet comprising a video stream packet comprising a video packet in a video stream associated with a multimedia program and an audio stream packet comprising an audio packet in an audio stream associated with the multimedia program; receiving the video stream packet at a first monitoring point in a multimedia network; decoding a timestamp associated with the video stream packet; determining a first inter-stream synchronization offset, the first inter-stream synchronization offset indicative of a time difference between arrival of the video stream packet at the first monitoring point and arrival of the audio stream packet at the first monitoring point; responsive to detecting the video stream packet and the audio stream packet at a second monitoring point of the multimedia network, determining a second inter-stream synchronization offset, the second inter-stream synchronization offset indicative of a time difference between arrival of the video stream packet at the second monitoring point and arrival of the audio stream packet at the second monitoring point, wherein the video stream packet traverses a first network path, comprising a first set of network switches, between the first monitoring point and the second monitoring point and the audio stream packet traverses a second network path, comprising a second set of network switches, different than the first set of network switches, between the first monitoring point and the second monitoring point; determining, by the server, a synchronization delta indicative of a difference between the first inter-stream synchronization offset and the second inter-stream synchronization offset, wherein the synchronization delta indicates a variation in inter-stream synchronization offset associated with the packet pair between two points in the multimedia network; and responsive to determining that the synchronization delta exceeds a predetermined threshold; logging, by the server, synchronization error information; and initiating, at a monitoring point, a corrective action procedure. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. An audio/video synchronization server comprising:
-
a processor having access to a non-transitory computer readable storage medium including processor-executable instructions that, when executed by a processor, cause the processor to perform operations comprising; identifying a packet pair, the packet comprising a video stream packet comprising a video packet in a video stream associated with a multimedia program and an audio stream packet comprising an audio packet in an audio stream associated with the multimedia program; responsive to detecting the video stream packet and the audio stream packet at a first monitoring point in a multimedia content network, determining a first inter-stream synchronization offset, the first inter-stream synchronization offset indicative of a time difference between arrival of the video stream packet at a first monitoring point of the multimedia content network and arrival of the audio stream packet at the first monitoring point; responsive to detecting the video stream packet and the audio stream packet at a second monitoring point in the multimedia content network, determining a second inter-stream synchronization offset, the second inter-stream synchronization offset indicative of a time difference between arrival of the video stream packet at the second monitoring point and arrival of the audio stream packet at the second monitoring point in the multimedia content network, wherein the video stream packet traverses a first network path, comprising a first set of network switches, between the first monitoring point and the second monitoring point and the audio stream packet traverses a second network path, comprising a second set of network switches, different than the first set of network switches, between the first monitoring point and the second monitoring point; determining a synchronization delta indicative of a difference between the first inter-stream synchronization offset and the second inter-stream synchronization offset, wherein the synchronization delta indicates a variation in inter-stream synchronization offset associated with the packet pair between two points in the multimedia content network; and responsive to determining the synchronization delta exceeding a predetermined threshold; logging synchronization error information, indicative of the audio stream packet, the video stream packet, the first inter-stream synchronization offset, and the second inter-stream synchronization offset; and initiating, at a monitoring point, a corrective action procedure by adding packets to at least one of;
a video stream carrying the video stream packet and an audio stream carrying the audio stream packet. - View Dependent Claims (11, 12, 13, 14)
-
-
15. A non-transitory computer readable medium including processor executable instructions that, when executed by a processor, cause the processor to perform operations comprising:
-
identifying a video stream packet, comprising a video packet in a video stream associated with a multimedia program, of a packet pair based on timestamp information in the video packet; identifying an audio stream packet, comprising an packet in an audio stream associated with the multimedia program, of the packet pair based on pulse code modulation data in the audio stream packet; responsive to detecting the video stream packet and the audio stream packet at an upstream monitoring point in a multimedia network, determining a first inter-stream synchronization offset, the first inter-stream synchronization offset indicative of a time difference between arrival of the video stream packet at the upstream monitoring point and arrival of the audio stream packet at the upstream monitoring point; and responsive to detecting the video stream packet and the audio stream packet at a midstream monitoring point in the multimedia network, determining a second inter-stream synchronization offset, the second inter-stream synchronization offset indicative of a time difference between arrival of the video stream packet at the midstream monitoring point and arrival of the audio stream packet at the midstream monitoring point in the multimedia network, wherein the video stream packet traverses a first network path, comprising a first set of network switches, between the first monitoring point and the second monitoring point and the audio stream packet traverses a second network path, comprising a second set of network switches, different than the first set of network switches, between the first monitoring point and the second monitoring point; determining a second inter-stream synchronization offset between the video stream packet and the audio stream packet at a midstream monitoring point in the multimedia network; determining a synchronization delta indicative a difference between the first inter-stream synchronization offset and the second inter-stream synchronization offset, wherein the synchronization delta indicates a variation in inter-stream synchronization offset associated with the packet pair between two points in the multimedia content delivery network; and responsive to determining the synchronization delta exceeding a predetermined threshold; logging synchronization error information, indicative of the audio stream packet, the video stream packet, the first inter-stream synchronization offset, and the second inter-stream synchronization offset; and initiating, at a monitoring point, a corrective action procedure by adding packets to at least one of;
a video stream carrying the video stream packet and an audio stream carrying the audio stream packet. - View Dependent Claims (16, 17, 18, 19, 20)
-
Specification