Audio-video synchronization for digital systems

• US 20030156342A1
• Filed: 02/20/2002
• Published: 08/21/2003
• Est. Priority Date: 02/20/2002
• Status: Active Grant

• Current Assignee: DIRECTV, LLC (AT&T, Inc.)
• Original Assignee: The DIRECTV Group, Inc. (AT&T, Inc.)
• Inventors: Lee, Hsien-Hua, Casavant, Scott, Yap, Adrian
• Granted Patent: US 7,379,653 B2
• Time in Patent Office: Days
• US Class Current: 360/39
• CPC Class Codes: G11B 20/10 Digital recording or reproducing
• CPC Class Codes: G11B 20/1217 on discs
• CPC Class Codes: G11B 2020/10537 Audio or video recording
• CPC Class Codes: G11B 2220/2562 DVDs [digital versatile discs]; Digital video discs; MMCDs; HDCDs
• CPC Class Codes: G11B 2220/90 Tape-like record carriers
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: H04N 21/4147 PVR [Personal Video Recorder] H04N5/76 takes precedence ; arrangements for broadcast with accumulation-type receivers H04H20/40
• CPC Class Codes: H04N 21/4305 Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 5/76 Television signal recording
• CPC Class Codes: H04N 5/781 on disks or drums
• CPC Class Codes: H04N 5/85 on discs or drums
• CPC Class Codes: H04N 5/907 using static stores, e.g. storage tubes or semiconductor memories H04N5/91 takes precedence
• CPC Class Codes: H04N 9/8042 involving data reduction

Abstract

The audio-video synchronization process ensures continuity of displayed AV data. To initialize the process, a transport processor determines whether an occupancy criterion of a buffer storing received audio and video frames has been met. If the criterion is met, the transport processor obtains an initial time stamp value from an initial frame, and a subsequent time stamp value from a subsequent frame. Initial and subsequent parameters are computed from these respective time stamp values, and are compared against each other. If the parameters coincide, the frame is valid, and corresponding audio or video frames may be decoded and displayed. If the parameters do not coincide, a recovery process is initiated. In either event, the invention makes it possible to achieve audio-video synchronization for both live and playback modes of a digital video recorder (DVR).

Claims

1. An audio-video (AV) synchronization process, comprising:

determining whether an occupancy criterion of a buffer storing received audio and video frames has been met, and if so

obtaining an initial time stamp value from an initial frame;

obtaining a subsequent time stamp value from a subsequent frame;

computing an initial parameter based on the initial time stamp value;

computing a subsequent parameter based on the subsequent time stamp value;

determining if the computed initial and subsequent parameters coincide, and if so

outputting corresponding audio and/or video frames for decoding and display.

2. The process of claim 1, wherein said initial and subsequent time stamp values are presentation time stamps of initial and subsequent video frames or presentation time stamps of initial and subsequent audio frames, each of the audio and video frames also including associated audio or video data.

3. The process of claim 1, wherein said initial and subsequent parameters are difference values, each computed as a time difference between when the corresponding time stamp is received by a processor and a time where the processor accesses a time from a system timer.

4. The process of claim 3, wherein said determining step compares whether the difference value representing the subsequent frame, Δ

t_new, is equal to the difference value representing the initial frame, Δ

t_old, the coincidence between these difference values representing a valid time stamp of the subsequent frame.

5. The process of claim 4, wherein the video frame is decoded for display when the timestamp is valid.

6. The process of claim 2, wherein if the initial and subsequent parameters do not coincide, a recovery process is performed.

7. The process of claim 6, wherein if the initial and subsequent parameters do not coincide because the presentation time stamp of the initial frame is corrupted but the corresponding video data is valid, or if a time base at which all presentation time stamps are obtained is changed, the video frame is decoded for display.

8. The process of claim 6, wherein, if the initial and subsequent parameters do not coincide because both the presentation time stamp and the corresponding video data of the initial frame are corrupted, the most recently processed video frame is repeated.

9. The process of claim 6, wherein the recovery process is performed up to T times, T being a selectable parameter, and wherein if T is exceeded, the recovery process is terminated.

10. The process of claim 2, wherein if the frames are audio frames, parameters representing a computed time are compared to a system time in order to determine if an audio frame is repeated in the process, skipped in the process, or decoded for display.

11. The process of claim 10, wherein if computed time exceeds system time by a half of an audio frame time, the audio frame is repeated.

12. The process of claim 10, wherein if computed time lags system time by a half of an audio frame time, the audio frame is skipped.

13. The process of claim 10, wherein if computed time exceeds system time by less than half of an audio frame time, or lags system time by less than half an audio frame, the audio frame is decoded for display.

14. An apparatus for synchronizing audio and video in a digital video recording (DVR) system, comprising:

a buffer for receiving a plurality of packets having data representing audio and video frames therein;

a processor for determining whether an occupancy criterion of the buffer storing said received audio and video frames has been met, wherein the processor obtains an initial time stamp value from an initial frame and from a subsequent frame, computes initial and subsequent parameters based on the respective initial and subsequent time stamp values, and determines whether the computed initial and subsequent parameters coincide if the occupancy criterion is met, and

a decoder for decoding audio and/or video frames for display if the parameters coincide.

15. The apparatus of claim 14, wherein said initial and subsequent time stamp values are presentation time stamps of initial and subsequent video frames or presentation time stamps of initial and subsequent audio frames, each of the audio and video frames also including associated audio or video data.

16. The apparatus of claim 14, wherein said initial and subsequent parameters are difference values, each computed as a time difference between when the corresponding time stamp is received by the processor and a time where the processor accesses a time from a system timer.

17. The apparatus of claim 16, wherein the processor compares whether the difference value representing the subsequent frame, Δ

t_new, is equal to the difference value representing the initial frame, Δ

t_old, the coincidence between these difference values representing a valid time stamp of the subsequent frame.

18. The apparatus of claim 17, wherein the video frame is decoded for display when the timestamp is valid.

19. The apparatus of claim 14, wherein if the initial and subsequent parameters do not coincide, the processor performs a recovery process.

20. The apparatus of claim 19, wherein if the initial and subsequent parameters do not coincide because a presentation time stamp of the initial frame is corrupted but corresponding video data of the frame is valid, or if a time base at which all presentation time stamps are obtained is changed, the video frame is decoded and displayed.

21. The apparatus of claim 19, wherein if the initial and subsequent parameters do not coincide because both a presentation time stamp and corresponding video data of the initial frame are corrupted, the most recent video frame processed is repeated.

22. The apparatus of claim 19, wherein the recovery process is performed up to T times, T being a selectable parameter, and wherein if T is exceeded, the recovery process is terminated.

23. The apparatus of claim 14, wherein if the frames are audio frames, parameters representing a computed time are compared to a system time in order to determine if an audio frame is repeated for processing, skipped for processing or decoded for display.

24. The apparatus of claim 23, wherein if computed time exceeds system time by a half of an audio frame time, the audio frame is repeated.

25. The apparatus of claim 23, wherein if computed time lags system time by a half of an audio frame time, the audio frame is skipped.

26. The apparatus of claim 23, wherein if computed time exceeds system time by less than half of an audio frame time, or lags system time by less than half an audio frame, the audio frame is decoded for display.

27. A method of synchronizing audio and video frames, comprising:

(a) computing an initial parameter based on an initial video time stamp of an initial video frame;

(b) computing a subsequent parameter based on a subsequent video time stamp value of a subsequent video frame;

(c) comparing the computed parameters, a coincidence between the two indicating a valid subsequent video time stamp, and

(d) synchronizing an audio frame to the subsequent video frame based on the valid subsequent video time stamp.

28. The method of claim 27, further comprising repeating steps (b) through (d) for all subsequent video and/or audio frames.

29. A processor for synchronizing audio and video frames, comprising:

a buffer for receiving a plurality of packets having data representing audio and video frames therein; and

circuitry for computing a initial parameter based on an initial time stamp value of an initial video frame, and for computing a subsequent parameter based on a subsequent time stamp value of a subsequent video frame,

wherein the circuitry determines whether the computed initial and subsequent parameters coincide, a coincidence between the two indicating a valid subsequent video time stamp, and

wherein the processor synchronizes an audio frame to the subsequent video frame based on the valid subsequent video time stamp.

30. The processor of claim 29, wherein an output of the processor contains corresponding audio and/or video frames for decoding and display.