Auto latency correction method and apparatus for MPEG playback system

• US 5,596,420 A
• Filed: 06/07/1995
• Issued: 01/21/1997
• Est. Priority Date: 12/14/1994
• Status: Expired due to Term

• Current Assignee: Cirrus Logic Incorporated
• Original Assignee: Cirrus Logic Incorporated
• Inventors: Daum, Daniel
• Primary Examiner(s): Tran, Thai Q.
• Application Number:
  US08/483,580
• Time in Patent Office: 594 Days
• Field of Search: 358/335, 358/310, 358/341, 358/342, 358/343, 360/19.1, 360/33.1, 348/515
• US Class Current: 386/241
• CPC Class Codes: G09G 5/393 Arrangements for updating the contents of the bit-mapped memory
• CPC Class Codes: G11B 20/00007 Time or data compression or expansion audio compression based on psychoacoustics G10L19/00; data processing for reproducing audio data at different playback speeds G10L21/04; video compression H04N19/00; data compression per se H03M7/30
• CPC Class Codes: G11B 20/10527 Audio or video recording; Data buffering arrangements G11B20/12 - G11B20/18 take precedence
• CPC Class Codes: G11B 2220/213 Read-only discs
• CPC Class Codes: G11B 2220/2545 CDs
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/3027 used signal is digitally coded
• CPC Class Codes: G11B 27/3036 Time code signal
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/42 characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/423 characterised by memory arrangements H04N19/433 takes precedence
• CPC Class Codes: H04N 19/427 Display on the fly, e.g. simultaneous writing to and reading from decoding memory
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 21/4143 embedded in a Personal Computer [PC]
• CPC Class Codes: H04N 21/4305 Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/4341 Demultiplexing of audio and video streams
• CPC Class Codes: H04N 5/775 between a recording apparatus and a television receiver
• CPC Class Codes: H04N 5/85 : on discs or drums
• CPC Class Codes: H04N 9/8042 : involving data reduction
• CPC Class Codes: H04N 9/8063 : using time division multiplex of the PCM audio and PCM video signals
• CPC Class Codes: H04N 9/8216 : using time division multiplex
• CPC Class Codes: H04N 9/87 : Regeneration of colour television signals H04N9/80 takes precedence

Abstract

A multimedia system including an audio/video decoder/decompresser for decoding/decompressing a compressed encoded audio/video data stream to generate video images for display and audio signals for audible reproduction. The multimedia system includes an integrated system and video decoder with an audio/video synchronization circuit for substantially synchronizing the display of video images with audio playback. Latency between Audio and Video decompression/decoding may be adjusted dynamically by internal hardware. Hardware signals indicating the need to skip or repeat a frame or frames of video or audio data are fed to a hardware circuit which may increment or decrement a latency value accordingly. The latency value is thus stepwise incremented or decremented until no further frame skips/repeats occur. The hardware implementation allows for better matching of latency value to audio and video decoder, frees up software for other tasks and reduces software code requirements. In addition, changes to the audio and video data paths (e.g., switching to an auxiliary audio decoder) are automatically and quickly compensated for. The hardware implementation may provide for additional levels of latency adjustment.

Claims

1. An apparatus for controlling playback of audio and video signals from an encoded data stream comprising at least audio data packets, video data packets, audio presentation time stamps and video presentation time stamps, said apparatus comprising:

a system decoder for receiving the encoded data stream and separating the data stream into an encoded audio data stream comprising at least the audio data packets and audio presentation time stamps and an encoded video data stream comprising at least the video data packets and video presentation time stamps;

an audio decoder, for receiving the audio data packets from the encoded audio data stream and decoding the audio data packets;

a video decoder, coupled to said system decoder and said audio decoder, for receiving a system time clock and the encoded video data stream and decoding the encoded video data stream in synchronization with the system time clock, said video decoder further comprising;

a latency value register for storing a video decoder latency value;

counter means, coupled to said latency value register, for adding the latency value to a video presentation time stamp value to produce a normalized video presentation time stamp value;

subtracter means, coupled to said counter means, for subtracting the normalized video presentation time stamp value from an audio presentation time stamp value to produce a difference value;

comparator means, coupled to said subtracter means, for comparing the difference value with a predetermined time drift threshold and outputting a video frame skip signal if the difference value exceeds the pre-determined time drift threshold and the difference value is negative and outputting a video frame repeat signal if the difference value exceeds the predetermined time drift threshold and the difference value is positive; and

latency value adjustment means, coupled to said comparator means and said latency value register, for adding a predetermined value to the latency value stored in said latency value register if the video frame repeat signal is generated by the comparator means and for subtracting the predetermined value from the latency value stored in said latency value register if the video frame skip signal is generated by the comparator means.

2. The apparatus of claim 1, wherein said comparator means compares the difference value with the predetermined time drift threshold and outputs a single video frame skip signal if the difference value exceeds the pre-determined time drift threshold by a first predetermined amount and the difference value is negative and outputs a multiple video frame skip signal if the difference value exceeds the predetermined time drift threshold by a second predetermined amount and the difference value is negative.

3. The apparatus of claim 2, wherein said latency value adjustment means subtracts a first predetermined value from the latency value stored in said latency value register if the single video frame skip signal is generated by the comparator means and subtracts a second predetermined value from the latency value stored in said latency value register if the multiple video frame skip signal is generated by the comparator means.

4. The apparatus of claim 1, wherein said comparator means compares the difference value with the predetermined time drift threshold and outputs a single video frame repeat signal if the difference value exceeds the pre-determined time drift threshold by a first predetermined amount and the difference value is positive and outputs a multiple video frame repeat signal if the difference value exceeds the predetermined time drift threshold by a second predetermined amount and the difference value is positive.

5. The apparatus of claim 4, wherein said latency value adjustment means adds a first predetermined value to the latency value stored in said latency value register if the single video frame repeat signal is generated by the comparator means and adds a second predetermined value to the latency value stored in said latency value register if the multiple video frame repeat signal is generated by the comparator means.

6. An apparatus for controlling playback of audio and video signals from an encoded data stream comprising at least audio data packets, video data packets, audio presentation time stamps and video presentation time stamps, said apparatus comprising:

a system decoder for receiving the encoded data stream and separating the data stream into an encoded audio data stream comprising at least the audio data packets and audio presentation time stamps and an encoded video data stream comprising at least the video data packets and video presentation time stamps;

a latency value register for storing a decoder latency value;

counter means, coupled to said latency value register, for adding the latency value to one of the video and audio presentation time stamp values to produce a normalized presentation time stamp value;

subtracter means, coupled to said counter means, for subtracting the normalized presentation time stamp value from the other of the audio and video presentation time stamp values to produce a difference value;

comparator means, coupled to said subtracter means, for comparing the difference value with a predetermined time drift threshold and outputting a frame skip/repeat signal if the difference value exceeds the pre-determined time drift threshold; and

latency value adjustment means, coupled to said comparator means and said latency value register, for modifying the latency value stored in said latency value register if the frame skip/repeat signal is generated by the comparator means.

7. The apparatus of claim 6, wherein said comparator means compares the difference value with the predetermined time drift threshold and outputs a single video frame skip/repeat signal if the difference value exceeds the predetermined time drift threshold by a first predetermined amount.

8. The apparatus of claim 7, wherein said latency value adjustment means modifies the latency value stored in said latency value register by a first predetermined value if the single video frame skip/repeat signal is generated by the comparator means.

9. The apparatus of claim 8, wherein said comparator means compares the difference value with the predetermined time drift threshold and outputs a multiple video frame skip/repeat signal if the difference value exceeds the predetermined time drift threshold by a second predetermined amount.

10. The apparatus of claim 9, wherein said latency value adjustment means modifies the latency value stored in said latency value register by a second predetermined value if the multiple video frame skip/repeat signal is generated by the comparator means.

11. A method for controlling playback of audio and video signals from an encoded data stream comprising at least audio data packets, video data packets, audio presentation time stamps and video presentation time stamps, the method comprising the steps of:

separating, in a system decoder the encoded data stream into an encoded audio data stream comprising at least the audio data packets and audio presentation time stamps and an encoded video data stream comprising at least the video data packets and video presentation time stamps,

decoding, in an audio decoder, the audio data packets from the encoded audio data stream,

decoding, in a video decoder, the encoded video data stream in synchronization with a system time clock,

adding a stored latency value to a video presentation time stamp value to produce a normalized video presentation time stamp value,

subtracting the normalized video presentation time stamp value from an audio presentation time stamp value to produce a difference value,

comparing the difference value with a predetermined time drift threshold and outputting a video frame skip signal if the difference value exceeds the pre-determined time drift threshold and the difference value is negative and outputting a video frame repeat if the difference value exceeds the predetermined time drift threshold and the difference value is positive,

adding a predetermined value to the latency value if the video frame repeat signal is generated in the comparing step, and

subtracting the predetermined value from the latency value if the video frame skip signal is generated in the comparing step.

12. The method of claim 11, wherein said comparing step further comprises the steps of:

comparing the difference value with the predetermined time drift threshold,

outputting a single video frame skip signal if the difference value exceeds the pre-determined time drift threshold by a first predetermined amount and the difference value is negative, and

outputting a multiple video frame skip signal if the difference value exceeds the pre-determined time drift threshold by a second predetermined amount and the difference value is negative.

13. The method of claim 12, wherein said step of subtracting the predetermined value from the latency value further comprises the steps of:

subtracting a first predetermined value from the latency value if the single video frame skip signal is generated in said comparing step, and

subtracting a second predetermined value from the latency value if the multiple video frame skip signal is generated in said comparing step.

14. The method of claim 11, wherein said comparator step further comprises the steps of:

comparing the difference value with the predetermined time drift threshold,

outputting a single video frame repeat signal if the difference value exceeds the pre-determined time drift threshold by a first predetermined amount and the difference value is positive, and

outputting a multiple video frame repeat signal if the difference value exceeds the pre-determined time drift threshold by a second predetermined amount and the difference value is positive.

15. The method of claim 14, wherein the step of adding the predetermined value to the latency value comprises the steps of:

adding a first predetermined value to the latency value if the single video frame repeat signal is generated in the comparing step, and

adding a second predetermined value to the latency value if the multiple video frame repeat signal is generated in the comparing step.

16. A method for controlling playback of audio and video signals from an encoded data stream comprising at least audio data packets, video data packets, audio presentation time stamps and video presentation time stamps, said apparatus comprising:

separating the encoded data stream into an encoded audio data stream comprising at least the audio data packets and audio presentation time stamps and an encoded video data stream comprising at least the video data packets and video presentation time stamps;

combining a decoder latency value to one of the video and audio presentation time stamp values to produce a normalized presentation time stamp value;

subtracting the normalized presentation time stamp value from the other of the audio and video presentation time stamp values to produce a difference value;

comparing the difference value with a predetermined time drift threshold and outputting a frame skip/repeat signal if the difference value exceeds the pre-determined time drift threshold; and

modifying the latency value if the frame skip/repeat signal is generated in the comparing step.

17. The method of claim 16, wherein said comparing step further comprises the steps of:

comparing the difference value with the predetermined time drift threshold, and

outputting a single video frame skip/repeat signal if the difference value exceeds the pre-determined time drift threshold by a first predetermined amount.

18. The method of claim 17, wherein step of modifying further comprises the step of combining the latency value with a first predetermined value if the single video frame skip/repeat signal is generated in the comparing step.

19. The method of claim 18, wherein said step of comparing further comprises the steps of:

comparing the difference value with the predetermined time drift threshold, and

outputting a multiple video frame skip/repeat signal if the difference value exceeds the pre-determined time drift threshold by a second predetermined amount.

20. The method of claim 19, wherein said modifying step further comprises the step of combining the latency value with a second predetermined value if the multiple video frame skip/repeat signal is generated by in the comparing step.