Video and audio reproducing device and video decoding device

• US 6,148,135 A
• Filed: 08/18/1997
• Issued: 11/14/2000
• Est. Priority Date: 01/29/1996
• Status: Expired due to Fees

• Current Assignee: Mitsubishi Denki Kabushiki Kaisha (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Denki Kabushiki Kaisha (Mitsubishi Electric Corporation)
• Inventors: Suzuki, Mitsuyoshi
• Primary Examiner(s): Garber, Wendy
• Assistant Examiner(s): PHAM, THANH J
• Application Number:
  US08/912,644
• Time in Patent Office: 1,184 Days
• Field of Search: 348/423, 348/515, 348/845.1-845.3, 348/464, 348/462, 348/512, 348/473, 348/484, 348/408, 386/39, 386/96, 386/98, 386/101, 386/104-106, 395/135, 395/12, 358/339
• US Class Current: 386/201
• CPC Class Codes: G11B 2220/213 Read-only discs
• CPC Class Codes: G11B 2220/2545 CDs
• CPC Class Codes: G11B 2220/2562 DVDs [digital versatile discs]; Digital video discs; MMCDs; HDCDs
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/3027 used signal is digitally coded
• CPC Class Codes: H04N 21/4305 Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/4341 Demultiplexing of audio and video streams
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 5/85 on discs or drums
• CPC Class Codes: H04N 9/8042 involving data reduction
• CPC Class Codes: H04N 9/8063 using time division multiplex of the PCM audio and PCM video signals

Abstract

The video and audio synchronization controller sets a system clock reference (SCR) included in the frame header portion of the coded video and audio data into a system time counter to set a timing of a system time clock (STC), immediately after a video and audio reproducing device has been powered on. The video and audio reproducing device decodes and displays a first arriving video data synchronous with a display system frame pulse, which is the closest to the value of a video time stamp included in a frame header of the first video frame on the time axis of the set STC. The video and audio synchronization controller sets again the value of the time stamp included in the first video frame to the system time counter to update the STC. The video and audio reproducing device outputs the video data for the second and subsequent video frames and all the audio data synchronous with the updated STC by outputting the video and audio data when the time indicated by the video and audio time stamps arrives on the time axis of the updated STC.

Claims

1. A video and audio reproducing device for synchronizing a decoded video output and a decoded audio output based on a video decoding start controlling signal and an audio decoding start controlling signal comprising:

a video and audio separator for separating a video time stamp, an audio time stamp coded video data and coded audio data from coded video and audio data, a video buffer memory for storing the coded video data, an audio buffer memory for storing the coded audio data, a video and audio synchronization controller for generating video decoding start controlling signal and audio decoding start controlling signal to start decoding of the coded video data and the coded audio data base on the video and audio time stamps, a video decoder for decoding the coded video data, and an audio decoder for decoding the coded audio data;

wherein said video and audio synchronization controller sets a system clock reference (SCR) included in the frame header portion of the coded video and audio data into a system time counter to set a timing of a system time clock (STC) immediately after the video and audio reproducing device has been powered on, decodes and displays a first arriving video data synchronous with a display system frame pulse, which is the closest to the value of a video time stamp included in a frame header of the first video frame on the time axis of the set STC, sets again the value of the time stamp included in the first video frame to the system time counter to update the STC, outputs the video data for the second and subsequent video frames and all the audio data synchronously with the updated STC by outputting the video and audio data when the time indicated by the video and audio time stamps arrives on the time axis of the updated STC.

2. A video and audio reproducing device for synchronizing a decoded video output and a decoded audio output based on a video decoding start controlling signal and an audio decoding start controlling signal comprising:

a video and audio separator for separating a video time stamp, an audio time stamp, coded video data and coded audio data from coded video and audio data, a video buffer memory for storing the coded video data, an audio buffer memory for storing the coded audio data, a video and audio synchronization controller for generating video decoding start controlling signal and audio decoding start controlling signal to start decoding of the coded video data and the coded audio data base on the video and audio time stamps, a video decoder for decoding the coded video data, and an audio decoder for decoding the coded audio data;

wherein said video and audio synchronization controller decodes and displays a first arriving video frame immediately after the video and audio reproducing device has been powered on, regardless of the time stamp included in a header of the first arriving video frame, sets a time stamp included in the first arriving video frame to the system time counter while the first arriving video frame is displayed to set a timing of a system time clock (STC), outputs the video data for the second and subsequent video frames and all the audio data synchronously with the set STC by outputting the video and audio data when the time indicated by the video and audio time stamps arrives on the time axis of the set STC.

3. A video and audio reproducing device for synchronizing a decoded video output and a decoded audio output based on a video decoding start controlling signal and an audio decoding start controlling signal comprising:

a video and audio separator for separating a video time stamp, an audio time stamp, coded video data and coded audio data from coded video and audio data, a video buffer memory for storing the coded video data, an audio buffer memory for storing the coded audio data, a video and audio synchronization controller for generating video decoding start controlling signal and audio decoding start controlling signal to start decoding of the coded video data and the coded audio data base on the video and audio time stamps, a video decoder for decoding the coded video data, and an audio decoder for decoding the coded audio data;

wherein on the video processing side, storing the separated video time stamps in the video time stamp memory, comparing a write address used for writing the coded video data to the video buffer memory with a read address used for reading an address at a video frame address comparator, reading and latching a head time stamp stored in the video time stamp memory, when the write address matches with the read address, supplying the latched time stamp to the video and audio synchronization controller as a video time stamp, and on the audio processing side, storing the separated audio time stamps in the audio time stamp memory, comparing a write address used for writing the coded audio data to the audio buffer memory with a read address used for reading an address temporarily stored in the audio buffer memory at an audio frame address comparator, reading and latching a head time stamp stored in the audio time stamp memory, when the write address matches with the read address, supplying the latched time stamp to the video and audio synchronization controller as an audio time stamp, and

said video and audio synchronization controller generates video and audio decoding starting signals based on the video time stamp and the audio stamp, respectively, and synchronizes the video output and the audio output by decoding the coded video data and the coded audio data based on the video decoding starting signal and the audio decoding starting signal, respectively.

4. A video and audio reproducing device for synchronizing a decoded video output and a decoded audio output based on a video decoding start controlling signal and an audio decoding start controlling signal comprising:

a video and audio separator for separating a video time stamp, an audio time stamp, coded video data and coded audio data from coded video and audio data, a video buffer memory for storing the coded video data, an audio buffer memory for storing the coded audio data, a video and audio synchronization controller for generating video decoding start controlling signal and audio decoding start controlling signal to start decoding of the coded video data and the coded audio data base on the video and audio time stamps, a video decoder for decoding the coded video data, and an audio decoder for decoding the coded audio data;

wherein on the video processing side, storing the separated video time stamps in the video time stamp memory, comparing a write video frame count used for writing the coded video data to the video buffer memory with a read video frame count of the read video frame at a video frame address comparator, reading and latching a head time stamp stored in the video time stamp memory, when the write video frame count matches with the read video frame count, supplying the latched time stamp to the video and audio synchronization controller as a video time stamp, and on the audio processing side, storing the separated audio time stamps in the audio time stamp memory, comparing a write audio frame count used for writing the coded audio data to the audio buffer memory with a read audio frame count of the read audio frame at an audio frame address comparator, reading and latching a head time stamp stored in the audio time stamp memory, when the write audio frame count matches with the read audio frame count, supplying the latched time stamp to the video and audio synchronization controller as an audio time stamp; and

said video and audio synchronization controller generates video and audio decoding starting signals based on the video time stamp and the audio stamp, respectively, and synchronizes the video output and the audio output by decoding the coded video data and the coded audio data based on the video decoding starting signal and the audio decoding starting signal, respectively.

5. A video and audio reproducing device for synchronizing a decoded video output and a decoded audio output based on a video decoding start controlling signal and an audio decoding start controlling signal comprising:

a video and audio separator for separating a video time stamp, an audio time stamp, coded video data and coded audio data from coded video and audio data, a video buffer memory for storing the coded video data, an audio buffer memory for storing the coded audio data, a video and audio synchronization controller for generating video decoding start controlling signal and audio decoding start controlling signal to start decoding of the coded video data and the coded audio data base on the video and audio time stamps, a video decoder for decoding the coded video data, and an audio decoder for decoding the coded audio data;

wherein on the video processing side, latching the separated video time stamps in the video time stamp latch, comparing a write video frame count used for writing the coded video data to the video buffer memory with a read video frame count of the read video frame at a video frame address comparator, reading a time stamp latched in the video time stamp latch, when the write video frame count matches with the read video frame count, supplying the latched time stamp to the video and video synchronization controller as a video time stamp, and on the audio processing side, latching the separated audio time stamps in the audio time stamp latch, comparing a write audio frame count used for writing the coded audio data to the audio buffer memory with a read audio frame count of the read audio frame at an audio frame address comparator, reading a time stamp latched in the audio time stamp latch, when the write audio fame count matches with the read audio frame count, supplying the latched time stamp to the video and audio synchronization controller as an audio time stamp; and

said video and audio synchronization controller generates video and audio decoding starting signals based on the video time stamp and the audio stamp, respectively, and synchronizes the video output and the audio output by decoding the coded video data and the coded audio data based on the video decoding starting signal and the audio decoding starting signal, respectively.

6. A video and audio reproducing device for synchronizing a decoded video output and a decoded audio output based on a video decoding start controlling signal and an audio decoding start controlling signal comprising:

a video and audio separator for separating a video time stamp, an audio time stamp, coded video data and coded audio data from coded video and audio data, a video buffer memory for storing the coded video data, an audio buffer memory for storing the coded audio data, a video and audio synchronization controller for generating video decoding start controlling signal and audio decoding start controlling signal to start decoding of the coded video data and the coded audio data base on the video and audio time stamps, a video decoder for decoding the coded video data, and an audio decoder for decoding the coded audio data; and

said video and audio synchronization controller includes;

a selector for selecting either a system clock reference (SCR) included in the frame header portion of the input coded video and audio data or a delayed coded video time stamp (V-TS) output from the video buffer memory;

a system time counter for generating a system time clock (STC) from the delayed coded video time stamp (V-TS) and the system clock reference (SCR);

an audio synchronization comparator for comparing the system time clock (STC) output from the system time counter and a delayed audio time stamp, and for outputting an audio decoding start controlling signal; and

a video synchronization comparator for comparing the system time clock (STC) output from the system time counter and the delayed video time stamp, and for outputting a video decoding start controlling signal;

wherein said video and audio synchronization controller sets a system clock reference (SCR) included in the frame header portion of the coded video and audio data into a system time counter to set a timing of a system time clock (STC), immediately after the video and audio reproducing device has been powered on, decodes and displays a first arriving video data synchronous with a display system frame pulse, which is the closest to the value of a video time stamp included in a frame header of the first video frame on the time axis of the set STC, sets again the value of the time stamp included in the first video frame to the system time counter to update the STC, outputs the video data for the second and subsequent video frames and all the audio data synchronously with the updated STC by outputting the video and audio data when the time indicated by the video and audio time stamps arrives on the time axis of the updated STC.

7. A video and audio reproducing device for synchronizing a decoded video output and a decoded audio output based on a video decoding start controlling signal and an audio decoding start controlling signal comprising:

a video and audio separator for separating a video time stamp, an audio time stamp, coded video data and coded audio data from coded video and audio data, a video buffer memory for storing the coded video data, an audio buffer memory for storing the coded audio data, a video and audio synchronization controller for generating video decoding start controlling signal and audio decoding start controlling signal to start decoding of the coded video data and the coded audio data base on the video and audio time stamps, a video decoder for decoding the coded video data, and an audio decoder for decoding the coded audio data; and

said video and audio synchronization controller includes;

a system time counter for generating a system time clock (STC) from the delayed coded video time stamp (V-TS);

an audio synchronization comparator for comparing the system time clock (STC) output from the system time counter and a delayed audio time stamp, and for outputting an audio decoding start controlling signal; and

a video synchronization comparator for comparing the system time clock (STC) output from the system time counter and the delayed video time stamp, and for outputting a video decoding start controlling signal;

wherein said video and audio synchronization controller decodes and displays a first arriving video frame immediately after the video and audio reproducing device has been powered on, regardless of the time stamp included in a header of the first arriving video frame, sets a time stamp included in the first arriving video frame to the system time counter while the first arriving video frame is displayed to set a timing of a system time clock (STC), outputs the video data for the second and subsequent video frames and all the audio data synchronously with the set STC by outputting the video and audio data when the time indicated by the video and audio time stamps arrives on the time axis of the set STC.

8. A video and audio reproducing device for synchronizing a decoded video output and a decoded audio output based on a video decoding start controlling signal and an audio decoding start controlling signal comprising:

a video and audio separator for separating a video time stamp, an audio time stamp, coded video data and coded audio data from coded video and audio data, a video buffer memory for storing the coded video data, an audio buffer memory for storing the coded audio data, a video and audio synchronization controller for generating video decoding start controlling signal and audio decoding start controlling signal to start decoding of the coded video data and the coded audio data base on the video and audio time stamps, a video decoder for decoding the coded video data, and an audio decoder for decoding the coded audio data, and

said device further comprises;

a video time stamp memory for storing the video time stamps separated by the audio and video separator;

a video frame address memory for sequentially storing the frame addresses of the coded video data written into the video buffer memory;

a video frame address comparator for comparing the head video frame address stored in the video frame address memory with the delayed video frame address read from the video buffer memory, and for outputting a video time stamp latch signal when they are matched with;

a video time stamp latch for latching the head video time stamp stored in the video time stamp memory by the input of the video time stamp latch signal from the video frame address comparator and for outputting the video time stamp;

an audio time stamp memory for storing the audio time stamps separated by the video and audio separator;

an audio frame address memory for sequentially storing the frame addresses of the coded audio data written into the audio buffer memory;

an audio frame address comparator for comparing the head audio frame address stored in the audio frame address memory with the delayed audio frame address read from the audio buffer memory, and for outputting an audio time stamp latch signal when they are matched with; and

an audio time stamp latch for latching the head audio time stamp stored in the audio time stamp memory by the input of the audio time stamp latch signal from the audio frame address comparator and for outputting the audio time stamp;

wherein on the video processing side, storing the separated video time stamps in the video time stamp memory, comparing a write address used for writing the coded video data to the video buffer memory with a read address used for reading an address at a video frame address comparator, reading and latching a head time stamp stored in the video time stamp memory, when the write address matches with the read address, supplying the latched time stamp to the video and audio synchronization controller as a video time stamp, and on the audio processing side, storing the separated audio time stamps in the audio time stamp memory, comparing a write address used for writing the coded audio data to the audio buffer memory with a read address used for reading an address temporarily stored in the audio buffer memory at an audio frame address comparator, reading and latching a head time stamp stored in the audio time stamp memory, when the write address matches with the read address, supplying the latched time stamp to the video and audio synchronization controller as an audio time stamp, and

said video and audio synchronization controller generates video and audio decoding starting signals based on the video time stamp and the audio stamp, respectively, and synchronizes the video output and the audio output by decoding the coded video data and the coded audio data based on the video decoding starting signal and the audio decoding starting signal, respectively.

9. A video and audio reproducing device for synchronizing a decoded video output and a decoded audio output based on a video decoding start controlling signal and an audio decoding start controlling signal comprising:

a video and audio separator for separating a video time stamp, an audio time stamp, coded video data and coded audio data from coded video and audio data, a video buffer memory for storing the coded video data, an audio buffer memory for storing the coded audio data, a video and audio synchronization controller for generating video decoding start controlling signal and audio decoding start controlling signal to start decoding of the coded video data and the coded audio data base on the video and audio time stamps, a video decoder for decoding the coded video data, and an audio decoder for decoding the coded audio data, and

said device further comprising;

a video time stamp memory for storing the video time stamps separated by the video and audio separator;

a video frame counter memory for sequentially storing the frame counts of the coded video data written into the video buffer memory;

a video frame count comparator for comparing the head frame count stored in the video frame counter memory with the delayed video frame count read from the video buffer memory, and for outputting the time stamp latch signal when they are matched with;

a video time stamp latch for latching the head video time stamp stored in the video time stamp memory by the video time stamp latch signal from the frame count comparator, and for outputting the video time stamp;

an audio time stamp memory for storing the audio time stamps separated by the video and audio separator;

an audio frame counter memory for sequentially storing the frame counts of the coded audio data written into the audio buffer memory;

an audio frame count comparator for comparing the head frame count stored in the audio frame counter memory with the delayed audio frame count read from the audio buffer memory, and for outputting the time stamp latch signal when they are matched with; and

an audio time stamp latch for latching the head audio time stamp stored in the audio time stamp memory by the audio time stamp latch signal from the frame count comparator, and for outputting the audio time stamp;

wherein on the video processing side, storing the separated video time stamps in the video time stamp memory, comparing a write video frame count used for writing the coded video data to the video buffer memory with a read video frame count of the read video frame at a video frame address comparator, reading and latching a head time stamp stored in the video time stamp memory, when the write video frame count matches with the read video frame count, supplying the latched time stamp to the video and audio synchronization controller as a video time stamp, and on the audio processing side, storing the separated audio time stamps in the audio time stamp memory, comparing a write audio frame count used for writing the coded audio data to the audio buffer memory with a read audio frame count of the read audio frame at an audio frame address comparator, reading and latching a head time stamp stored in the audio time stamp memory, when the write audio frame count matches with the read audio frame count, supplying the latched time stamp to the video and audio synchronization controller as an audio time stamp; and

said video and audio synchronization controller generates video and audio decoding starting signals based on the video time stamp and the audio stamp, respectively, and synchronizes the video output and the audio output by decoding the coded video data and the coded audio data based on the video decoding starting signal and the audio decoding starting signal, respectively.

10. A video and audio reproducing device for synchronizing a decoded video output and a decoded audio output based on a video decoding start controlling signal and an audio decoding start controlling signal comprising:

a video and audio separator for separating a video time stamp, an audio time stamp, coded video data and coded audio data from coded video and audio data, a video buffer memory for storing the coded video data, an audio buffer memory for storing the coded audio data, a video and audio synchronization controller for generating video decoding start controlling signal and audio decoding start controlling signal to start decoding of the coded video data and the coded audio data base on the video and audio time stamps, a video decoder for decoding the coded video data, and an audio decoder for decoding the coded audio data, and

said device further comprising;

a video frame count latch for latching the frame count of the coded video data written into the video buffer memory;

a frame count comparator for comparing the video frame count stored in the video frame count latch with the delayed video frame count read from the video buffer memory, and for outputting a video time stamp latch signal when they are matched with;

a video time stamp latch for latching the video time stamp separated by the video and audio separator, and for outputting the video time stamp latched by the video time stamp latch signal from the video frame count comparator;

an audio frame count latch for latching the frame count of the coded audio data written into the audio buffer memory;

a frame count comparator for comparing the audio frame count stored in the audio frame count latch with the delayed audio frame count read from the audio buffer memory, and for outputting an audio time stamp latch signal when they are matched with; and

an audio time stamp latch for latching the audio time stamp separated by the video and audio separator, and for outputting the audio time stamp latched by the audio time stamp latch signal from the audio frame count comparator;

wherein on the video processing side, latching the separated video time stamps in the video time stamp latch, comparing a write video frame count used for writing the coded video data to the video buffer memory with a read video frame count of the read video frame at a video frame address comparator, reading a time stamp latched in the video time stamp latch, when the write video frame count matches with the read video frame count, supplying the latched time stamp to the video and video synchronization controller as a video time stamp, and on the audio processing side, latching the separated audio time stamps in the audio time stamp latch, comparing a write audio frame count used for writing the coded audio data to the audio buffer memory with a read audio frame count of the read audio frame at an audio frame address comparator, reading a time stamp latched in the audio time stamp latch, when the write audio frame count matches with the read audio frame count, supplying the latched time stamp to the video and audio synchronization controller as an audio time stamp; and

said video and audio synchronization controller generates video and audio decoding starting signals based on the video time stamp and the audio stamp, respectively, and synchronizes the video output and the audio output by decoding the coded video data and the coded audio data based on the video decoding starting signal and the audio decoding starting signal, respectively.

11. A video decoder for decoding video coded data by a video decoding start controlling signal comprising:

a vbv separator for separating a vbv delay included in a header of coded video data, and for obtaining a corresponding vbv storage capacity; and

a vbv hold judgement comparator for comparing the vbv storage capacity with a current actual video buffer memory capacity, for sending an ON hold signal when the actual video buffer memory capacity<

vbv storage capacity, and for sending an OFF hold signal when the actual video buffer memory capacity≧

vbv storage capacity;

wherein said decoder holds the decoding when the ON hold signal is appeared regardless of the video decoding start controlling signal, and starts the decoding when the OFF hold signal is appeared.

12. The video decoder according to claim 11 further comprising:

an almost full skip judgement comparator for comparing the predetermined almost full threshold value and the current actual video buffer memory capacity;

wherein the almost full skip control signal is activated for skipping the next stage decoding when the actual buffer storage capacity is at least equal to the almost full threshold value, and non-activated for starting the next stage decoding according the decoding starting control signal when the actual buffer storage capacity is smaller than the almost full threshold value.

13. The video decoder according to claim 11 further comprising:

a flip-flop to be set when there is no time stamp in the video frame, and to be reset when a first video time stamp arrives;

wherein the decoding is controlled by the output signal from the vbv hold judgement comparator when the flip-flop is set, and the decoding is controlled by the video output starting control signal after the first video time stamp have arrived.

14. The video decoder according to claim 11, wherein said vbv hold judgement comparator comprises:

a first means for outputting the signal to recognize the first frame;

a selector for selecting a predetermined offset level (A) for preventing the video signal from being held by decoding timing jitter when the output of the first means have recognized the first frame, and for selecting zero offset level after the output of the first means have once recognized the first frame; and

an adder for adding the output of the selector and the vbv storage capacity output from the vbv separator;

wherein the comparator compares the output of the adder with the actual video buffer memory capacity, for the first frame, the comparator outputs an ON hold signal when the actual video buffer memory capacity<

output of the adder (=vbv storage capacity+A) is satisfied, and outputs an OFF hold signal when the actual video buffer memory capacity≧

the output of the adder (=vbv storage capacity+A) is satisfied, for the second and subsequent frames, the comparator outputs an ON hold signal when the actual video buffer memory capacity<

output of the adder (=vbv storage capacity) is satisfied, and outputs an OFF hold signal when the actual video buffer memory capacity≧

the output of the adder (=vbv storage capacity) is satisfied.