Method and apparatus for preserving synchronization of audio and video presentation when splicing transport streams

• US 6,181,383 B1
• Filed: 05/28/1997
• Issued: 01/30/2001
• Est. Priority Date: 05/29/1996
• Status: Expired due to Term

• Current Assignee: Sarnoff Corporation (SRI International, Inc.)
• Original Assignee: Sarnoff Corporation (SRI International, Inc.)
• Inventors: Fox, Edward C., Wine, Charles, Lyons, Paul Wallace
• Primary Examiner(s): Peng, John K.
• Assistant Examiner(s): DESIR, JEAN WICEL
• Application Number:
  US08/864,321
• Time in Patent Office: 1,343 Days
• Field of Search: H04/N.9475, H04/N.708, H04/N.504, 348/515-520, 348/512, 348/500, 348/584, 348/423, 348/426, 348/845.3, 386/39, 386/96, 386/98, 386/99, 386/104-106, 370/537, 370/535, 370/474
• US Class Current: 348/515
• CPC Class Codes: H04N 19/179 the unit being a scene or a shot
• CPC Class Codes: H04N 19/87 involving scene cut or scene change detection in combination with video compression
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/23608 Remultiplexing multiplex streams, e.g. involving modifying time stamps or remapping the packet identifiers
• CPC Class Codes: H04N 21/2365 Multiplexing of several video streams
• CPC Class Codes: H04N 21/2368 Multiplexing of audio and video streams
• CPC Class Codes: H04N 21/242 Synchronization processes, e.g. processing of PCR [Program Clock References] arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18
• CPC Class Codes: H04N 21/4302 Content synchronisation processes, e.g. decoder synchronisation
• CPC Class Codes: H04N 21/4305 Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/4341 Demultiplexing of audio and video streams
• CPC Class Codes: H04N 21/4344 Remultiplexing of multiplex streams, e.g. by modifying time stamps or remapping the packet identifiers
• CPC Class Codes: H04N 21/4347 Demultiplexing of several video streams
• CPC Class Codes: H04N 21/44016 involving splicing one content stream with another content stream, e.g. for substituting a video clip
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content

Abstract

A method and apparatus for preserving audio and video presentation synchronization during the splicing operation by selectively deleting, if necessary, an audio/video access unit to avoid overlapping of audio/video frames in the spliced output stream.

Claims

1. A method for preserving lip sync during a splicing operation where a plurality of data streams is multiplexed into an output stream, where each of the data streams contains an audio stream having a plurality of audio access units and a video stream having a plurality of video access units that are in lip sync, said method comprising the steps of:

(a) splicing a portion of a first data stream into the output stream;

(b) splicing a portion of a second data stream into the output stream;

(c) determining at a splice point whether an overlap condition exists between a last audio access unit from said first data stream and a first audio access unit from said second data stream; and

(d) deleting either said last audio access unit or said first audio access unit if an overlap condition exists.

2. The method of claim 1, wherein said data streams are transport streams.

3. The method of claim 1, wherein said determining step (c) determines said overlap condition in accordance with:

4. The method of claim 1, further comprises the step of:

(b′

) storing said portions of said first and second data streams in a buffer.

5. The method of claim 1, wherein said splicing step (b) is premised on maintaining picture presentation continuity.

6. A method for preserving lip sync during a splicing operation where a plurality of data streams is multiplexed into an output stream, where each of the data streams contains an audio stream having a plurality of audio access units and a video stream having a plurality of video access units that are in lip sync, said method comprising the steps of:

(a) splicing a portion of a first data stream into the output stream;

(b) splicing a portion of a second data stream into the output stream;

(c) determining at a splice point whether an overlap condition exists between a last video access unit from said first data stream and a first video access unit from said second data stream; and

(d) deleting either said last video access unit or said first video access unit if an overlap condition exists.

7. The method of claim 6, wherein said data streams are transport streams.

8. The method of claim 6, wherein said determining step (c) determines said overlap condition in accordance with:

9. The method of claim 6, further comprises the step of:

(b′

) storing said portions of said first and second data streams in a buffer.

10. The method of claim 6, wherein said splicing step (b) is premised on maintaining audio presentation continuity.

11. An apparatus for preserving lip sync during a splicing operation where a plurality of data streams is multiplexed into an output stream, where each of the data streams contains a first substream having a plurality of access units and a second substream having a plurality of access units, said apparatus comprising:

a splicer for splicing a portion of a first data stream and a portion of a second data stream into the output stream;

a means, coupled to said splicer, for determining at a splice point whether an overlap condition exists between said access units from said first data stream and said second data stream; and

a means, coupled to said determining means, for deleting one or more of said access units if an overlap condition exists.

12. The apparatus of claim 11, wherein said first substream is an audio stream having a plurality of audio access units and said second substream is a video stream having a plurality of video access units that are in lip sync to said audio access units.

13. The apparatus of claim 12, wherein said deleted access unit is an overlapping audio access unit.

14. The apparatus of claim 12, wherein said deleted access unit is an overlapping video access unit.

15. The apparatus of claim 13, wherein said overlap condition is determined in accordance with:

16. The apparatus of claim 14, wherein said overlap condition is determined in accordance with:

17. The apparatus of claim 11, wherein said determining means further comprises:

a buffer for storing said portions of said first and second data streams.

18. The apparatus of claim 11, wherein said splicer operates in accordance with maintaining picture presentation continuity.

19. The apparatus of claim 11, wherein said splicer operates in accordance with maintaining audio presentation continuity.