Method for generating and processing transition streams

• US 6,909,743 B1
• Filed: 10/29/1999
• Issued: 06/21/2005
• Est. Priority Date: 04/14/1999
• Status: Active Grant

• Current Assignee: MediaTek, Inc.
• Original Assignee: Sarnoff Corporation (SRI International, Inc.)
• Inventors: Ward, Christopher, Hurst, Robert Norman
• Primary Examiner(s): An, Shawn S.
• Application Number:
  US09/430,631
• Time in Patent Office: 2,062 Days
• Field of Search: 375/240, 375/240.01, 375/240.25, 375/240.26, 375/240.28, 348/722, 348/400, 348/402, 348/411, 348/415, 348/584, 348/473, 348/409, 348/419, 348/700
• US Class Current: 375/240.01
• CPC Class Codes: G11B 2220/41 Flat as opposed to hierarchical combination, e.g. library of tapes or discs, CD changer, or groups of record carriers that together store one title
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: H04N 19/00 Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 21/23406 involving management of server-side video buffer
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/23608 Remultiplexing multiplex streams, e.g. involving modifying time stamps or remapping the packet identifiers
• CPC Class Codes: H04N 21/23655 Statistical multiplexing, e.g. by controlling the encoder to alter its bitrate to optimize the bandwidth utilization
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/44016 involving splicing one content stream with another content stream, e.g. for substituting a video clip
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content

Abstract

A method for generating a transition stream and processing video, audio or other data within the transition stream using, respectively, pixel domain processing, audio domain processing or other data domain processing.

Claims

1. A method for generating a transition stream for transitioning from a first transport stream to a second transport stream in a substantially seamless manner, said method comprising the steps of:

decoding a portion of said first transport stream including at least a target out-frame representing a last image frame of said first transport stream to be presented;

decoding a portion of said second transport stream including at least a target in-frame representing a first image frame of said second transport stream to be presented;

processing, using a pixel domain process, at least one of said decoded image frames; and

encoding a plurality of said decoded image frames, including said target out-frame and said target in-frame, to produce said transition stream.

2. The method of claim 1, wherein said pixel domain process comprises at least one of a morph, fade, wipe, dissolve, push, reveal, black-frame, freeze-frame and chroma-keying pixel domain process.

3. The method of claim 1, further comprising the steps of:

extracting, from said first and second transport streams, audio data associated with said video frames used to form said transition stream; and

inserting, into said transition stream, said extracted audio data.

4. The method of claim 1, further comprising the steps of:

extracting, from said first and second transport streams, non-video data associated with said video frames used to form said transition stream; and

inserting, into said transition stream, said extracted non-video data.

5. The method of claim 4, wherein said non-video data comprises at least one of audio data, meta-data, data essence, ancillary data and auxiliary data.

6. The method of claim 3, further comprising the step of:

processing, using an audio domain process, at least a portion of said extracted audio data.

7. The method of claim 4, further comprising the step of:

processing, using an non-video domain process, at least a portion of said extracted non-video data.

8. The method of claim 4, wherein said step of encoding said plurality of decoded image includes the step of transport encoded said encoded plurality of image frames, said method further comprising the steps of:

reserving a plurality of transport packets within said transition stream, said reserved packets not being utilized to store encoded image information;

utilizing at least a portion of said reserved plurality of transport packets to store said extracted audio data.

9. The method of claim 5, wherein said step of encoding said plurality of decoded image includes the step of transport encoded said encoded plurality of image frames, said method further comprising the steps of:

reserving a plurality of transport packets within said transition stream, said reserved packets not being utilized to store encoded image information;

utilizing at least a portion of said reserved plurality of transport packets to store said extracted non-video data.

10. The method of claim 4, wherein said first transport stream and said second transport stream are multiplexed into respective first and second multiple program transport streams, said method further comprising the step of:

determining, for each multiple program transport stream including a transport stream to be processed, a maximum extent of all image frames to be included in a transition stream; and

demultiplexing each multiple program transport stream to accommodate its respective determined maximum extent.

11. The method of claim 10, wherein said step of determining said image data extent includes the step of determining a maximum extent of all non-video data associated with image frames to be included in a transition stream, said maximum extent comprising a combination of the image data extent and the non-video data extent.

12. The method of claim 1, wherein first portion of said first transport stream to be decoded is determined according to the steps of:

decoding, in display order, said out-frame and all preceding frames from said out-frame to a prior I-frame;

in the case of said out-frame comprising a B-frame, defining the frame immediately preceding said prior I-frame in transmission order as a final first transport stream frame in said transition stream;

in the case of said out-frame not comprising a B-frame, defining said out-frame as said final first transport stream frame in said transition stream.

13. The method of claim 1, wherein said first portion of said second transport stream to be decoded is determined according to the steps of:

decoding, in display order, said in-frame and all non-I-frames following said in-frame up to a next I-frame.

14. The method of claim 1, further comprising the step of indexing each of said first and second transport streams, said step of indexing comprising the steps of:

parsing a transport layer of a stream to be indexed to identify packets associated with at least one of sequence headers, picture headers and predefined splicing syntax;

determining, for each frame in said stream to be indexed, at least one of a picture number, a picture coding type, a start of frame transport packet number, an end of frame transport packet number, a presentation time stamp (PTS) and a decode time stamp (DTS).

15. The method of claim 4, wherein said determinations for each frame are stored in a meta file for subsequent use in generating a transition stream.

16. The method of claim 1, wherein each of said first and second transport streams are associated with a respective video buffering verifier (VBV) parameter, said method further comprising the step of:

determining if a difference exists between said from-stream VBV parameter and said to-stream VBV parameter; and

adapting, in response to said determination, step of re-encoding.

17. The method of claim 16, wherein said step of adapting comprises the steps of:

increasing a rate control bit allocation in response to a determination that said from-stream VBV parameter exceeds said to-stream VBV parameter by a first threshold level; and

decreasing said rate control bit allocation in response to a determination that said to-stream VBV parameter exceeds said from-stream VBV parameter by a second threshold level.

18. A method for generating a transition stream for transitioning from a first transport stream to a second transport stream in a substantially seamless manner, said method comprising the steps of:

decoding a portion of said first transport stream including at least a target out-frame representing a last image frame of said first transport stream to be presented, said decoded portion including non-video data associated with at least one decoded image frame of said first transport stream;

decoding a portion of said second transport stream including at least a target in-frame representing a first image frame of said second transport stream to be presented, said decoded portion including non-video data associated with at least one decoded image frame of said second transport stream;

encoding a plurality of said decoded image frames, including said target out-frame and said target in-frame, to produce said transition stream; and

inserting, into said transition stream, said extracted non-video data.

19. The method of claim 18, wherein said non-video data comprises at least one of audio data, meta-data, data essence, ancillary data and auxiliary data.

20. The method of claim 18, further comprising the step of:

processing, using a pixel domain process, at least a portion of said extracted audio data.

21. The method of claim 20, wherein said pixel domain processing comprises at least one of a morph, fade, wipe, dissolve, push, reveal, black-frame and freeze-frame pixel domain process.

22. The method of claim 18, further comprising the step of:

processing, using an non-video domain process, at least a portion of said extracted non-video data.

23. The method of claim 18, wherein said step of encoding said plurality of decoded image includes the step of transport encoded said encoded plurality of image frames, said method further comprising the steps of:

reserving a plurality of transport packets within said transition stream, said reserved packets not being utilized to store encoded image information;

utilizing at least a portion of said reserved plurality of transport packets to store said extracted non-video data.

24. The method of claim 18, wherein said generated transition stream is constrained to a pre-determined number of image frames.

25. The method of claim 18, wherein each of said first and second transport streams are associated with a respective video buffering verifier (VBV) parameter, said method further comprising the step of:

determining if a difference exists between said from-stream VBV parameter and said to-stream VBV parameter; and

adapting, in response to said determination, step of re-encoding.

26. The method of claim 25, wherein said step of adapting comprises the steps of:

increasing a rate control bit allocation in response to a determination that said from-stream VBV parameter exceeds said to-stream VBV parameter by a first threshold level; and

decreasing said rate control bit allocation in response to a determination that said to-stream VBV parameter exceeds said from-stream VBV parameter by a second threshold level.