Method and system to replace sections of an encoded video bitstream

• US 6,026,232 A
• Filed: 07/13/1995
• Issued: 02/15/2000
• Est. Priority Date: 07/13/1995
• Status: Expired due to Term

• Current Assignee: Time Warner Entertainment Company LP (Charter Communications Incorporated), Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Time Warner Entertainment Company LP (Charter Communications Incorporated), Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Unno, Hiroaki, Ng, Sheau-Bao, Mimura, Hideki, Ichikawa, Teiichi, Kitamura, Tetsuya, Cookson, Christopher J., Yogeshwar, Jay, Thagard, Greg B., Rosen, Andrew Drusin
• Primary Examiner(s): Black, Thomas G.
• Assistant Examiner(s): Loomis, John C.
• Application Number:
  US08/502,012
• Time in Patent Office: 1,678 Days
• Field of Search: 395/615, 358/311, 358/537, 358/538, 358/539, 382/232, 382/235, 382/236, 382/251
• US Class Current: 715/719
• CPC Class Codes: G11B 2220/2545 CDs
• CPC Class Codes: G11B 2220/2562 DVDs [digital versatile discs]; Digital video discs; MMCDs; HDCDs
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/036 Insert-editing
• CPC Class Codes: G11B 27/329 on a disc [VTOC]
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D

Abstract

A method and system in which a section of a previously encoded digital bit stream is replaced with another section of encoded video. In order to remove a section of encoded video, it is necessary to determine the beginning and ending points of the encoded video in the digitally encoded bit stream. The addresses of the beginning and ending points of the encoded data cannot be simply looked up but are calculated by summing the bits consumed by each picture, the number of bits of the sequence header of each picture, the number of bits of each Group of Pictures (GOP) header, and all stuff bits. In order to encode the video which is to be inserted in place of the removed video, a process is performed which prevents decoding artifacts from appearing. This is accomplished by determining the last P-picture before the edit point and using it as a reference frame for frames after the edit point. Additionally, a short period of the original video encoded using its original quantizer values is included at the end of the video to be substituted. Alternatively, the encoder may be run so that an earlier encoded reference picture is not used. However, this requires extra bits to maintain picture quality because initial bidirectional frames of the substitute video will use only one reference frame instead of two.

Claims

1. A method for adding video to a bitstream of encoded video, comprising the steps of:

determining a start point and an end point at which the encoded video is to be modified;

obtaining unencoded video including video for insertion between the start point and end points, and video representing encoded video immediately after the end point;

encoding the unencoded video by performing the steps of;

encoding a first group of pictures (GOP) of the video for insertion without using reference frames outside of the start and end points, and allocating additional bits for bidirectionally predicted frames without a second reference frame;

encoding GOPs after the first GOP up to the end point of the video to be inserted; and

encoding the video representing the encoded video immediately after the end point; and

placing encoded video generated by each of said encoding steps into said encoded bitstream.

2. A method according to claim 1, further comprising the step of:

determining a number of said additional bits using a number of bits used for a frame having single direction prediction.

3. A method according to claim 1, wherein said step of encoding the video representing encoded video immediately after the end point comprises encoding the video representing the encoded video using quantizer values which are equal to quantizer values used during an original encoding of said unencoded video.

4. A method according to claim 1, wherein the determining step includes:

determining the start and end points by adding numbers of bits corresponding to video frames without referring to directory information of the encoded video.

5. A method according to claim 1, further comprising the steps of:

obtaining video using a camera;

encoding the obtained video to generate the bitstream of encoded data.

6. A memory containing data structures for storing the video of the encoded bitstream generated by the process of claim 1, comprising:

a data structure stored in said memory including;

a field for storing a number of menu cells, a cell being a unit of information;

a field for a start cell of a title menu;

a field for a start cell of an audio selection menu;

a field for a start cell of a menu used to select sub-picture information, the sub-picture information being used to present bit mapped information different from the video;

a field for a start cell for a menu used to select a plurality of programs, when a plurality of programs exist in the memory; and

a field for a start cell for an angle menu used to select a camera angle, when a plurality of angle views exist in the memory;

a menu cell information table containing table entries for each menu to be displayed, each of said table entries including;

a field indicating whether a menu corresponding to the table entry is a title menu, a program menu, an audio menu, a sub-picture menu, or an angle menu;

a field indicating a number of selection items;

a field describing at least a portion of a system clock reference of a start portion of the corresponding menu cell;

a field describing a start address of the corresponding menu cell; and

a field describing a number of blocks of the corresponding menu cell;

a data structure for storing information displayed for each menu;

a data structure for storing audio; and

a data structure for storing the video of the encoded bitstream generated by the process of claim 1.

7. A memory according to claim 6, wherein the memory is an optical disc.

8. A memory containing data structures for storing the video of the encoded bitstream generated by the process of claim 1, comprising:

a data structure for storing file management information, including;

a file management table storing address and size information of information in the memory;

a sequence information table storing information describing an order in which cells are reproduced, a cell being a unit of information, the sequence information table including a table entry for each sequence, each table entry including;

a field storing whether a corresponding sequence is a completion-type sequence which terminates after being reproduced, a head of a connection-type sequence which is part of sequences which are sequentially reproduced, a mid of a connection-type sequence, and an end of a connection-type sequence;

a field storing a number of cells in the corresponding sequence;

a field storing a reproduction time of the corresponding sequence; and

a field storing a sequence which is reproduced after the corresponding sequence;

a cell information table storing information describing cells, the cell information table including a table entry for each cell, each table entry including;

a field describing a reproduction time of a corresponding cell;

a field describing at least a portion of a system clock reference of the corresponding cell

a field describing an address of the corresponding cell; and

a field describing a number of logical blocks of the corresponding cell;

a data structure for storing audio; and

a data structure for storing the video of the encoded bitstream generated by the process of claim 1.

9. A memory according to claim 8, wherein the memory is an optical disc.

10. A memory containing data structures for storing the video of the encoded bitstream generated by the process of claim 1, the data structures also for storing audio, and descriptive information including timing information, comprising:

a data structure containing descriptive information, including;

a field for storing a system clock reference of the descriptive information data structure;

a field for storing a start time of a corresponding video picture which is decoded without reference to other pictures;

a field for storing a time stamp of audio information having a reproduction start time immediately before the reproduction start time of said corresponding video picture;

a field for storing an address of said audio information, a first bit indicating whether the audio information is located before or after the description information data structure; and

a field describing addresses of other descriptive information data structures occurring both before and after said descriptive information data structure;

a data structure for storing audio; and

a data structure for storing the video of the encoded bitstream generated by the process of claim 1.

11. A memory according to claim 10, wherein the memory is an optical disc.

12. A memory containing data structures for storing the video of the encoded bitstream generated by the process of claim 1, the data structures also for storing audio, and sub-picture information, comprising:

a data structure stored in said memory for storing a sub-picture unit, including;

a field for storing a sub-picture unit header;

a field for storing a bit mapped image; and

a field for storing a display control table including a plurality of table entries, each of said table entries including;

a sub-field which stores a display start time of the bit-mapped image;

a sub-field which stores an address of a subsequent table entry; and

a sub-field for storing at least one display control command selected from a group of commands which include a commands indicating a display area, pixel color, pixel contrast, and color and contrast changes of the pixels which occur as video frames are displayed;

a data structure for storing audio; and

a data structure for storing the video of the encoded bitstream generated by the process of claim 1.

13. A memory according to claim 12, wherein the sub-picture unit data structure comprises a plurality of packs containing information in the sub-picture unit data structure.

14. A memory according to claim 12, wherein the memory is an optical disc.

27. A memory containing data structures for storing data generated by the editing process of claim 15 which includes video frames, comprising:

a data structure stored in said memory including;

a field for storing a number of menu cells, a cell being a unit of information;

a field for a start cell of a title menu;

a field for a start cell of an audio selection menu;

a field for a start cell of a menu used to select sub-picture information, the sub-picture information being used to present bit mapped information different from the video frames;

a field for a start cell for a menu used to select a plurality of programs, when a plurality of programs exist in the memory; and

a field for a start cell for an angle menu used to select a camera angle, when a plurality of angle views exist in the memory;

a menu cell information table containing table entries for each menu to be displayed, each of said table entries including;

a field indicating whether a menu corresponding to the table entry is a title menu, a program menu, an audio menu, a sub-picture menu, or an angle menu;

a field indicating a number of selection items;

a field describing at least a portion of a system clock reference of a start portion of the corresponding menu cell;

a field describing a start address of the corresponding menu cell; and

a field describing a number of blocks of the corresponding menu cell;

a data structure for storing information displayed for each menu;

a data structure for storing audio; and

a data structure for storing said data edited according to claim 1 which includes the video frames.

28. A memory according to claim 27, wherein the memory is an optical disc.

29. A memory containing data structures for storing data generated by the editing process of claim 15 which includes video frames, comprising:

a data structure for storing file management information, including;

a file management table storing address and size information of information in the memory;

a sequence information table storing information describing an order in which cells are reproduced, a cell being a unit of information, the sequence information table including a table entry for each sequence, each table entry including;

a field storing whether a corresponding sequence is a completion-type sequence which terminates after being reproduced, a head of a connection-type sequence which is part of sequences which are sequentially reproduced, a mid of a connection-type sequence, and an end of a connection-type sequence;

a field storing a number of cells in the corresponding sequence;

a field storing a reproduction time of the corresponding sequence; and

a field storing a sequence which is reproduced after the corresponding sequence;

a cell information table storing information describing cells, the cell information table including a table entry for each cell, each table entry including;

a field describing a reproduction time of a corresponding cell;

a field describing at least a portion of a system clock reference of the corresponding cell

a field describing an address of the corresponding cell; and

a field describing a number of logical blocks of the corresponding cell;

a data structure for storing audio; and

a data structure for storing said data edited according to claim 1 which includes the video frames.

30. A memory according to claim 29, wherein the memory is an optical disc.

31. A memory containing data structures for storing data generated by the editing process of claim 15 which includes video frames, the data structures also for storing audio, and descriptive information including timing information, comprising:

a data structure containing descriptive information, including;

a field for storing a system clock reference of the descriptive information data structure;

a field for storing a start time of a corresponding video picture which is decoded without reference to other pictures;

a field for storing a time stamp of audio information having a reproduction start time immediately before the reproduction start time of said corresponding video picture;

a field for storing an address of said audio information, a first bit indicating whether the audio information is located before or after the description information data structure; and

a field describing addresses of other descriptive information data structures occurring both before and after said descriptive information data structure;

a data structure for storing audio; and

a data structure for storing said data edited according to claim 1 which includes the video frames.

32. A memory according to claim 31, wherein the memory is an optical disc.

33. A memory containing data structures for storing data generated by the editing process of claim 15 which includes video frames, the data structures also for storing audio, and sub-picture information, comprising:

a data structure stored in said memory for storing a sub-picture unit, including;

a field for storing a sub-picture unit header;

a field for storing a bit mapped image; and

a field for storing a display control table including a plurality of table entries, each of said table entries including;

a sub-field which stores a display start time of the bit-mapped image;

a sub-field which stores an address of a subsequent table entry; and

a sub-field for storing at least one display control command selected from a group of commands which include a commands indicating a display area, pixel color, pixel contrast, and color and contrast changes of the pixels which occur as video frames are displayed;

a data structure for storing audio; and

a data structure for storing said data edited according to claim 1 which includes the video frames.

34. A memory according to claim 33, wherein the sub-picture unit data structure comprises a plurality of packs containing information in the sub-picture unit data structure.

35. A memory according to claim 34, wherein the memory is an optical disc.

15. A method for editing a bitstream of encoded data, comprising the steps of:

determining a start point and an end point at which the encoded data is to be modified;

reconstructing reference data before the start point;

obtaining unencoded data including data representing the encoded data immediately after the end point;

encoding the unencoded data using the reference data which has been reconstructed; and

replacing the encoded data in the bitstream immediately after the end point with the data encoded in said encoding step.

16. A method according to claim 15, further comprising the step of:

obtaining unencoded data for insertion between the start point and end points,

wherein;

said encoding step includes encoding said unencoded data for insertion and encoding said unencoded data representing encoded data immediately after the end point, and said reference data is referred to during encoding of the unencoded data for insertion; and

said replacing step includes adding said encoded data for insertion between the start and end points.

17. A method according to claim 16, wherein said replacing step includes:

removing the encoded between the start and end points before adding said encoded data for insertion between the start and end points.

18. A method according to claim 15, further comprising the step of:

removing the encoded data between the start and end points.

19. A method according to claim 15, wherein said encoding step comprises encoding said unencoded data representing the encoded data immediately after the end point, using quantizer values which are equal to quantizer values used during an original encoding of said unencoded data.

20. A method according to claim 15, wherein said step of reconstructing reference data comprises:

constructing a predicted video frame immediately before the start point which is referred to by two bidirectionally predicted video frames after the start point.

21. A method according to claim 20, wherein said step of reconstructing reference data further comprises:

reconstructing the predicted video frame using quantizer values used to originally encode the predicted video frame.

22. A method according to claim 21, further comprising the step of:

determining said quantizer values used to originally encode the predicted frame by determining buffer information, an allocated number of bits, and an activity level of the predicted frame.

23. A method according to claim 21, further comprising the step of:

determining said quantizer values used to encode the predicted frame by looking said up information describing said predicted frame in a log file.

24. A method according to claim 21, further comprising the step of:

determining said quantizer values used to encode the predicted frame by looking said up said quantizer values in a file.

25. A method according to claim 15, wherein the determining step includes:

determining the start and end points by adding numbers of bits corresponding to video frames without referring to directory information of the encoded video.

26. A method according to claim 15, further comprising the steps of:

obtaining video using a camera;

encoding the obtained video to generate the bitstream of encoded data.

36. A system for adding video to a bitstream of encoded video, comprising:

means for determining a start point and an end point at which the encoded video is to be modified;

means for obtaining unencoded video including video for insertion between the start point and end points, and video representing encoded video immediately after the end point;

means for encoding the unencoded video, comprising;

means for encoding a first group of pictures (GOP) of the video for insertion without using reference frames outside of the start and end points, and allocating additional bits for bidirectionally predicted frames without a second reference frame;

means for encoding GOPs after the first GOP up to the end point of the video to be inserted; and

means for encoding the video representing the encoded video immediately after the end point; and

means for placing encoded video generated by each of said encoding steps into said encoded bitstream.

37. A system according to claim 36, further comprising:

means for determining a number of said additional bits using a number of bits used for a frame having single direction prediction.

38. A system according to claim 36, wherein said means for encoding the video representing encoded video immediately after the end point comprises means for encoding the video representing the encoded video using quantizer values which are equal to quantizer values used during an original encoding of said unencoded video.

39. A system according to claim 36, wherein the determining means includes:

means for determining the start and end points by adding numbers of bits corresponding to video frames without referring to directory information of the encoded video.

40. A system according to claim 36, further comprising:

a camera for obtaining video;

means for encoding the obtained video to generate the bitstream of encoded data.

41. A system for editing a bitstream of encoded data, comprising:

means for determining a start point and an end point at which the encoded data is to be modified;

means for reconstructing reference data before the start point;

means for obtaining unencoded data including data representing the encoded data immediately after the end point;

means for encoding the unencoded data using the reference data which has been reconstructed; and

means for replacing the encoded data in the bitstream immediately after the end point with the data encoded by said means for encoding.

42. A system according to claim 41, further comprising:

means for obtaining unencoded data for insertion between the start point and end points,

wherein;

said encoding means includes means for encoding said unencoded data for insertion and means for encoding said unencoded data representing encoded data immediately after the end point, and said reference data is referred to during encoding of the unencoded data for insertion; and

said replacing means includes means for adding said encoded data for insertion between the start and end points.

43. A system according to claim 41, wherein said replacing means includes:

means for removing the encoded between the start and end points before adding said encoded data for insertion between the start and end points.

44. A system according to claim 41, further comprising:

means for removing the encoded data between the start and end points.

45. A system according to claim 41, wherein said encoding means comprises means for encoding said unencoded data representing the encoded data immediately after the end point, using quantizer values which are equal to quantizer values used during an original encoding of said unencoded data.

46. A system according to claim 41, wherein said means for reconstructing reference data comprises:

means for constructing a predicted video frame immediately before the start point which is referred to by two bidirectionally predicted video frames after the start point.

47. A system according to claim 46, wherein said means for reconstructing reference data further comprises:

means for reconstructing the predicted video frame using quantizer values used to originally encode the predicted video frame.

48. A system according to claim 47, further comprising:

means for determining said quantizer values used to encode the predicted frame by looking said up information describing said predicted frame in a log file.

49. A system according to claim 47, further comprising:

means for determining said quantizer values used to encode the predicted frame by looking said up said quantizer values in a file.

50. A system according to claim 46, further comprising:

means for determining said quantizer values used to originally encode the predicted frame by determining buffer information, an allocated number of bits, and an activity level of the predicted frame.

51. A system according to claim 41, wherein the determining means includes:

means for determining the start and end points by adding numbers of bits corresponding to video frames without referring to directory information of the encoded video.

52. A system according to claim 41, further comprising:

a camera for obtaining video;

means for encoding the obtained video to generate the bitstream of encoded data.