Method and apparatus for modifying encoded digital video for improved channel utilization

• US 5,956,088 A
• Filed: 11/21/1995
• Issued: 09/21/1999
• Est. Priority Date: 11/21/1995
• Status: Expired due to Term

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: Imedia Corp. (Motorola Solutions, Inc.)
• Inventors: Arazi, Efraim, Shen, Paul, Krause, Edward A., Tom, Adam S.
• Primary Examiner(s): Kostak, Victor R.
• Application Number:
  US08/561,296
• Time in Patent Office: 1,400 Days
• Field of Search: 348/385, 348/387, 348/409, 348/413, 348/415, 348/416, 348/845.2, 348/705, 348/390, 370/365, 370/368, 370/470, 370/232, 370/477, 370/517, 370/536-543
• US Class Current: 375/240.25
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/37 with arrangements for assigning different transmission priorities to video input data or to video coded data
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 21/23608 Remultiplexing multiplex streams, e.g. involving modifying time stamps or remapping the packet identifiers
• CPC Class Codes: H04N 21/23614 Multiplexing of additional data and video streams arrangements for simultaneous broadcast of plural pieces of information H04H20/28
• CPC Class Codes: H04N 21/2365 Multiplexing of several video streams
• CPC Class Codes: H04N 21/2401 Monitoring of the client buffer
• CPC Class Codes: H04N 21/4344 Remultiplexing of multiplex streams, e.g. by modifying time stamps or remapping the packet identifiers
• CPC Class Codes: H04N 21/4348 Demultiplexing of additional data and video streams
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 21/631 Multimode Transmission, e.g. transmitting basic layers and enhancement layers of the content over different transmission paths or transmitting with different error corrections, different keys or with different transmission protocols
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments

Abstract

A method for increasing channel utilization for a data channel transmitting a multiplex of a set of one or more encoded program streams, each program stream in being decodeable by a corresponding decoder, each corresponding decoder including a corresponding decoder buffer, the decoder buffers having a maximum allowable size, the method comprising selecting encoded pictures to be modified, said selecting according to a criterion which includes preventing any underflow of any decoder buffer, modifying each said selected encoded picture to form a corresponding modified encoded picture, said modified encoded picture having less data than said selected encoded picture, and transmitting the corresponding modified encoded pictures through the channel in place of the selected encoded pictures.

Claims

1. A method of increasing utilization of a first channel when transmitting a multiplex of a set of one or more encoded program streams, each program stream in said set being decodeable by a corresponding decoder, each corresponding decoder including a corresponding decoder buffer, said decoder buffers having a maximum allowable size, said method comprising:

(a) selecting from at least one of said encoded program streams encoded pictures to be modified that use bidirectional prediction, said selecting being carried out according to a criterion;

(b) modifing each said selected encoded picture by reducing prediction error data to form a corresponding modified encoded picture, said modified encoded picture having less data than said selected encoded picture; and

(c) transmitting the corresponding modified encoded pictures through said first channel in place of the selected encoded pictures;

wherein said criterion includes preventing any underflow of any decoder buffer.

2. The method of claim 1, wherein said step of modifying deletes each said selected encoded picture, said modified encoded picture having no data.

7. The method of claim 1, further comprising the step of:

(d) transmitting the selected encoded pictures.

8. The method of claim 7, wherein said step (d) of transmitting transmits through a second channel.

13. A method in accordance with claim 1, wherein the prediction error data of the selected encoded picture is completely reduced so that it is eliminated.

33. A method in accordance with claim 7, wherein said step (d) of transmitting transmits the selected encoded pictures through said first channel in available time intervals.

3. A method of increasing utilization of a first channel when transmitting a multiplex of a set of one or more encoded program streams, each program stream in said set being decodeable by a corresponding decoder, each corresponding decoder including a corresponding decoder buffer, said decoder buffers having a maximum allowable size, said method comprising:

(a) selecting encoded pictures to be modified, said selecting according to a criterion;

(b) modifying each said selected encoded picture to form a corresponding modified encoded picture, said modified encoded picture having less data than said selected encoded picture; and

(c) transmitting the corresponding modified encoded pictures through said first channel in place of the selected encoded pictures;

whereby said criterion includes preventing any underflow of any decoder buffer;

wherein said encoded program streams include predictively encoded pictures, each said predictively encoded picture having one or more corresponding anchor pictures, each said predictively encoded picture having prediction data and prediction error data, said step of selecting selects predictively encoded pictures that are not anchor pictures, and said step of modifying deletes the prediction error data from each selected encoded picture.

4. The method of claim 3, wherein said encoded program streams are MPEG encoded program streams, and said step of selecting selects B-pictures.

5. The method of claim 3, wherein said step of selecting includes a second criterion.

6. The method of claim 5, wherein said second criterion includes maintaining essentially constant perceived image quality.

9. A method of increasing utilization of a first channel when transmitting a multiplex of a set of one or more encoded program streams, each program stream in said set being decodeable by a corresponding decoder, each corresponding decoder including a corresponding decoder buffer, said decoder buffers having a maximum allowable size, said method comprising:

(a) selecting encoded pictures to be modified, said selecting according to a criterion;

(b) modifying each said selected encoded picture to form a corresponding modified encoded picture, said modified encoded picture having less data than said selected encoded picture;

(c) transmitting the corresponding modified encoded pictures through said first channel in place of the selected encoded pictures;

whereby said criterion includes preventing any underflow of any decoder buffer;

(d) transmitting the selected encoded pictures;

(e) determining whether it is possible to decode said selected encoded picture from said first channel; and

(f) whenever possible, decoding said selected encoded picture rather than the corresponding modified encoded picture.

10. The method of claim 9, wherein said encoded program streams include predictively encoded pictures, each said predictively encoded picture having one or more corresponding anchor pictures, each said predictively encoded picture having prediction data and prediction error data, said step of selecting selects predictively encoded pictures that are not anchor pictures, and said step of modification deleted the prediction error data from each said selected encoded picture.

11. The method of claim 10, wherein said encoded program streams are MPEG encoded program streams, and said step of selecting selects B-pictures.

12. The method of claim 11, wherein said step of determining includes comparing one or more time stamps of the selected encoded pictures and one or more time stamps of the corresponding modified encoded pictures.

14. A method of increasing utilization of a first channel when transmitting a multiplex of a set of one or more encoded program streams, each program stream in said set being decodeable by a corresponding decoder, each corresponding decoder including a corresponding decoder buffer, said decoder buffers having a maximum allowable size, said method comprising:

(a) selecting from at least one of said encoded program streams encoded pictures to be modified that use bidirectional prediction, said selecting being carried out according to a criterion that includes preventing any underflow of any decoder buffer;

(b) modifying each said selected encoded picture by reducing prediction error data included therein to form a corresponding modified encoded picture having less data than said selected encoded picture; and

(c) transmitting the corresponding modified encoded pictures through said first channel in place of the selected encoded pictures.

15. A method in accordance with claim 14, wherein the prediction error data of the selected encoded picture is completely reduced so that it is eliminated.

16. A method in accordance with claim 14, further comprising:

(d) transmitting the selected encoded pictures.

17. A method in accordance with claim 16, wherein said step (d) of transmitting transmits through a second channel.

18. A method in accordance with claim 16, further comprising:

(e) receiving the selected encoded pictures and the corresponding modified encoded pictures;

(f) determining whether to decode said selected encoded pictures; and

(g) decoding said selected encoded pictures in response to a positive determination in step (f).

34. A method in accordance with claim 16, wherein said step (d) of transmitting transmits the selected encoded pictures through said first channel in available time intervals.

19. A method of increasing utilization of a first channel when transmitting a multiplex of a set of one or more encoded program streams, each program stream in said set being decodeable by a corresponding decoder, each corresponding decoder including a corresponding decoder buffer, said decoder buffers having a maximum allowable size, said method comprising:

(a) selecting encoded pictures to be modified, said selecting being carried out according to a criterion that includes preventing any underflow of any decoder buffer;

(b) modifying each said selected encoded picture by reducing prediction error data included therein to form a corresponding modified encoded picture having less data than said selected encoded picture;

(c) transmitting the corresponding modified encoded pictures through said first channel in place of the selected encoded pictures;

(d) transmitting the selected encoded pictures;

(e) receiving the selected encoded pictures and the corresponding modified encoded pictures;

(f) determining whether to decode said selected encoded pictures; and

(g) decoding said selected encoded pictures in response to a positive determination in step (f);

wherein said step (f) of determining further includes,

comparing one or more time stamps associated with the selected encoded pictures with one or more time stamps associated with the corresponding modified encoded pictures.

20. An apparatus including an encoded digital video modification system for transmitting one or more encoded program streams which are decodeable by a corresponding decoder having a decoder buffer, the decoder buffers having a maximum allowable size, the encoded digital video modification system comprising:

a program analyzer and modifier configured to receive the one or more encoded program streams and to select encoded pictures to be modified according to a criterion that includes preventing any underflow of any decoder buffer, the program analyzer and modifier further configured to modify each said selected encoded picture to form a corresponding modified encoded picture having less data than said selected encoded picture;

a primary channel modulator, coupled to the program analyzer and modifier, configured to modulate the corresponding modified encoded pictures onto a primary channel;

an overflow channel modulator, coupled to the program analyzer and modifier, configured to modulate the selected encoded pictures onto an overflow channel; and

a combiner, coupled to the primary and overflow channel modulators, configured to combine the primary channel with the overflow channel.

21. An apparatus in accordance with claim 20, wherein the program analyzer and modifier modifies each said selected encoded picture by reducing prediction error data included therein.

22. An apparatus in accordance with claim 20, wherein the program analyzer and modifier modifies each said selected encoded picture by eliminating prediction error data included therein.

23. An apparatus in accordance with claim 20, further comprising:

a multiplexer, coupled to the encoded digital video modification system, configured to provide a multiplex of a set of the one or more encoded program streams to the encoded digital video modification system.

24. An apparatus in accordance with claim 20, further comprising a receiver for receiving the combined primary and overflow channels, the receiver comprising:

primary and overflow demodulators configured to demodulate data from the primary and overflow channels, respectively;

a first demultiplexer, coupled to the primary demodulator, configured to extract a primary packet stream from an output of the primary demodulator;

a second demultiplexer, coupled to the overflow demodulator, configured to extract an overflow packet stream from an output of the overflow demodulator;

a buffer coupled to the second demultiplexer;

a time stamp comparator, coupled to the first demultiplexer and the buffer, configured to compare a time stamp associated with a next packet from the primary packet stream with a time stamp associated with a next packet from the overflow packet stream; and

a packet multiplexer, coupled to the first demultiplexer, the buffer and the time stamp comparator, configured to select one of the next packets from the primary packet stream and the overflow packet stream in response to a comparison made by the time stamp comparator.

25. An apparatus in accordance with claim 24, wherein the time stamp comparator instructs the packet multiplexer to select the next packet from the primary packet stream when the time stamp associated with the next packet from the primary packet stream is less than the time stamp associated with the next packet from the overflow packet stream.

26. An apparatus in accordance with claim 25, wherein the next packet from the overflow packet stream is saved in the buffer for comparison with a following packet from the primary packet steam.

27. An apparatus in accordance with claim 24, wherein the time stamp comparator instructs the packet multiplexer to select the next packet from the overflow packet stream when the time stamp associated with the next packet from the primary packet stream is equal to the time stamp associated with the next packet from the overflow packet stream.

28. An apparatus including an encoded digital video modification system for transmitting one or more encoded program streams which are decodeable by a corresponding decoder having a decoder buffer, the decoder buffers having a maximum allowable size, the encoded digital video modification system comprising:

a program analyzer and modifier configured to receive the one or more encoded program streams and to select encoded pictures to be modified according to a criterion that includes preventing any underflow of any decoder buffer, the program analyzer and modifier further configured to modify each said selected encoded picture to form a corresponding modified encoded picture having less data than said selected encoded picture;

a primary packet storage device , coupled to the program analyzer and modifier, configured to store packets that include the corresponding modified encoded pictures;

an overflow packet storage device, coupled to the program analyzer and modifier, configured to store packets that include the selected encoded pictures; and

a packet multiplexer coupled to and configured to select packets from the primary and overflow packet storage devices and to provide an output multiplex of selected packets.

29. An apparatus in accordance with claim 28, wherein the program analyzer and modifier modifies each said selected encoded picture by reducing prediction error data included therein.

30. An apparatus in accordance with claim 28, wherein the program analyzer and modifier modifies each said selected encoded picture by eliminating prediction error data included therein.

31. An apparatus in accordance with claim 28, further comprising:

a multiplexer, coupled to the encoded digital video modification system, configured to provide a multiplex of a set of the one or more encoded program streams to the encoded digital video modification system.

32. An apparatus in accordance with claim 28, further comprising a receiver for receiving a modulation of the output multiplex of selected packets, the receiver comprising:

a demodulator configured to demodulate data from the modulation of the output multiplex of selected packets;

a first demultiplexer, coupled to the demodulator, configured to extract a primary packet stream from an output of the demodulator;

a second demultiplexer, coupled to the demodulator, configured to extract an overflow packet stream from an output of the demodulator;

a storage device coupled to the second demultiplexer;

a time stamp comparator, coupled to the first demultiplexer and the storage device, configured to compare a time stamp associated with a next packet from the primary packet stream with a time stamp associated with a next packet from the overflow packet stream; and

a packet multiplexer, coupled to the first demultiplexer, the storage device and the time stamp comparator, configured to select one of the next packets from the primary packet stream and the overflow packet stream in response to a comparison made by the time stamp comparator.