Rational input buffer arrangements for auxiliary information in video and audio signal processing systems

• US 5,481,543 A
• Filed: 06/10/1994
• Issued: 01/02/1996
• Est. Priority Date: 03/16/1993
• Status: Expired due to Term

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Veltman, Markus H.
• Primary Examiner(s): Kizou, Hassan
• Application Number:
  US08/258,248
• Time in Patent Office: 571 Days
• Field of Search: 370/79, 370/84, 370/94.1, 370/94.2, 370/100.1, 370/105.3, 370/109, 370/110.1, 370/112, 370/118, 348/423, 348/461, 348/464, 348/465, 348/512, 348/518, 348/473, 348/474, 348/384, 348/390, 358/320, 358/337, 358/339, 360/36.1, 360/36.2, 375/106, 375/111, 375/118, 375/25, 381/29, 381/31
• US Class Current: 370/473
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/50 using predictive coding H04N19/61 takes precedence
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 21/23406 involving management of server-side video buffer
• CPC Class Codes: H04N 21/23614 Multiplexing of additional data and video streams arrangements for simultaneous broadcast of plural pieces of information H04H20/28
• CPC Class Codes: H04N 21/2368 Multiplexing of audio and video streams
• CPC Class Codes: H04N 21/2401 Monitoring of the client buffer
• CPC Class Codes: H04N 21/242 Synchronization processes, e.g. processing of PCR [Program Clock References] arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/4341 Demultiplexing of audio and video streams
• CPC Class Codes: H04N 21/4348 Demultiplexing of additional data and video streams
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 21/8455 involving pointers to the content, e.g. pointers to the I-frames of the video stream
• CPC Class Codes: H04N 5/9206 the additional signal being a character code signal
• CPC Class Codes: H04N 5/9264 using transform coding
• CPC Class Codes: H04N 5/9265 with processing of the sound signal
• CPC Class Codes: H04N 7/56 Synchronising systems therefor

Abstract

A method of generating a bit stream by multiplexing non-compressed auxiliary information with an information stream. The information stream is obtained by compressing fixed-size units of an information signal with a varying compression ratio to provide varying-sized units of the information stream. The auxiliary information is for use in subsequently processing the information stream. Units of the auxiliary information correspond to the units of the information signal. In the method, the information stream is divided in time into information stream portions. The non-compressed auxiliary information is also divided in time into auxiliary information portions. The information stream portions and the auxiliary information portions are interleaved to provide the bit stream. Finally, the information stream dividing, auxiliary information dividing, and interleaving steps are controlled by emulating decoding of the bit stream by a hypothetical system target decoder. The hypothetical system target decoder includes a demultiplexer that demultiplexes the bit stream, a serial arrangement of an information stream buffer and an information stream decoder, and a serial arrangement of an auxiliary information buffer and an auxiliary information processor. Each serial arrangement is connected to the demultiplexer. The information stream dividing, auxiliary information dividing, and interleaving steps are controlled such that the information stream buffer and the auxiliary information buffer neither overflow nor underflow.

Claims

1. A method of generating a bit stream by multiplexing non-compressed auxiliary information with an information stream, the information stream being obtained by compressing fixed-size units of an information signal with a varying compression ratio to provide varying-sized units of the information stream, the auxiliary information being for use in subsequently processing the information stream, the auxiliary information being composed of auxiliary information units, each of the auxiliary information units corresponding to one of the units of the information signal, the method comprising steps of:

dividing the information stream in time into information stream portions;

dividing the non-compressed auxiliary information in time into auxiliary information portions;

interleaving the information stream portions and the auxiliary information portions to provide the bit stream; and

controlling the information stream dividing, auxiliary information dividing, and interleaving steps by emulating decoding of the bit stream by a hypothetical system target decoder including demultiplexer means for demultiplexing the bit stream, a serial arrangement of an information stream buffer and an information stream decoder, and a serial arrangement of an auxiliary information buffer and an auxiliary information processor, each serial arrangement being connected to the demultiplexer means, the information stream dividing, auxiliary information dividing, and interleaving steps being controlled such that the information stream buffer and the auxiliary information buffer neither overflow nor underflow.

2. The method of claim 1, wherein, in the step of controlling the information stream dividing, auxiliary information dividing, and interleaving steps:

the demultiplexer means receives the bit stream and extracts therefrom the information stream and the auxiliary information for feeding to the information stream buffer and the auxiliary information buffer, respectively;

the information stream buffer has a first target size;

the auxiliary information buffer has a second target size;

the information stream decoder removes the varying-sized units of the information stream from the information stream buffer at a first target timing; and

the auxiliary information processor removes the corresponding auxiliary information units from the auxiliary information buffer at a second target timing.

3. The method of claim 2, wherein, in the interleaving step:

the bit stream comprises plural layers; and

the information stream portions and the auxiliary information portions are interleaved in the same one of the plural layers of the bit stream.

4. The method of claim 3, wherein the auxiliary information includes directory information for the information stream.

5. The method of claim 4, wherein:

the information stream includes plural access points, access points being ones of the varying-sized units of the information stream that are expandable without requiring expansion of others of the varying-sized units of the information stream; and

the directory information includes a unit of directory information relating to each of the access points.

6. The method of claim 5, wherein:

in the step of dividing the auxiliary information into auxiliary information portions, the directory information is divided into directory packets each including a number of the units of the directory information determined by the second target size;

in the step of dividing the information stream into information stream portions, each one of the information stream portions into which the information stream is divided includes the access points to which the directory information in a respective one of the directory packets relates; and

in the interleaving step, the respective one of the directory packets is interleaved adjacent the one of the information stream portions.

7. The method of claim 2, wherein, in the interleaving step:

the bit stream comprises plural layers; and

the information stream portions are interleaved in a first layer of the bit stream, and the auxiliary information portions are interleaved in a second layer of the bit stream, different from the first layer.

8. The method of claim 7, wherein the auxiliary information is a set of time stamps for decoding ones of the variable-sized units units of the information stream.

9. The method of claim 8, wherein:

in the controlling step, the auxiliary information buffer has an occupancy determined by the second target size, the auxiliary information fed from the demultiplexer means, and the auxiliary information removed by the auxiliary information processor;

the step of dividing the information stream into information stream portions divides the information stream into plural information packets;

the step of dividing the auxiliary information into auxiliary information portions results in auxiliary information portions each composed of one of the time stamps;

the step of interleaving the information stream additionally includes a step of generating an information packet header for each of the information packets; and

in the step of interleaving the information stream portions and the auxiliary information portions, one of the auxiliary information portions is inserted into the information packet header of ones of the information packets selected according to the occupancy of the auxiliary information buffer.

10. The method of claim 8, wherein:

in the controlling step, the auxiliary information buffer has an occupancy determined by the second target size, the auxiliary information fed from the demultiplexer means, and the auxiliary information removed by the auxiliary information processor;

the step of dividing the information stream into information stream portions divides the information stream into plural information packets;

the step of dividing the auxiliary information into auxiliary information portions results in auxiliary information portions each composed of one of the time stamps;

the step of interleaving the information stream portions additionally includes a step of generating an information packet header for each of the information packets;

in the step of interleaving the information stream portions and the auxiliary information portions, the auxiliary information portions are periodically inserted into the information packet header of the information packets at a time stamp coding frequency; and

in the controlling step, at least one of the time stamp coding frequency and the second target size is controlled in such a manner that maximizes the occupancy of the information stream buffer without causing the information stream buffer to overflow.

11. The method of claim 7, wherein:

the information stream decoder is one of plural information stream decoders, the information stream decoders being phase locked; and

the auxiliary information buffer has a size set to accommodate one and no more than one auxiliary information unit.

12. An encoder for generating a bit stream, the encoder comprising:

means for compressing fixed-size units of an information signal with a varying compression ratio to provide varying-sized units of an information stream;

information stream dividing means for dividing the information stream in time into information stream portions;

auxiliary information dividing means for dividing non-compressed auxiliary information in time into auxiliary information portions, the auxiliary information being for use in subsequently processing the information stream, the auxiliary information being composed of auxiliary information units, each of the auxiliary information units corresponding to one of the units of the information signal;

multiplexing means for sequentially arranging the information stream portions and the auxiliary information portions to provide the bit stream, the multiplexing means including a control means for controlling the information stream dividing means and the auxiliary information dividing means by emulating decoding of the bit stream by a system target decoder including demultiplexer means for demultiplexing the bit stream, a serial arrangement of an information stream buffer and an information stream decoder, and a serial arrangement of an auxiliary information buffer and an auxiliary information processor, each of the serial arrangements being connected to the demultiplexer means, the control means controlling the information stream dividing means and the auxiliary information dividing means such that the information stream buffer and the auxiliary information buffer neither underflow nor overflow.

13. The encoder of claim 12, wherein:

the demultiplexer means receives the bit stream and extracts therefrom the information stream and the auxiliary information for feeding to the information stream buffer and the auxiliary information buffer, respectively;

the information stream buffer has a first target size;

the auxiliary information buffer has a second target size;

the information stream decoder removes the varying-sized units of the information stream from the information stream buffer at a first timing; and

the auxiliary information processor removes the corresponding auxiliary information units from the auxiliary information buffer at a second target timing.

14. The encoder of claim 12, wherein:

the bit stream provided by the multiplexing means comprises plural layers; and

the multiplexing means arranges the information stream portions and the auxiliary information portions in the same one of the plural layers of the bit stream.

15. The system of claim 12, wherein:

the bit stream provided by the multiplexing means comprises plural layers; and

the multiplexing means arranges the information stream portions in a first layer of the bit stream and arranges the auxiliary information portions in a second layer of the bit stream, different from the first layer.

16. A system wherein an information signal is compressed for transfer, together with non-compressed auxiliary information, to a medium as a bit stream, and wherein the bit stream is transferred from the medium and is processed to recover the information signal by expansion, and to recover the auxiliary information, the auxiliary information being for use in subsequently processing the information signal, the system comprising:

an encoder comprising;

means for compressing the information signal to provide an information stream, fixed-sized units of the information signal being compressed using a varying compression ratio to provide varying-sized units of the information stream, and

multiplexing means for sequentially arranging time-divided portions of the information stream and time-divided portions of the non-compressed auxiliary information to provide the bit stream for transfer to the medium, the multiplexing means including control means for determining a division of the information stream and of the auxiliary information into the respective time-divided portions by emulating decoding of the bit stream by a system target decoder including demultiplexer means for demultiplexing the bit stream, a serial arrangement of an information stream buffer and an information stream decoder, and a serial arrangement of an auxiliary information buffer and an auxiliary information processor, each of the serial arrangements being connected to the demultiplexer means, the information stream buffer and the auxiliary information buffer each having a size; and

a decoder, comprising;

demultiplexing means for extracting the information stream and the auxiliary information from the bit stream transferred from the medium,

first input buffer means for receiving the auxiliary information from the demultiplexing means, the first input buffer means having a size of at least the size of the auxiliary information buffer,

means for removing each of the auxiliary information units from the first input buffer means,

second input buffer means for receiving the information stream from the demultiplexing means, the second input buffer means having a size of at least the size of the information stream buffer, and

decoder means for removing each one of the varying-sized units of the information stream from the second input buffer means, and for expanding the one of the varying-sized units of the information stream removed from the second input buffer means to recover a respective one of the fixed-sized units of the information signal.

17. The system of claim 16, wherein the control means determines the division of the information stream and of the auxiliary information into the respective time-divided portions such that the bit stream, when subject to the emulated decoding by the system target decoder causes the information stream buffer and the auxiliary information buffer neither to underflow nor overflow.

18. The system of claim 16, wherein:

the bit stream provided by the multiplexing means has plural layers; and

the multiplexing means arranges the time-divided portions of the information stream and of the non-compressed auxiliary information in the same one of the plural layers of the bit stream.

19. The system of claim 18, wherein the auxiliary information is directory information relating to the information stream.

20. The system of claim 19, wherein:

the information stream includes plural access points, access points being ones of the varying-sized units of the information stream that are expandable without requiring expansion of others of the varying-sized units of the information stream; and

the directory information includes a unit of directory information relating to each one of the access points.

21. The system of claim 19, wherein:

the control means determines a division of the directory information into directory packets each including a number of units of the directory information, and determines a division of the information stream into the information stream portions such that each one of the information stream portions into which the information stream is divided includes the access points to which the directory information in a respective one of the directory;

packets relates; and

the multiplexing means multiplexes the respective one of the directory packets adjacent the one of the information stream portions.

22. The system of claim 16, wherein:

the bit stream provided by the multiplexing means has plural layers; and

the multiplexing means arranges the information stream portions in a first layer of the bit stream and arranges the auxiliary information portions in a second layer of the bit stream, different from the first layer.

23. The system of claim 22, wherein the auxiliary information is a set of time stamps for decoding ones of the varying-sized units of the information stream.

24. The system of claim 23, wherein:

the auxiliary information buffer has an occupancy determined by the size of the auxiliary information buffer, the auxiliary information fed from the demultiplexer means, and the auxiliary information removed by the auxiliary information processor;

the control means is for;

determining a division of the information stream into plural information packets and generating an information packet header for each of the information packets,

determining a division of the set of time stamps into auxiliary information portions each composed of one of the time stamps;

periodically inserting ones of the auxiliary information portions into the information packet header of selected ones of the information packets at a time stamp coding frequency; and

controlling at least one of the time stamp coding frequency and the size of the auxiliary information buffer in such a manner that maximizes the occupancy of the information stream buffer without causing the information stream buffer to overflow.

25. A method of deriving a bit stream from an information signal, the method comprising steps of:

compressing units of the information signal to provide units of an information stream, the units of the information stream including access points, access points being ones of the units of the information stream that are expandable without requiring expansion of others of the units of the information stream;

deriving pointers from the information stream, each of the pointers pointing to one of the access points in the information stream; and

multiplexing the information stream with the pointers to provide the bit stream, the information stream being multiplexed with the pointers by;

generating information packets each including a portion of the information stream,

generating pointer packets each including different ones of the pointers, and

locating a set of the information packets containing plural consecutive ones of the access points adjacent a one of the pointer packets containing ones of the pointers pointing only to the plural consecutive ones of the access points.

26. The method of claim 25, wherein the multiplexing step includes steps of:

generating dummy pointers, and generating pointer packets each including the dummy pointers;

multiplexing the information packets with the pointer packets including the dummy pointers prior to the deriving step; and

overwriting the dummy pointers in each of the pointer packets in the bit stream with ones of the pointers derived in the deriving step, the ones of the pointers overwritten into each one of the pointer packets being the ones of the pointers pointing to the plural consecutive ones of the access points immediately preceding the one of the pointer packets in the bit stream.

27. A method of deriving a bit stream from an information signal, the method comprising steps of:

providing an encoder including;

means for compressing units of the information signal to provide units of an information stream,

first buffer means, having a size, for buffering the units of the information stream,

means for generating a time stamp when the first buffer means receives each of the units of the information stream,

second buffer means, having a size, for buffering the time stamps, and

multiplexing means for multiplexing the information stream from the first buffer means and the time stamps from the second buffer means to provide the bit stream;

defining a hypothetical system target decoder, the hypothetical system target decoder including demultiplexer means for demultiplexing the bit stream, a serial arrangement of an information stream buffer and an information stream decoder, and a serial arrangement of a time stamp buffer and a time stamp processor, each serial arrangement being connected to the demultiplexer means;

determining a first size for the first buffer means and a second size for the second buffer means by emulating decoding of the bit stream using the hypothetical system target decoder; and

encoding the information signal using the encoder with the size of the first buffer means and the size of the second buffer means respectively set to the first size and the second size determined by the determining step.

28. The method of claim 27, wherein:

in the step of defining the system target decoder;

the information stream buffer and the time stamp buffer each have a size, and

the information stream decoder decodes the information stream in response to ones of the time stamps removed from the time stamp buffer by the time stamp processor; and

in the determining step, the first size for the first buffer means and the second size for of the second buffer means are determined from the size of the information stream buffer and the size of the time stamp buffer.

29. The method of claim 28, wherein:

in the encoder, the multiplexing means periodically inserts the time stamps into the bit stream at a time stamp coding frequency;

the information stream has a bit rate; and

the determining step includes steps of;

determining a buffering delay from the time stamp coding frequency and the bit rate, and

determining the first size for the first buffer means and the second size for the second buffer means from the buffering delay.

30. A decoder for a bit stream obtained by multiplexing non-compressed auxiliary information with an information stream, the information stream being obtained by compressing fixed-size units of an information signal with a varying compression ratio to provide varying-sized units of the information stream, the auxiliary information being for use in subsequently processing the information stream, the auxiliary information including auxiliary information units, each of the auxiliary information units corresponding to one of the units of the information signal, the decoder comprising:

demultiplexing means for extracting the information stream and the auxiliary information from the bit stream;

first input buffer means for receiving the auxiliary information from the demultiplexing means;

removing means for removing each of the auxiliary information units from the first input buffer means;

second input buffer means for receiving the information stream from the demultiplexing means; and

decoder means for removing one of the varying-sized units of the information stream from the second input buffer means in response to each of the auxiliary information units removed from the first input buffer means by the removing means, and for expanding the one of the varying-sized units removed from the second input buffer means to recover a respective one of the fixed-size units of the information signal.

31. The decoder of claim 30, wherein the decoder means removes the one of the varying-sized units of the information stream from the second input buffer means at a time indicated by each of the auxiliary information units removed from the first input buffer means by the removing means.