Elementary stream multiplexer

• US 6,490,250 B1
• Filed: 03/09/1999
• Issued: 12/03/2002
• Est. Priority Date: 03/09/1999
• Status: Expired due to Term

• Current Assignee: Quarterhill Inc. (f/k/a Wi-LAN Inc.)
• Original Assignee: Conexant Systems Incorporated (Synaptics Incorporated)
• Inventors: Kizhepat, Govind, Hinchley, Ronald E., Lowe, Phillip
• Primary Examiner(s): Kizou, Hassan
• Assistant Examiner(s): ODLAND, DAVID E
• Application Number:
  US09/265,486
• Time in Patent Office: 1,365 Days
• Field of Search: 370/230, 370/231, 370/232, 370/233, 370/234, 370/235, 370/506, 370/537, 370/538, 370/545, 370/465, 725/148, 725/149, 709/231, 709/233, 709/234
• US Class Current: 370/232
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/127 Prioritisation of hardware or computational resources
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/162 User input
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/177 the unit being a group of pictures [GOP]
• CPC Class Codes: H04N 19/42 characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/23655 Statistical multiplexing, e.g. by controlling the encoder to alter its bitrate to optimize the bandwidth utilization
• CPC Class Codes: H04N 21/2368 Multiplexing of audio and video streams
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/4341 Demultiplexing of audio and video streams
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream

Abstract

An integrated multimedia encoding system is disclosed. Multimedia encoders which are capable of adjusting bit rates receive multimedia data to compress the data. After compressing the data, the multimedia encoders adjust the bit rates of the elementary streams responsive to a control input. Bit rates are increased or decreased using delays or, for video data, by allocating more or less bits to each macroblock, frame or group of frames. A unified memory module is coupled to the multimedia encoders to store the multimedia elementary stream data, the Program or Transport stream data, and data from other sources as needed. The unified memory is capable of adjusting storage allocations responsive to the realtime requirements of the incoming multimedia streams and the outgoing Program or Transport stream data. A stream processor is coupled to the unified memory module and the multimedia encoders for multiplexing the elementary streams into a single stream, and monitoring the actual bit rate of the combined multimedia stream. Monitoring the actual bit rate as a function of number of bits passed over a period of time provides accurate feedback as to the system throughput. A multimedia processor then determines the bit rates of the elementary streams, and generating a control signal to adjust the bit rates of the encoder to ensure that an optimal bit rate is continuously achieved by the system. The stream processor also operates using dedicated instructions which allow the stream processor to efficiently multiplex the incoming streams together.

Claims

1. In a system for combining input multimedia data streams to form an output multimedia data stream, an apparatus for adjusting rates of the input multimedia data streams comprising:

a data rate analyzer, coupled to the output multimedia data stream, for determining the data rate of the output multimedia data stream; and

a multimedia processor, coupled to the data rate analyzer, for comparing the determined output multimedia data stream data rate to a target output data stream data rate, and generating rate control signals for adjusting the data rates of the input multimedia data streams responsive to the comparison.

2. The apparatus of claim 1 wherein the multimedia processor further comprises:

a rate comparator, having a first input for receiving a target output data stream data rate and a second input coupled to the output of the data rate analyzer, for generating a rate recommendation signal;

a data rate optimizer, having a control input coupled to the rate comparator for receiving the rate recommendation signal, and generating rate control signals to be used by encoders to alter the data rates of the input multimedia data streams responsive to the rate recommendation signal.

3. The apparatus of claim 2 wherein the multimedia processor further comprises:

a data rate analyzer, having inputs for receiving compressed multimedia data of the data streams and outputs coupled to the data rate optimizer, the outputs defining a data rate for compressed multimedia data of the data streams; and

wherein the data rate optimizer analyzes the data rates of the compressed multimedia streams and the rate recommendation signal to generate rate control signals.

4. The apparatus of claim 1 wherein the input multimedia data stream is a video elementary stream.

5. The apparatus of claim 1 wherein the output multimedia data stream is a program stream.

6. The apparatus of claim 1 wherein the output multimedia data stream is a transport stream.

7. In a device for providing compression to input multimedia data streams, combining the compressed multimedia data streams into an output combined multimedia data stream, and storing compressed multimedia data streams and output combined multimedia data streams, an apparatus comprising:

at least one multimedia data encoder, comprising;

a compression engine, for compressing a multimedia data stream and providing a compressed multimedia data stream output; and

a data rate adjuster, coupled to the output of the compression engine, for adjusting the data rate of the compressed multimedia data stream output responsive to a rate control signal generated by a multimedia stream control unit;

a unified memory module, comprising;

a plurality of reallocatable multimedia data buffers;

a memory allocator, coupled to the multimedia data stream buffers, for determining storage requirements of the compressed multimedia data stream and the output multimedia data stream as the streams are received, and allocating buffers responsive to the determined storage requirements;

the multimedia stream control unit being coupled to the unified memory module and the multimedia data encoder, and comprising;

a data rate analyzer, coupled to the output combined multimedia data stream, for determining the data rate of the output combined multimedia data streams;

and a data rate feedback adjuster, coupled to the data rate analyzer and the at least one multimedia data encoder, for comparing the determined output combined multimedia data stream data rate to the target output combined data stream data rate, generating rate control signals for adjusting the data rates of the compressed multimedia data streams responsive to the comparison, and transmitting the rate control signals to the at least one multimedia data encoder.

8. The apparatus of claim 7 wherein the buffers comprise an instruction part and a data part, and the instruction part comprises at least one of a set of instructions for directing the multimedia stream control unit to perform data manipulations on information in the data part.

9. The apparatus of claim 8 wherein the instruction set includes program control instructions, data access instructions and data processing instructions.

10. The apparatus of claim 8 wherein the buffers are organized using link lists, and the buffers are accessed in accordance with the link lists.

11. The apparatus of claim 7 wherein there are two multimedia data encoders, a first encoder for encoding video data and a second encoder for encoding audio data.

12. The apparatus of claim 7 wherein the multimedia stream control unit comprises a target data rate input for receiving a target output combined multimedia data stream data rate.

13. A method for generating a combined multimedia data stream output having a data rate within a tolerance of a target data rate, in a device having multimedia encoders for generating encoded data streams and a data stream multiplexer, comprising the steps of:

combining the encoded data streams to form a single output data stream;

determining the data rate of the combined output data stream;

comparing the determined data rate to a target data rate;

responsive to the target data rate exceeding the determined data rate, adjusting the data rates of the encoded data streams until the combined output data stream data rate is within a tolerance of the target data rate; and

responsive to the determined data rate exceeding the target data rate, adjusting the data rates of the encoded data streams until the combined output data stream data rate is within a tolerance of the target data rate.

14. The method of claim 13 further comprising the step of:

monitoring the data rate of the encoded data streams;

wherein the responsive to the determined data rate exceeding the target data rate, adjusting the data rates step further comprises

comparing the monitored data rates of the encoded data streams to nominal data rates for the encoded data streams;

responsive to a data rate of an encoded data stream exceeding its nominal data rate, reducing the data rate of the encoded data stream; and

wherein the responsive to the target data rate exceeding the determined data rate, adjusting the data rates step further comprises;

comparing the monitored data rates of the encoded data streams to nominal data rates for the encoded data streams;

responsive to a nominal data rate of an encoded data stream exceeding the data rate of the encoded data stream, increasing the data rate of the encoded data stream.