System and method for merging multiple audio streams

• US 6,311,161 B1
• Filed: 03/22/1999
• Issued: 10/30/2001
• Est. Priority Date: 03/22/1999
• Status: Expired due to Term

• Current Assignee: MediaTek, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Foster, Eric M., Franklin, Dennis Edward, Anderson, Richard Eugene
• Primary Examiner(s): Dorvil, Richemond
• Application Number:
  US09/273,919
• Time in Patent Office: 953 Days
• Field of Search: 704/200, 704/201, 704/500, 704/220, 704/200.1, 704/501, 704/502, 704/503, 704/504, 704/212, 704/223
• US Class Current: 704/500
• CPC Class Codes: G10L 19/008 Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
• CPC Class Codes: H04H 60/04 Studio equipment; Interconnection of studios
• CPC Class Codes: H04N 21/4305 Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/439 Processing of audio elementary streams

Abstract

A device and method are provided for merging one or more secondary audio channels, received from either a streaming application or from memory, with a primary stream of audio data output from a main audio decoder. In addition to the audio decoder, the device/method employs a controller for processing received secondary audio data in a streaming application, and retrieving from system memory audio PCM data once stored. An audio playback device is provided for formatting the PCM data for mixing with the primary stream of audio data output from the audio decoder. Multiple digital-to-analog converters convert the multiple streams of audio data into analog signals which are then mixed into a single merged audio signal for presentation.

Claims

1. A system for mixing multiple audio streams, said system comprising:

a first audio playback device comprising an audio decoder for decoding a primary stream of audio data, said audio decoder producing a decoded primary stream of audio data in digital, serial format;

means for retrieving a secondary stream of audio data from system memory, said secondary stream comprising audio pulse code modulated (PCM) data;

a second audio playback device comprising an audio PCM playback macro for formatting said secondary stream of audio data in digital, serial format for mixing with said decoded primary stream of audio data in digital, serial format; and

a mixer for mixing said decoded primary stream of audio data and said formatted secondary stream of audio data for output as a merged audio signal.

2. The system of claim 1, wherein said secondary stream of audio data comprises uncompressed audio PCM data.

3. The system of claim 1, wherein said primary stream of audio data and said secondary stream of audio data comprise independent audio streams.

4. The system of claim 3, wherein said mixer comprises an analog mixer, and wherein said system further comprises a first digital-to-analog converter for converting said decoded primary stream of audio data to an analog signal and a second digital-to-analog converter for converting said formatted secondary stream of audio data to an analog signal, said analog mixer being coupled to said first digital-to-analog converter and to said second digital-to-analog converter for receiving said analog signals and producing therefrom said merged audio signal.

5. The system of claim 1, wherein said primary stream of audio data comprises MPEG encoded audio data, and said audio decoder comprises an MPEG audio decoder.

6. The system of claim 1, wherein said system comprises a set top box system.

7. The system of claim 1, further comprising a transport demultiplexer, said transport demultiplexer providing said primary stream of audio data to said first playback device and said secondary stream of audio data to said system memory.

8. A system for mixing multiple audio streams, said system comprising:

an audio decoder for decoding a primary stream of audio data, said audio decoder producing a decoded primary stream of audio data;

means for retrieving a secondary stream of audio data from system memory, said secondary stream comprising audio pulse code modulated (PCM) data;

an audio playback device for formatting said secondary stream of audio data for mixing with said decoded primary stream of audio data;

a mixer for mixing said decoded primary stream of audio data and said formatted secondary stream of audio data for output as a merged audio signal; and

wherein said audio playback device further comprises a working buffer and a pending buffer coupled to receive in parallel portions of said audio PCM data, a buffer multiplexer receiving as input audio data read from said working buffer and said pending buffer, a serial shifter coupled to receive output from said buffer multiplexer and produce therefrom a serial PCM sample stream, a formatter coupled to receive said serial PCM sample stream from said serial shifter, said formatter converting said serial PCM sample stream to a digital, serial format, and a clock multiplexer for providing a clock to drive said formatter using one of a plurality of available clock signals.

9. The system of claim 8, wherein said plurality of clock signals comprises at least some of an external system clock signal, an internal system clock signal and a decoder clock signal received from said audio decoder.

10. A method for mixing multiple audio signals, said method comprising:

decoding a primary stream of audio data and producing therefrom a decoded primary stream of audio data in digital, serial format, said decoding comprising employing a first audio digital playback device comprising an audio decoder;

retrieving a secondary stream of audio data from system memory, said secondary stream comprising audio pulse code modulated (PCM) data;

formatting said secondary stream of audio data in digital, serial format for mixing with said decoded primary stream of audio data in digital, serial format, said formatting employing a second audio digital playback device comprising an audio PCM playback macro; and

mixing said decoded primary stream of audio data and said formatted secondary stream of audio data for output as a merged audio signal.

11. The method of claim 10, further comprising converting said decoded primary stream of audio data to an analog signal and separately converting said formatted secondary stream of audio data to an analog signal, and wherein said mixing comprises mixing said analog signals into said merged audio signal.

12. The method of claim 10, wherein said primary stream of audio data and said secondary stream of audio data have different sample frequencies prior to said mixing thereof.

13. The method of claim 10, wherein said primary stream of audio data and said secondary stream of audio data comprise PCM data with different sample widths.

14. The method of claim 10, wherein said primary stream of audio data and said second stream of audio data comprise independent audio streams.

15. The method of claim 10, further comprising employing a transport demultiplexer to provide said primary stream of audio data to said first audio digital playback device and said secondary stream of audio data to said system memory.

16. A method for mixing multiple audio signals, said method comprising:

decoding a primary stream of audio data and producing therefrom a decoded primary stream of audio data;

retrieving a secondary stream of audio data from system memory, said secondary stream comprising audio pulse code modulated (PCM) data;

formatting said secondary stream of audio data for mixing with said decoded primary stream of audio data;

mixing said decoded primary stream of audio data and said formatted secondary stream of audio data for output as a merged audio signal; and

wherein said formatting comprises formatting said audio PCM data in digital, serial format, and wherein said decoding comprises outputting said decoded primary stream of audio data as audio PCM data in I²S format.

17. A method for mixing multiple audio signals, said method comprising:

decoding a primary stream of audio data and producing therefrom a decoded primary stream of audio data;

retrieving a secondary stream of audio data from system memory, said secondary stream comprising audio pulse code modulated (PCM) data;

formatting said secondary stream of audio data for mixing with said decoded primary stream of audio data;

mixing said decoded primary stream of audio data and said formatted secondary stream of audio data for output as a merged audio signal; and

wherein said formatting comprises initially buffering in parallel portions of said audio PCM data, multiplexing the parallel buffered portions of said audio PCM data, serially shifting output of said multiplexing to produce a serial PCM sample stream, and converting the serial PCM sample stream to a digital, serial format, said converting employing one of a plurality of available clock signals.

18. The method of claim 17, further comprising providing said plurality of available clock signals to include at least some of an external system clock signal, an internal system clock signal and a decoder clock signal employed by said decoding.