Managing audio data using a graphics display controller

• US 5,553,220 A
• Filed: 09/07/1993
• Issued: 09/03/1996
• Est. Priority Date: 09/07/1993
• Status: Expired due to Term

• Current Assignee: NVIDIA Corporation
• Original Assignee: Cirrus Logic Incorporated
• Inventors: Keene, David
• Primary Examiner(s): Zimmerman, Mark K.
• Assistant Examiner(s): Feild, Joseph H.
• Application Number:
  US08/118,153
• Time in Patent Office: 1,092 Days
• Field of Search: 395/154, 4/162, 381/150, 379/67-88, 369/50, 369/69, 345/203, 370/108, 370/119, 348/480-482, 348/232, 348/434, 348/462, 348/476-477, 348/479
• US Class Current: 345/520
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: H04N 21/426 Internal components of the client ; Characteristics thereof H04N5/44 takes precedence
• CPC Class Codes: H04N 5/602 for digital sound signals
• CPC Class Codes: H04N 7/04 Systems for the transmission of one television signal, i.e. both picture and sound, by a single carrier H04N7/084, H04N7/087 take precedence
• CPC Class Codes: H04N 9/802 involving processing of the sound signal H04N9/806, H04N9/835 take precedence

Abstract

A graphics display controller is used to manage audio data within a computer-based multimedia system. A portion of the video display memory associated with the graphics display controller is allocated to one or more audio data buffers to hold digitized data for audio recording and/or playback. Preferably, the size and location of the audio buffer(s) relative to the video data is dynamically configureable within the memory. The timing of the audio data transfer into or out of the video display memory is preferably controlled by the video horizontal sync (HSYNC) signal, so that audio data transfers occur substantially during the pauses in video-memory access that correspond to the retrace time at the end of each video display line. A single, unified host CPU interface and related circuitry is preferably used for control or data-transfer functions related both to video graphics and to audio. For threshold-sensitive audio recording, a look-back buffer is preferably used to hold audio-data samples up to a fixed number of samples prior to the first sample that exceeds the current threshold value.

Claims

1. A computer-based multimedia system, comprising:

a host CPU;

a display device;

an audio transducer;

an audio codec to transfer audio signals to or from said audio transducer;

a digital, random access video display memory having a plurality of buffer segments, at least one said buffer segment for video data and at least one said buffer segment for audio data;

a video graphics controller, responsive to said host CPU, to control the display of said video data on said display device;

an audio resources manager, responsive to said host CPU, to control said audio codec and to control the transfer of said audio data between said host CPU and said video display memory and between said video display memory and said audio codec;

a video memory controller, responsive to said video graphics controller and to said audio resources manager, to store and retrieve said video data and said audio data within said video display memory; and

a host CPU interface to couple said host CPU to both said video graphics controller and to said audio resources manager.

5. The computer-based multimedia system of any one of claims 1 through 4, wherein:

said audio data is stored and retrieved using a circular sequence of addresses within said audio data buffer segment.

6. The computer-based multimedia system of any one of claims 1 through 4, wherein:

said transfer of audio data to or from said audio codec occurs over a serial interface bus.

2. An high-performance computer-based multimedia system, comprising:

a host CPU;

a display device;

an audio transducer;

an audio codec to transfer audio signals to or from said audio transducer;

a digital, random access video display memory having a plurality of buffer segments, at least one said buffer segment for video data and at least one said buffer segment for audio data;

a video graphics controller, responsive to said host CPU, to control the displaying of lines of said video data on said display device, said displaying having temporal pauses between said adjacent lines;

an audio resources manager, responsive to said host CPU, to control said audio codec and to control the transfer of said audio data between said host CPU and said video display memory and between said video display memory and said audio codec, said transfer between said video display memory and said audio codec occurring substantially only during said pauses;

a video memory controller, responsive to said video graphics controller and to said audio resources manager, to store and retrieve said video data and said audio data within said video display memory; and

a host CPU interface to couple said host CPU to both said video graphics controller and to said audio resources manager.

7. The computer-based multimedia system of any one of claims 2 or 4, wherein:

said audio resources manager includes at least one first-in-first-out buffer through which said audio data is transferred to or from said audio codec, said buffers being of a predetermined depth such that they accommodate as many audio samples as are consumed or produced by said audio codec between adjacent pairs of said pauses.

3. A threshold-sensitive computer-based multimedia system, comprising:

a host CPU;

a display device;

an audio transducer;

an audio codec to transfer audio signals to or from said audio transducer;

a digital, random access video display memory having a plurality of buffer segments, at least one said buffer segment for video data and at least one said buffer segment for audio data;

a video graphics controller, responsive to said host CPU, to control the displaying of said video data on said display device;

an audio resources manager, responsive to said host CPU, to control said audio codec and to transfer said audio data between said host CPU and said video display memory and between said video display memory and said audio codec, said audio resources manager including a host read pointer to address said video display memory during the transfer of said audio data to said host CPU, the transfer from said audio codec to said video display memory advancing the value of said host read pointer during a period prior to when a value of any sample of said audio data exceeds a predetermined threshold value and subsequent to when a predetermined number of samples have been transferred into said video display memory whereby said host read pointer addresses a sample that preceded the first over-threshold sample by up to said predetermined number of samples;

a video memory controller, responsive to said video graphics controller and to said audio resources manager, to store and retrieve said video data and said audio data in and from said video display memory; and

a host CPU interface to couple said host CPU to both said video graphics controller and to said audio resources manager.

4. An high-performance, threshold-sensitive computer-based multimedia system, comprising:

a host CPU;

a display device;

an audio transducer;

an audio codec to transfer audio signals to or from said audio transducer;

a digital, random access video display memory having a plurality of buffer segments, at least one said buffer segment for video data and at least one said buffer segment for audio data;

a video graphics controller, responsive to said host CPU, to control the displaying of lines of said video data on said display device, said displaying having temporal pauses between said adjacent lines;

an audio resources manager, responsive to said host CPU, to control said audio codec and to transfer said audio data between said host CPU and said video display memory and between said video display memory and said audio codec, said transfer between said video display memory and said audio codec occurring substantially during said pauses, said audio resources manager including a host read pointer to address said video display memory during the transfer of said audio data to said host CPU, the transfer from said audio codec to said video display memory advancing the value of said host read pointer during a period prior to when the value of any sample of said audio data exceeds a predetermined threshold value and subsequent to when a predetermined number of samples have boon transferred into said video display memory whereby said host read pointer addresses a sample that preceded the first over-threshold sample by up to said predetermined number of samples;

a video memory controller, responsive to said video graphics controller and to said audio resources manager, to store and retrieve said video data and said audio data in and from said video display memory; and

a host CPU interface to couple said host CPU to both said video graphics controller and to said audio resources manager.

8. A storage subsystem for audio and video data within a computer-based multimedia system, comprising:

a system bus;

an audio codec;

a digital, random access video display memory having a plurality of buffer segments, at least one said buffer segment for video data and at least one said buffer segment for audio data;

a video graphics controller, responsive to commands presented on said system bus, to control the displaying of said video data;

an audio resources manager, responsive to commands presented on said system bus, to control said audio codec and to control the transfer of said audio data between said system bus and said video display memory and between said video display memory and said audio codec;

a video memory controller, responsive to said video graphics controller and to said audio resources manager, to store and retrieve said video data and said audio data in and from said video display memory; and

a bus interface to couple said system bus to both said video graphics controller and to said audio resources manager.

9. An high-performance storage subsystem for audio and video data within a computer-based multimedia system, comprising:

a system bus;

an audio codec;

a digital, random access video display memory having a plurality of buffer segments, at least one said buffer segment for video data and at least one said buffer segment for audio data;

a video graphics controller, responsive to commands presented on said system bus, to control the displaying of lines of said video data, said displaying having temporal pauses between said adjacent lines;

an audio resources manager, responsive to commands presented on said system bus, to control said audio codec and to control the transfer of said audio data between said system bus and said video display memory and between said video display memory and said audio codec, said transfer between said video display memory and said audio codec occurring substantially during said pauses;

a video memory controller, responsive to said video graphics controller and to said audio resources manager, to store and retrieve said video data and said audio data in and from said video display memory; and

a bus interface to couple said system bus to both said video graphics controller and to said audio resources manager.

10. A threshold-sensitive storage subsystem for audio and video data within a computer-based multimedia system, comprising:

a system bus;

an audio codec;

a digital, random access video display memory having a plurality of buffer segments, at least one said buffer segment for video data and at least one said buffer segment for audio data;

a video graphics controller, responsive to commands presented on said system bus, to control the displaying of said video data;

an audio resources manager, responsive to commands presented on said system bus, to control said audio codec and to control the transfer of said audio data between said system bus and said video display memory and between said video display memory and said audio codec, said audio resources manager including a system read pointer to address said video display memory during the transfer of said audio data to said system bus, the transfer from said audio codec to said video display memory advancing the value of said system read pointer during a period prior to when the value of any sample of said audio data exceeds a predetermined threshold value and subsequent to when a predetermined number of samples have been transferred into said video display memory whereby said system read pointer addresses a sample that preceded the first over-threshold sample by up to said predetermined number of samples;

a video memory controller, responsive to said video graphics controller and to said audio resources manager, to store and retrieve said video data and said audio data in and from said video display memory; and

a bus interface to couple said system bus to both said video graphics controller and to said audio resources manager.

12. The subsystem of any one of claims 8 through 11, wherein:

said audio data is stored and retrieved using a circular sequence of addresses within said audio data buffer segment.

13. The computer-based multimedia system of any one of claims 8 through 11, wherein:

said transfer of audio data to or from said audio codec occurs over a serial interface bus.

11. An high-performance, threshold-sensitive storage subsystem for audio and video data within a computer-based multimedia system, comprising:

a system bus;

an audio codec;

a digital, random access video display memory having a plurality of buffer segments, at least one said buffer segment for video data and at least one said buffer segment for audio data;

a video graphics controller, responsive to commands presented on said system bus, to control the displaying of lines of said video data, said displaying having temporal pauses between said adjacent lines;

an audio resources manager, responsive to commands presented on said system bus, to control said audio codec and to control the transfer of said audio data between said system bus and said video display memory and between said video display memory and said audio codec, said transfer between said video display memory and said audio codec occurring substantially during said pauses, said audio resources manager including a system read pointer to address said video display memory during the transfer of said audio data to said system bus, the transfer from said audio codec to said video display memory advancing the value of said system read pointer during a period prior to when the value of any sample of said audio data exceeds a predetermined threshold value and subsequent to when a predetermined number of samples have been transferred into said video display memory whereby said system read pointer addresses a sample that preceded the first over-threshold sample by up to said predetermined number of samples;

a video memory controller, responsive to said video graphics controller and to said audio resources manager, to store and retrieve said video data and said audio data in and from said video display memory; and

a bus interface to couple said system bus to both said video graphics controller and to said audio resources manager.

14. The computer-based multimedia system of any one of claims 9 or 11, wherein:

said audio resources manager includes at least one first-in-first-out buffer through which said audio data is transferred to or from said audio codec, said buffers being of a predetermined depth such that they accommodate as many audio samples as are consumed or produced by said audio codec between adjacent pairs of said pauses.

15. A multimedia controller, comprising:

a system bus interface;

a memory controller to control the storage and retrieval of both video data and audio data within a single digital, random access video display memory;

a video graphics controller, responsive to commands received on said system bus interface, to control the transfer of said video data from said system bus interface to said memory controller, and to control display of said video data; and

an audio resources manager, responsive to commands received on said system bus interface, to control the transfer of said audio data between said system bus interface and said memory controller, and to control recording or playback of said audio data.

19. The multimedia controller of any one of claims 15 through 18, wherein:

said audio resources manager addresses said audio data using a circular sequence of addresses.

20. The multimedia controller of any one of claims 15 through 18, further including:

a serial interface through which said audio data is transferred to or from said audio resources controller for recording or playback.

16. An high-performance multimedia controller, comprising:

a system bus interface;

a memory controller to control the storage and retrieval of both video data and audio data within a single digital, random access video display memory;

a video graphics controller, responsive to commands received on said system bus interface, to control the transfer of said video data from said system bus interface to said video display memory, and to control the display of lines of said video data, said display having temporal pauses between said adjacent lines; and

an audio resources manager, responsive to commands received on said system bus interface, to control the transfer of said audio data between said system bus interface and said memory controller, and to control recording or playback of said audio data, the storage or retrieval of said audio data for said recording or playback occurring substantially during said pauses.

21. The multimedia controller of any one of claims 16 or 18, wherein:

said audio resources manager includes one or more first-in-first-out buffers to transfer said audio data for recording or playback, said buffers being of a predetermined depth such that they accommodate as many audio samples as are consumed or produced between adjacent pairs of said pauses.

17. A threshold-sensitive multimedia controller, comprising:

a system bus interface;

a memory controller to control the storage and retrieval of both video data and audio data within a single digital, random access video display memory;

a video graphics controller, responsive to commands received on said system bus interface, to control the transfer of said video data from said system bus interface to said video display memory, and to control displaying of said video data; and

an audio resources manager, responsive to commands received on said system bus interface, to control the transfer of said audio data from said memory controller to said system bus interface, and to control recording said audio data from sound, said audio resources manager including a system read pointer to address said audio data during its transfer from said memory controller to said system bus interface, the value of said system read pointer advancing during a period prior to when the value of any sample of said audio data exceeds a predetermined threshold value and subsequent to when a predetermined number of samples have been taken whereby said system read pointer addresses a sample that preceded the first over-threshold sample by up to said predetermined number of samples.

18. An high-performance, threshold-sensitive multimedia controller, comprising:

a system bus interface;

a memory controller to control the storage and retrieval of both video data and audio data within a single digital, random access video display memory;

a video graphics controller, responsive to commands received on said system bus interface, to control the transfer of said video data from said system bus interface to said video display memory, and to control the display of lines of said video data, said display having temporal pauses between said adjacent lines; and

an audio resources manager, responsive to commands received on said system bus interface, to control the transfer of said audio data from said memory controller to said system bus interface, and to control recording said audio data from sound, the storage of said audio data during said recording occurring substantially during said pauses, said audio resources manager including a system read pointer to address said audio data during its transfer from said memory controller to said system bus interface, the value of said system read pointer advancing during a period prior to when the value of any sample of said audio data exceeds a predetermined threshold value and subsequent to when a predetermined number of samples have been taken whereby said system read pointer addresses a sample that preceded the first over-threshold sample by up to said predetermined number of samples.

22. A method of audio playback within a computer-based multimedia system, comprising:

allocating a segment of a digital, random access video display memory to hold video data;

allocating a segment of said video display memory to hold audio data;

transferring said video data from a host CPU to a system bus, then to a system bus interface, then to said video display memory, and subsequently, retrieving said video data from said video display memory and displaying said video data on a display device; and

transferring said audio data from said host CPU to said system bus, then to said system bus interface, then to said video display memory, and subsequently, retrieving said audio data from said video display memory and reproducing it by means of an audio codec and a transducer.

23. A method of audio recording within a computer-based multimedia system, comprising:

allocating a segment of a digital, random access video display memory to hold video data;

allocating a segment of said video display memory to hold audio data;

transferring said video data from a host CPU to a system bus, then to a system bus interface, then to said video display memory, and subsequently, retrieving said video data from said video display memory and displaying it on a display device; and

generating audio data from sound by means of a transducer and an audio codec, then storing said audio data in said video display memory, and subsequently, retrieving said audio data from said video display memory, transferring it to said system bus interface, then to said system bus, and then to said host CPU.

24. An high-performance method of storing audio playback data and video data within a computer-based multimedia system, comprising:

allocating a segment of a digital, random access video display memory to hold video data;

allocating a segment of said video display memory to hold audio data;

storing said video data from a system bus to said video display memory, and subsequently, retrieving from said video display memory lines of said video data to be displayed, said video data retrieving having temporal pauses between said adjacent lines; and

storing said audio data from said system bus into said video display memory, and subsequently, retrieving said audio data from said video display memory and reproducing it on by means of an audio codec and a transducer, said audio data retrieving occurring substantially during said pauses.

25. An high-performance method of storing audio recording data and video data within a computer-based multimedia system, comprising:

allocating a segment of a digital, random access video display memory to hold video data;

allocating a segment of said video display memory to hold audio data;

storing said video data from a system bus to said video display memory, and subsequently, retrieving from said video display memory lines of said video data to be displayed, said video data retrieving having temporal pauses between said adjacent lines; and

generating audio data from sound by means of a transducer and an audio codec, then storing said audio data in said video display memory, and subsequently, retrieving said audio data from said video display memory to said system bus, said audio data storing occurring substantially during said pauses.

26. A threshold-sensitive method of storing audio recording data and video data within a computer-based multimedia system, comprising:

allocating a segment of a digital, random access video display memory to hold video data;

allocating a segment of said video display memory to hold audio data;

transferring said video data from a host CPU to a system bus, then to a system bus interface, then to said video display memory, and subsequently, retrieving said video data from said video display memory and displaying it on a display device; and

generating audio data from sound by means of a transducer and an audio codec, then storing said audio data in said video display memory, and subsequently, retrieving said audio data from said video display memory to said system bus, said audio data retrieving being addressed by a system read pointer within said audio resources manager, the value of said system read pointer being advanced during a period prior to when the value of any sample of said audio data exceeds a predetermined threshold value and subsequent to when a predetermined number of samples have been taken; and

making available via said system read pointer up to said predetermined number of samples that occurred immediately prior to the first sample that exceeded said predetermined threshold.

27. An high-performance, threshold-sensitive method of storing audio recording data and video data within a computer-based multimedia system, comprising:

allocating a segment of a digital, random access video display memory to hold video data;

allocating a segment of said video display memory to hold audio data;

storing said video data from a system bus to said video display memory, and subsequently, retrieving from said video display memory lines of said video data to be displayed, said video data retrieving having temporal pauses between said adjacent lines;

generating audio data from sound by means of a transducer and an audio codec, then storing said audio data in said video display memory, and subsequently, retrieving said audio data from said video display memory to said system bus, said audio data storing occurring substantially during said pauses, said audio data retrieving being addressed by a system read pointer within said audio resources manager, the value of said system read pointer being advanced during a period prior to when the value of any sample of said audio data exceeds a predetermined threshold value and subsequent to when a predetermined number of samples have been taken; and

making available via said system read pointer up to said predetermined number of samples that occurred immediately prior to the first sample that exceeded said predetermined threshold.