Synchronizing digital audio to digital video

• US 5,506,932 A
• Filed: 04/16/1993
• Issued: 04/09/1996
• Est. Priority Date: 04/16/1993
• Status: Expired due to Term

• Current Assignee: Acoustic Technology LLC (Intellectual Ventures LLC)
• Original Assignee: Data Translation, Inc. (Emerson Electric Company)
• Inventors: Tarr, Morton H., Molnar, John W., Holmes, Daniel J.
• Primary Examiner(s): MacDonald, Allen R.
• Assistant Examiner(s): Dorvil, Richemond
• Application Number:
  US08/049,987
• Time in Patent Office: 1,089 Days
• Field of Search: 395/2, 395/154, 395/153, 395/275, 395/2.14, 395/2.79, 395/154, 395/200.04, 370/62, 348/15, 348/552, 358/335, 360/19.1
• US Class Current: 704/205
• CPC Class Codes: G11B 2220/90 Tape-like record carriers
• CPC Class Codes: G11B 27/024 on tapes G11B27/028, G11B27/029 take precedence
• CPC Class Codes: G11B 27/032 on tapes G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 21/426 Internal components of the client ; Characteristics thereof H04N5/44 takes precedence
• CPC Class Codes: H04N 21/4305 Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/4341 Demultiplexing of audio and video streams
• CPC Class Codes: H04N 5/04 Synchronising for television systems using pulse code modulation H04N7/56
• CPC Class Codes: H04N 5/08 Separation of synchronising signals from picture signals
• CPC Class Codes: H04N 5/602 for digital sound signals
• CPC Class Codes: H04N 5/765 Interface circuits between an apparatus for recording and another apparatus
• CPC Class Codes: H04N 7/56 Synchronising systems therefor
• CPC Class Codes: H04N 9/641 Multi-purpose receivers, e.g. for auxiliary information H04N9/642 takes precedence
• CPC Class Codes: H04N 9/802 involving processing of the sound signal H04N9/806, H04N9/835 take precedence

Abstract

An audio/video input/output (I/O) port apparatus for acquiring digital audio samples from one or multiple channels of input audio and synthesizing digital audio samples into one or multiple channels of output audio. The apparatus comprises a video I/O port, a frequency synthesizer, and an audio I/O port. The video I/O port generates a video-rate clock, and is configured to digitize input video into digital video, and to synthesize output video from digital video. The frequency synthesizer is configured to derive an audio sampling clock based on the video-rate clock. The audio I/O port is configured to sample input audio and convert it into digital audio samples according to the sampling clock, and to synthesize digital audio samples into output audio according to the sampling clock. The apparatus ensures that the video and audio data track together, both when inputting the information from an external source and when outputting the audio/video data streams. The technique is particularly valuable in video editing, where it is critical to establish and maintain synchronization between the video of a speaking person and the audio representing the spoken material.

Claims

1. An audio/video input/output (I/O) apparatus for acquiring digital audio samples from one or multiple channels of input audio and, alternately, synthesizing digital audio samples into one or multiple channels of output audio, the apparatus comprising:

a video I/O port circuit generating a video-rate clock, said video I/O port circuit comprising a video input port configured to convert input analog video into digital video, and a video output port configured to synthesize output analog video from digital video; and

an audio I/O port circuit comprising a frequency synthesizer connected to receive said video-rate clock from said video I/O port and configured to derive am audio sampling clock based on said video-rate clock, an audio input port configured to sample and convert input analog audio into digital audio samples according to said sampling clock, and an audio output port configured to synthesize digital audio samples into output analog audio according to said sampling clock.

2. The audio/video I/O apparatus of claim 1 further comprising:

a general purpose computer with a storage device and a peripheral slot; and

peripheral interface circuitry installed in said peripheral slot and configured to transfer said digital video and digital audio samples from said audio input and video input ports to said storage device, and to transfer data from said storage device to said audio output and video output ports.

3. The audio/video I/O apparatus of claim 2 wherein said peripheral interface circuitry further comprises FIFOs for coupling asynchronous operation of said computer to synchronous operation of said video and audio input and output ports.

4. The audio/video I/O apparatus of claim 2 wherein said peripheral interface circuitry further comprises a CODEC for compressing said digital video into compressed video data for storing on said storage device and for decompressing said video data as they are transferred from said storage device to said video output port.

5. The audio/video I/O apparatus of claim 1 wherein the video input port circuit further comprises:

a detector configured to detect a synchronizing signal in said input analog video;

a frequency generator for generating said video-rate clock and clocking said video output port;

a pulse generator configured to provide an input synchronization pulse to said audio I/O port circuit, said input synchronization pulse being generated for each frame of said input analog video and being based on said detected synchronizing signal; and

a pulse generator configured to provide a synchronizing pulse to said audio I/O port circuit.

6. The audio/video I/O apparatus of claim 1 wherein said video I/O port circuit further comprises a frequency generator configured to generate said video-rate clock independent of both input and output video.

7. The audio/video I/O apparatus of claim 1 wherein said frequency synthesizer comprises an m/n phase lock loop to generate said audio sampling clock as an adjustable ratio of said video-rate clock.

8. The audio/video I/O apparatus of claim 7 wherein said m/n phase lock loop includes an m counter, an n counter, a phase detector, and an oscillator.

9. The audio/video I/O apparatus of claim 8 further comprising a digital signal processor programmed to:

selectively associate a varying number of digital audio samples to each respective frame of said digital video to achieve a precise ratio of the number of said digital audio samples over a finite time to the number of associated frames of said digital video over said finite time; and

selectively associate a varying number of said digital audio samples to each respective frame of said digital video as said digital audio samples and said digital video are presented to said video and audio output ports.

10. The audio/video I/O apparatus of claim 1 wherein said frequency synthesizer further comprises:

an oversampled clock synthesizer that generates an oversampled clock whose frequency is at least twice as great as the frequency of said audio sampling clock; and

a frequency divider that frequency divides said oversampled clock to generate said audio sampling clock.

11. The audio/video I/O apparatus of claim 1 further comprising a digital signal processor programmed to:

selectively associate a varying number of digital audio samples to each respective frame of said digital video to achieve a precise ratio of the number of said digital audio samples over a finite time to the number of associated frames of said digital video over said finite time; and

selectively associate a varying number of said digital audio samples to each respective frame of said digital video as said digital audio samples and said digital video are presented to said video and audio output ports.

12. The audio/video I/O apparatus of claim 1 wherein said audio input port further comprises:

an analog-to-digital converter for converting said input analog audio into said digital audio samples, said analog-to-digital converter being clocked by said audio sampling clock; and

a digital-to-analog converter for converting said digital audio samples to said output analog audio, said digital-to-analog converter being clocked by said audio sampling clock.

13. The audio/video I/O apparatus of claim 1 further comprising:

a host computer with a storage device for storing said digital audio samples and said digital video;

first and second FIFOs for storing said digital audio samples between said audio input port and said storage device;

third and fourth FIFOs for storing said digital audio samples between said storage device and said audio output port; and

a digital signal processor for controlling flow of said digital audio samples between said first and second FIFOs and said third and fourth FIFOs.

14. An audio/video input apparatus for acquiring one or multiple channels of digital audio samples, the apparatus comprising:

a video input port configured to convert input analog video into digital video and generate a video-rate clock;

a frequency synthesizer connected to receive said video-rate clock from said video input-port and configured derive an audio sampling clock based on said video-rate clock; and

an audio input port configured to sample and convert input analog audio into digital audio samples according to said sampling clock.

15. The audio/video input apparatus of claim 14 further comprising:

a general purpose computer with a storage device and a peripheral slot; and

peripheral interface circuitry installed in said peripheral slot and configured to transfer said digital video and digital audio samples from said audio input and video input ports to said storage device.

16. The audio/video input apparatus of claim 15 wherein said peripheral interface circuitry further comprises a FIFO for coupling asynchronous operation of said computer to synchronous operation of said video and audio input ports.

17. The audio/video input apparatus of claim 15 wherein said peripheral interface circuitry further comprises a CODEC for compressing said digital video into compressed video data for storing on said storage device.

18. The audio/video input apparatus of claim 14 wherein the video input port further comprises:

a detector configured to detect a synchronizing signal of said input analog video; and

a pulse generator configured to provide an input synchronization pulse to said audio input port, said input synchronization pulse being generated for each frame of said input analog video and being based on said detected synchronizing signal.

19. The audio/video input apparatus of claim 14 wherein said video input port further comprises a frequency generator configured to generate said video-rate clock independent of input analog video.

20. The audio/video input apparatus of claim 14 wherein said frequency synthesizer comprises an m/n phase lock loop to generate said audio sampling clock as an adjustable ratio of said video-rate clock.

21. The audio/video input apparatus of claim 20 wherein said m/n phase lock loop includes an m counter, an n counter, a phase detector, and an oscillator.

22. The audio/video input apparatus of claim 21 further comprising:

a digital signal processor programmed to selectively associate a varying number of digital audio samples to each respective frame of said digital video to achieve an accurate ratio of the number of said digital audio samples over a finite time to the number of associated frames of said digital video over said finite time.

23. The audio/video input apparatus of claim 14 wherein said frequency synthesizer further comprises:

an oversampled clock synthesizer that generates an oversampled clock whose frequency is at least twice as great as the frequency of said audio sampling clock; and

a frequency divider that frequency divides said oversampled clock to generate said audio sampling clock.

24. The audio/video input apparatus of claim 14 further comprising:

a digital signal processor programmed to selectively associate a varying number of digital audio samples to each respective frame of said digital video to achieve an accurate ratio of the number of said digital audio samples over a finite time to the number of associated frames of said digital video over said finite time.

25. The audio/video input apparatus of claim 14 wherein

said audio input port further comprises an analog-to-digital converter for converting said input analog audio into said digital audio samples, said analog-to-digital converter being clocked by said audio sampling clock.

26. The audio/video input apparatus of claim 14 further comprising:

a host computer with a storage device for storing said digital audio samples and said digital video;

first and second FIFOs for storing said digital audio samples between said audio input port and said storage device; and

a digital signal processor for controlling flow of said digital audio samples between said first and second FIFOs.

27. An audio/video output apparatus for synthesizing one or multiple channels of digital audio samples into output audio, the apparatus comprising:

a video output port configured to synthesize output analog video from digital video and generating a video-rate clock;

a frequency synthesizer connected to receive said video-rate clock from said video output port and configured to derive an audio sampling clock based on said video-rate clock; and

an audio output port configured to synthesize output analog audio from digital audio samples according to said sampling clock.

28. The audio/video output apparatus of claim 27 further comprising:

a general purpose computer with a storage device and a peripheral slot; and

peripheral interface circuitry installed in said peripheral slot and configured to transfer data from said storage device to said audio output and video output ports.

29. The audio/video output apparatus of claim 28 wherein said peripheral interface circuitry further comprises a FIFO for coupling asynchronous operation of said computer to synchronous operation of said video and audio output ports.

30. The audio/video output apparatus of claim 28 wherein said video peripheral further comprises a CODEC for decompressing said video data as they are transferred from said storage device to said video output port.

31. The audio/video output apparatus of claim 27 wherein said video output port further comprises:

a frequency generator for generating said video-rate clock and clocking said video output port; and

a pulse generator configured to provide a synchronizing pulse to said audio output port.

32. The audio/video output apparatus of claim 27 wherein said video output port further comprises a frequency generator configured to generate said video-rate clock independent of output video.

33. The audio/video output port apparatus of claim 27 wherein said frequency synthesizer comprises an m/n phase lock loop to generate said audio sampling clock as an adjustable ratio of said video-rate clock.

34. The audio/video output apparatus of claim 33 wherein said m/n phase lock loop includes an m counter, an n counter, a phase detector, and an oscillator.

35. The audio/video output apparatus of claim 27 wherein said frequency synthesizer further comprises:

an oversampled clock synthesizer that generates an oversampled clock whose frequency is at least twice as great as the frequency of said audio sampling clock; and

a frequency divider that frequency divides said oversampled clock to generate said audio sampling clock.

36. The audio/video output apparatus of claim 27 further comprising:

a digital signal processor programmed to selectively associate a varying number of said digital audio samples to each respective frame of said digital video as said digital audio samples and said digital video are presented to said video and audio output ports.

37. The audio/video output apparatus of claim 27 wherein

said audio output port further comprises a digital-to-analog converter for converting said digital audio samples to said output analog audio, said digital-to-analog converter being clocked by said audio sampling clock.

38. The audio/video output apparatus of claim 27 further comprising:

a host computer with a storage device for storing said digital audio samples and said digital video;

first and second FIFOs for storing said digital audio samples between said storage device and said audio output port; and

a digital signal processor for controlling flow of said digital audio samples between said first and second FIFOs.

39. A method for acquiring video and one or more channels of audio and storing them in digital form on a storage device of a computer, the method comprising the steps:

receiving said video at a video input port;

detecting one or more synchronization signals in said video;

frequency-synthesizing an audio sampling clock from said synchroniztion signals in a frequency synthesizing circuit connected to receive said synchronization signals from said video input port;

receiving said audio at an audio input port, said audio input port clocked by said audio sampling clock; and

storing said received video and received audio on the storage device.