Apparatus and method for time synchronization of a plurality of multimedia streams

• US 7,024,575 B2
• Filed: 11/14/2002
• Issued: 04/04/2006
• Est. Priority Date: 11/14/2002
• Status: Active Grant

• Current Assignee: Micron Technology, Inc.
• Original Assignee: Intel Corporation
• Inventors: Yeung, Minerva M., Kozintsev, Igor V., Lienhart, Rainer W.
• Primary Examiner(s): Browne, Lynne H.
• Assistant Examiner(s): Patel, Anand B.
• Application Number:
  US10/294,271
• Publication Number:
  US 20040098633A1
• Time in Patent Office: 1,237 Days
• Field of Search: None
• US Class Current: 713/400
• CPC Class Codes: H04N 21/2368 Multiplexing of audio and video streams
• CPC Class Codes: H04N 21/242 Synchronization processes, e.g. processing of PCR [Program Clock References] arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18
• CPC Class Codes: H04N 21/4341 Demultiplexing of audio and video streams

Abstract

An apparatus and method for time synchronization of a plurality of multimedia streams are described. In one embodiment, the method includes the concurrent capture of multimedia data via a plurality of multimedia platforms. During the concurrent capture of the multimedia data, each of the multimedia platforms receives a synchronization signal from a synchronization generation unit. In response, each platform processes a received synchronization signal to generate a common reference clock signal among each of the platforms. Once the common clock signal is generated, each of the platforms synchronizes captured multimedia data to form multimedia stream data according to the common reference clock signal. As such, the plurality of multimedia platforms are able to perform collaborative signal processing tasks of multimedia streams, including, for example, array signal processing algorithms.

Claims

1. A method comprising:

capturing, via a plurality of multimedia platforms, multimedia data;

receiving, during capture of the multimedia data, a synchronization signal by each of the plurality of multimedia platforms;

processing, by each multimedia platform, a received synchronization signal to generate a common clock reference signal among the plurality of multimedia platforms; and

synchronizing captured multimedia data according to the common clock reference signal.

2. The method of claim 1, wherein receiving further comprises:

receiving the synchronization signal from one of a wire link and a radio frequency (RF) link.

3. The method of claim 1, wherein receiving further comprises:

providing a synchronization signal via a respective channel of one or more capture devices of the respective multimedia platform while each capture device captures multimedia data via one or more additional channels of the respective capture device;

generating, by each capture device, a multimedia stream for the captured multimedia data; and

embedding, by each capture device, the received synchronization signal within the generated multimedia stream to form a plurality of generated multimedia stream data.

4. The method of claim 1, wherein processing further comprises:

receiving a radio frequency (RF) signal as the synchronization signal;

demodulating the received RF signal to form a synchronization signal;

providing the synchronization signal to one or more capture devices of the respective multimedia platform; and

forming, by each capture device, a target sampling frequency as the common clock reference signal according to the received synchronization signal.

5. The method of claim 1, further comprising:

receiving synchronized multimedia stream data generated by each platform during capture of the multimedia data; and

performing array signal processing using the received multimedia streams according to the generated common clock reference signal.

6. The method of claim 1, wherein synchronizing further comprises:

receiving an audio synchronization signal as the synchronization signal;

determining criteria used to form the received audio synchronization signal;

analyzing the received audio synchronization signal according to the determined criteria;

detecting one or more errors within captured multimedia data according to the audio synchronization signal analysis; and

correcting the one or more errors within the captured multimedia data to form corrected multimedia stream data.

7. The method of claim 6, wherein detecting further comprises:

detecting a delay offset between samples of an expected audio synchronization signal and the received audio synchronization signal;

detecting variance between samples of the expected audio synchronization signal and the received audio synchronization signal; and

detecting a frequency offset between samples of the expected audio synchronization signal and received audio synchronization signal.

8. The method of claim 6, wherein correcting the multimedia data further comprises:

resampling the corrected multimedia stream data according to a delay offset, a sample variance offset and a frequency offset detected between an expected audio synchronization signal and the received audio synchronization signal to form synchronized multimedia stream data.

9. The method of claim 1, wherein processing further comprises:

determining a target sampling rate according to the received synchronization signal;

comparing the target sampling rate to a data capture sampling rate used to capture the multimedia data; and

calculating a delay offset, sampling variance, and frequency offset between the target sampling rate and the data capture sampling rate.

10. The method of claim 1, wherein synchronization further comprises:

selecting, by each respective multimedia platform, a calculated delay offset, sampling variance, and frequency offset between a target sampling rate and a data capture sampling rate; and

resampling, by each respective multimedia platform, the captured multimedia data according to the calculated delay offset, sampling rate variance, and frequency shift to form synchronized multimedia stream data.

11. A computer readable storage media containing executable computer program instructions which when executed cause a digital processing system to perform a method comprising:

capturing, via a plurality of multimedia platforms, multimedia data;

receiving, during capture of the multimedia data, a synchronization signal by each of the plurality of multimedia platforms;

processing, by each multimedia platform, a received synchronization signal to generate a common clock reference signal among the plurality of multimedia platforms; and

synchronizing captured multimedia data according to the common clock reference signal.

12. The computer readable storage media of claim 11, wherein receiving the synchronization signal causes the digital processing system to perform a method further comprising:

receiving the synchronization signal from one of a wire link and a radio frequency (RF) link.

13. The computer readable storage media of claim 11, wherein receiving the synchronization signal causes the digital processing system to perform a method further comprising:

providing a synchronization signal via a respective channel of one or more capture devices of the respective multimedia platform while each capture device captures multimedia data via one or more additional channels of the respective capture device;

generating, by each capture device, a multimedia stream for the captured multimedia data; and

embedding, by each capture device, the received synchronization signal within the generated multimedia stream to form a plurality of generated multimedia stream data.

14. The computer readable storage media of claim 11, wherein processing the synchronization signal causes the digital processing system to perform a method further comprising:

receiving a radio frequency (RF) signal as the synchronization signal;

demodulating the received RF signal to form a synchronization signal;

providing the synchronization signal to one or more capture devices of the respective multimedia platform; and

forming, by each capture device, a target sampling frequency as the common clock reference signal according to the received synchronization signal.

15. The computer readable storage media of claim 11, wherein the digital processing system is caused to perform a method further comprising:

receiving synchronized multimedia stream data generated by each platform during capture of the multimedia data; and

performing array signal processing using the received multimedia streams according to the generated common clock reference signal.

16. The computer readable storage media of claim 11, wherein synchronizing captured multimedia data causes the digital processing system to perform a method further comprising:

receiving an audio synchronization signal as the synchronization signal;

determining criteria used to form the received audio synchronization signal;

analyzing the received audio synchronization signal according to the determined criteria;

detecting one or more errors within captured multimedia data according to the audio synchronization signal analysis; and

correcting the one or more errors within the captured multimedia data to form corrected multimedia stream data.

17. The computer readable storage media of claim 16, wherein detecting errors causes the digital processing system to perform a method further comprising:

detecting a delay offset between samples of an expected audio synchronization signal and the received audio synchronization signal;

detecting variance between samples of the expected audio synchronization signal and the received audio synchronization signal; and

detecting a frequency offset between samples of the expected audio synchronization signal and received audio synchronization signal.

18. The computer readable storage media of claim 16, wherein correcting the multimedia data causes the digital processing system to perform a method further comprising:

resampling the corrected multimedia data according to a delay offset, a sample variance offset and a frequency offset detected between an expected audio synchronization signal and the received audio synchronization signal to form synchronized multimedia stream data.

19. The computer readable storage media of claim 11, wherein processing the synchronization signal causes the digital processing system to perform a method further comprising:

determining a target sampling rate according to the received synchronization signal;

comparing the target sampling rate to a data capture sampling rate used to capture the multimedia data; and

calculating a delay offset, sampling variance, and frequency offset between the target sampling rate and the data capture sampling rate.

20. The computer readable storage media of claim 11, wherein synchronizing captured multimedia data causes the digital processing system to perform a method further comprising:

selecting, by each respective multimedia platform, a calculated delay offset, sampling variance, and frequency shift between a target sampling rate and a data capture sampling rate; and

resampling, by each respective multimedia platform, the captured multimedia data according to the calculated delay offset, sampling rate variance, and frequency offset to form synchronized multimedia stream data.

21. A method comprising:

determining a plurality of multimedia platforms to concurrently capture multimedia data;

generating synchronization information for the plurality of determined multimedia platforms; and

broadcasting the synchronization information to the plurality of platforms, via respective dedicated channels, as a multimedia synchronization signal.

22. The method of claim 21 wherein prior to determining, the method further comprises:

selecting a plurality of multimedia platforms to capture multimedia data;

arranging the plurality of platforms to enable capture of the multimedia data; and

configuring a multimedia platform to perform array signal processing on synchronized multimedia stream data generated by the plurality of multimedia platforms.

23. The method of claim 21, wherein generating the synchronization signal further comprises:

producing the multimedia synchronization signal for the multimedia platforms using a clock signal of a synchronization generator.

24. The method of claim 23, wherein producing further comprises:

selecting a carrier wave signal;

modulating the carrier wave signal according to the clock signal to form an audio synchronization signal as the multimedia synchronization signal.

25. The method of claim 21, wherein broadcasting further comprises:

transmitting the multimedia synchronization signal to each multimedia platform via a respective one of a wired link and a wireless radio frequency (RF) channel.

26. A system comprising:

a synchronization generator to generate a synchronization signal according to a clock signal of the synchronization generator; and

a plurality of multimedia platforms arranged to capture multimedia data via one or more capture devices and synchronize multimedia stream data generated from the captured multimedia data according to the synchronization signal generated by the synchronization generator.

27. The system of claim 26, further comprising:

a multimedia processing device to receive synchronized multimedia stream data from the plurality of multimedia platforms and perform array signal processing using the received, synchronized multimedia stream data.

28. The system of claim 26, wherein each multimedia platform further comprises:

one of a wired link and a radio frequency (RF) receiver to receive the multimedia synchronization signal.

29. The system of claim 26, wherein the synchronization generator further comprises:

a respective one of a wired link and a radio frequency (RF) transmitter to transmit the multimedia synchronization signal each of the plurality of multimedia platforms.

30. An apparatus, comprising:

a processor having circuitry to execute instructions;

one or more capture devices coupled to the processor, each capture device captures multimedia data while receiving a synchronization signal and embeds the synchronization signal within multimedia stream generated from captured multimedia data; and

a storage device coupled to the processor, having sequences of instructions stored therein, which when executed by the processor cause the processor to;

synchronize the generated multimedia stream data according to the received synchronization signal.

31. The apparatus of claim 30, wherein an instruction to synchronize the multimedia stream data further causes the processor to:

receive an audio synchronization signal as the synchronization signal;

determine criteria used to form the received audio synchronization signal;

analyze the received audio synchronization signal according to the determined criteria;

detect one or more errors within captured multimedia data according to the audio synchronization signal analysis; and

correct the one or more errors within the captured multimedia data to form corrected multimedia stream data.

32. The apparatus of claim 30, wherein an instruction to detect one or more errors further causes the processor to:

detect a delay offset between samples of an expected audio synchronization signal and the received audio synchronization signal;

detect variance between samples of the expected audio synchronization signal and the received audio synchronization signal; and

detect a frequency offset between samples of the expected audio synchronization signal and received audio synchronization signal.

33. The apparatus of claim 30, wherein an instruction to correct the multimedia data further causes the processor to:

resample the corrected multimedia stream data according to a delay offset, a sample variance offset and a frequency offset detected between an expected audio synchronization signal and the received audio synchronization signal to form synchronized multimedia stream data.

34. A method comprising:

capturing, via a plurality of multimedia platforms, multimedia data;

receiving, during capture of the multimedia data, a synchronization signal by each of the plurality of multimedia platforms; and

embedding, by each capture device of each multimedia platform, the received synchronization signal within generated multimedia streams to form a plurality of generated multimedia stream data.

35. The method of claim 34, further comprising:

processing, by each multimedia platform, a received synchronization signal to generate a common clock reference signal among the plurality of multimedia platforms; and

synchronizing, by each multimedia platform, captured multimedia data according to the common clock reference signal.

36. The method of claim 34, wherein embedding further comprises:

providing, by each multimedia platform, the generated multimedia stream to a synchronization platform; and

synchronizing, by the synchronization platform, each received multimedia stream according to a common clock reference signal determined according to a synchronization signal embedded within the received multimedia streams.

37. The method of claim 34, wherein embedding further comprises:

converting the received synchronization signal into a watermark; and

encoding the watermark onto a multimedia stream generated from a channel of a capture device used to capture multimedia scene data.

38. The method of claim 34, wherein embedding further comprises:

converting the received synchronization signal into a watermark; and

adding the watermark to audio/video data captured by a channel of a capture device used to capture multimedia scene data.

39. A computer readable storage media containing executable computer program instructions which when executed cause a digital processing system to perform a method comprising:

capturing, via a plurality of multimedia platforms, multimedia data;

receiving, during capture of the multimedia data, a synchronization signal by each of the plurality of multimedia platforms; and

embedding, by each capture device of each multimedia platform, the received synchronization signal within generated multimedia stream to form a plurality of generated multimedia stream data.

40. The computer readable storage medium of claim 39 which when executed cause a digital processing system to perform a method further comprising:

processing, by each multimedia platform, a received synchronization signal to generate a common clock reference signal among the plurality of multimedia platforms; and

synchronizing, by each multimedia platform, captured multimedia data according to the common clock reference signal.

41. The computer readable storage medium of claim 39, wherein embedding the synchronization signal causes the digital processing system to perform a method further comprising:

providing, by each multimedia platform, the generated multimedia stream to a synchronization platform; and

synchronizing, by the synchronization platform, each received multimedia stream according to a common clock reference signal determined according to the synchronization signal embedded within the received multimedia streams.

42. The computer readable storage medial of claim 39 wherein embedding the synchronization signal causes the digital processing system to perform a method further comprising:

converting the received synchronization signal into a watermark; and

encoding the watermark onto a multimedia stream generated from a channel of a capture device used to capture multimedia scene data.

43. The computer readable storage medial of claim 39, wherein embedding the synchronization signal causes the digital processing system to perform a method further comprising:

converting the received synchronization signal into a watermark; and

adding the watermark to a multimedia data captured by a channel of a capture device used to capture multimedia scene data.