Method and system for synchronization of an auxiliary sound source to motion picture film, video tape, or other picture source containing a sound track

• US 4,839,733 A
• Filed: 12/15/1987
• Issued: 06/13/1989
• Est. Priority Date: 12/15/1987
• Status: Expired due to Term

• Current Assignee: Daniel W. Gravereaux, John J. Karamon
• Original Assignee: Daniel W. Gravereaux, John J. Karamon
• Inventors: Gravereaux, Daniel W., Karamon, John J.
• Primary Examiner(s): MCELHENY JR, DONALD E
• Application Number:
  US07/132,295
• Time in Patent Office: 546 Days
• Field of Search: 358/311, 358/335, 358/341, 360/12, 360/14.1, 360/14.3, 360/15, 360/31, 360/79, 360/80, 360/61, 360/38.1, 360/33.1, 360/26, 369/54, 369/58, 369/70, 369/43, 369/44, 369/83, 369/84, 369/85, 352/31, 352/131
• US Class Current: 386/203
• CPC Class Codes: G03B 31/04 in which sound track is not on, but is synchronised with, a moving-picture film
• CPC Class Codes: G11B 2220/2545 CDs
• CPC Class Codes: G11B 2220/90 Tape-like record carriers
• CPC Class Codes: G11B 2220/913 Digital audio tape [DAT] format
• CPC Class Codes: G11B 27/002 Programmed access in sequence to a plurality of record carriers or indexed parts, e.g. tracks, thereof, e.g. for editing; transfer of record carriers from magazine G11B15/68, G11B17/10; G11B17/22
• CPC Class Codes: G11B 27/038 Cross-faders therefor
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: H04N 9/802 involving processing of the sound signal H04N9/806, H04N9/835 take precedence

Abstract

A method and system for synchronizing a digital signal of an audio message at higher quality with another signal of the same audio message at lower quality. The invention synchronizes an auxiliary high quality audio digital sound signal source with the ordinary quality analog sound signal coming from a conventional analog sound track on motion picture film as the motion picture is being shown and maintains effective synchronization in spite of missing segments of the sound track due to film splices. The invention does not require synchronization tracks, markers, codes, time codes or other extrinsic data. The only requirement is that the higher qualtiy audio message recording, e.g. in a digital medium, be made from the same "master recording" (or a high quality duplicate thereof) as the sound track so that the auxiliary recording have the same informational content as the sound track. In order to maintain synchronization of the high quality audio message in spite of missing segments of the lower quality audio message, the higher quality message is caused to precede the lower quality message, and is temporarily stored in time-delay means, such as a FIFO overwriting memory store. Correlation is achieved by re-iterative subtractions between digital numbers representative of characteristics of the absolute value envelope of the ordinary quality message signal and digital numbers representative of characteristics of the absolute value envelope of the high quality message. When the missing segments of the cinema sound track amount to more than the time-delay storage capability of the time-delay means, the system immediately smoothly "fades" over to utilize the conventional analog sound track of the cinema film for uninterrupted sound accompanying the motion picture being viewed by the theater audience. Re-synchronism is quickly achieved, and the system immediately automatically smoothly "fades" back to the higher quality sound.

Claims

1. In the showing of viewable images depicting motion and wherein the viewable images are provided by operation of a first medium containing first means providing a first sound signal to provide an audio message of first quality accompanying the viewable images and wherein said means for providing said first sound signal are made from a master recording, the method of synchronizing with said first sound signal a second sound signal of higher quality than said first quality but containing the same audio message to accompany the showing of the viewable images, said method comprising the steps of:

providing a second medium containing second means for providing said second sound signal;

making said second means from the master recording for enabling said second means to provide said second sound signal of said higher quality but containing the same audio message;

starting operation of said second medium a predetermined time interval before commencing the operation of the first medium;

thereafter simultaneously operating both said second medium and said first medium for simultaneously providing both said second sound signal and said first sound signal but wherein the audio message of said second sound signal is preceding by said predetermined time interval the audio message of said first sound signal;

accessibly placing in temporary storage a continuously updated current portion of said second sound signal occurring over a current time interval at least equal to said predetermined time interval; and

continiously removing from temporary storage successive increments of said second sound signal correlated with the currently occurring first sound signal for providing said second sound signal synchronized with said first sound signal; and

utilizing the synchronized second sound signal for providing the same audio message as said first sound signal but at a higher quality to accompany the showing of the viewable images.

2. The method of synchronizing with a first sound signal a second sound signal of higher quality as claimed in claim 1, wherein:

said predetermined time interval is at least 3 seconds.

3. The method of synchronizing with a first sound signal a second sound signal of higher quality as claimed in claim 2, wherein:

said predetermined time interval is in the range between about 6 seconds and about 10 seconds.

4. The method of synchronizing with a first sound signal a second sound signal of higher quality as claimed in claim 1, wherein:

the successive increments of said second sound signal are correlated with the currently occurring first sound signal by the steps of;

deriving a first sequence of values representative of the characteristics of the syllabic structure of the currently occurring first sound signal;

deriving a second sequence of values representative of the syllabic structure of said current portion of said second sound signal in temporary storage;

selecting those successive values of the second sequence which correlate with said first sequence of values; and

removing from temporary storage those successive increments of said second signal whose values correlate with said first values.

5. The method of synchronizing with a first sound signal a second sound signal of higher quality as claimed in claim 4, wherein:

said step of selecting those successive values of the second sequence which correlate with said first sequence of values includes the steps of;

performing subtractions between the values of said first sequence and the values of said first sequence and the values the said second sequence; and

determining the minimum differences resulting from the subtractions for selecting those successive values of the second sequence which correlate with said first sequence of values.

6. The method of synchronizing with a first sound signal a second sound signal of higher quality as claimed in claim 1, wherein said first sound signal is an analog sound signal and said second sound signal is a digital sound signal, said method including the steps of:

converting said first sound signal, which is an analog sound signal, into a first sequence of numbers representative of the envelope of said first sound signal;

converting said current portion of said second sound signal, which is a digital sound signal, into a second sequence of numbers representative of the envelope of said second sound signal;

performing re-iterative subtractions between said numbers of said first sequence and said numbers of said second sequence;

determining the minimum differences continuously resulting from said re-iterative subtractions; and

removing from temporary storage successive increments of said second sound signal corresponding with numbers of said second sequence yeilding said minimum differences for providing said second sound signal synchronized with said first sound signal.

7. The method of synchronizing with a first sound signal a second sound signal of higher quality as claimed in claim 1 including the steps of:

converting said first sound signal into a first sequence of numbers representative of the audio pattern of said first sound signal;

converting said current portion of said second sound signal into a second serquence of numbers representative of the audio pattern of said second sound signal;

performing re-iterative subtractions between said numbers of said first sequence of numbers and said numbers of said second sequence;

determining the minimum differences continuously resulting from said re-iterative subtractions; and

removing from temporary storage successive increments of said second sound signal corresponding with numbers of said second sequence yielding said minimum differences for providing said second sound signal synchronised with said first sound.

8. The method claimed in claim 6, including the steps of:

converting said first sound signal into a first sequence of digital numbers representative of the syllabic structure of the envelope of said first sound signal;

converting said current portion of said second sound signal into a second sequence of digital numbers representative of the syllabic structure of the envelope of said second sound signal;

performing re-iterative subtractions between said digital numbers of said first sequence and said digital numbers of said second sequence; and

determining the minimum differences continuously resulting from said re-iterative subtractions between said digital numbers of said first and second sequences of digital numbers.

9. The method claimed in claim 6, including the steps of:

converting said first sound signal into a first sequence of digital numbers representative of the absolute value envelope of said first sound signal;

converting said current portion of said second sound signal into a second sequence of digital numbers representative of the absolute value envelope of said second sound signal;

performing re-iterative subrractions between said digital numbers of said first sequence and said digital numbers of said second sequence; and

determining the minimum differences resulting from said re-iterative subtractions.

10. The method of claim 7, including the steps of:

converting said first sound signal into a first sequence of digital numbers representative of the absolute value envelope of the audio pattern of said first sound signal;

converting the current portion of said second sound signal into a second sequence of digital numbers representative of the absolute value envelope of the audio pattern of said second sound signal;

performing re-iterative subtractions between said digital numbers of said first sequence and said digital numbers of said second sequence;

determining the minimum differences continuously resulting from said re-iterative subtractions; and

removing from temporary storage successive increments of said second sound signal corresponding with said digital numbers of said second sequence yielding said minimum differences resulting from said second sound signal for providing said second sound signal synchronized with said first sound signal.

11. The method of claim 6, wherein:

said first sequence of numbers is representative of frequency-limited characteristics of the envelope of said first sound signal;

said second sequence of numbers is representative of frequency-limited characteristics of the envelope of said current portion of said second sound signal; and

said frequency-limited characteristics are both limited below the same predetermined upper frequency limit.

12. The method of claim 11, wherein:

said predetermined upper frequency limit is 50 Hz.

13. The method claim 7, wherein:

said first sequence of numbers is representative of audio pattern characteristics limited below a predetermined upper frequency limit; and

said second sequence of numbers is representative of audio pattern characteristics limited below said predetermined upper frequency limit.

14. The method of claim 13, wherein:

said predetermined upper frequency limit is 50 Hz.

15. The method of claim 8, wherein:

said first sequence of digital numbers is representative of syllabic structure characteristics of the envelope of said first sound, said characteristics being confined to those occurring below a predetermined upper limit;

said second sequence of digital numbers is representative of syllabic structure characteristics of the envelope of said second sound signal, said characteristics being confined to those occurring below said predetermined upper frequency limit.

16. The method of claim 15, wherein:

said predetermined upper frequency limit is 50 Hertz.

17. The method of claim 9, wherein:

said first sequence of digital numbers is representative of absolute value envelope characteristics limited below a predetermined upper frequency limit; and

said second sequence of digital numbers is representative of absolute value envelope characteristics limited below said predetermined upper frequency limit.

18. The method of claim 17, wherein:

said predetermined upper frequency limit is 50 Hertz.

19. The method of claim 10, wherein:

said first sequence of digital numbers is representative of absolute value envelope characteristics limited below a predetermined upper frequency limit; and

said second sequence of digital numbers is representative of absolute value envelope characteristics limited below said predetermined upper frequency limit.

20. The method of claim 19, wherein:

said predetermined frequency limit is 50 Hertz.

21. The method of synchronizing with an analog signal of an audio message a digital signal of said audio message, and wherein the audio message of said analog signal is derived from a master recording, said method comprising the steps of:

causing said digital signal of the audio message to be derived from the master recording; and

correlating intrinsic characteristics of said audio message of said digital signal with intrinsic characteristics of said audio message of said analog signal for synchronizing said digital signal with said analog signal.

22. The method of claim 21, wherein:

said intrinsic characteristics comprise syllabic structure characteristics of said audio message of said digital and analog signals, respectively.

23. The method of claim 22, wherein:

said syllabic structure characteristics are limited to those occurring below a predetermined upper frequency limit.

24. The method of claim 23, wherein:

said predetermined frequency limit is 50 Hertz.

25. The method of claim 24, including the steps of:

correlating said intrinsic characteristics of said analog signal by re-iterative subtractions;

wherein lowest continuous differences resulting from said re-iterative subtractions are indicative of desired correlation.

26. The method of claim 21, wherein:

said intrinsic characteristics comprise audio pattern characteristics of said digital and analog signals, respectively.

27. The method of claim 21 wherein:

said intrinsic characteristics comprise syllabic structure characteristics of the absolute value envelope of said digital and analog signals, respectively.

28. The method of claim 21 wherein:

said intrinsic characteristics comprise audio pattern characteristics of the absolute value envelope of said digital and analog signals, respectively, limited to below a predetermined upper frequency limit.

29. The method of claim 28 wherein:

said predetermined upper frequency limit is 50 Hertz.

30. The method of claim 27 including the steps of:

causing the audio message of said digital signal to precede the audio message of said analog signal by a predetermined time difference;

temporarily storing a continuously updated current segment of said digital signal at least equal in time duration to said time difference; and

withdrawing from said current segment of said digital signal portions if said current segment having intrinsic characteristics correlating with intrinsic characteristics of said analog signal for synchronizing said digital signal with said analog signal.

31. Method of claim 30, wherein:

said intrinsic characteristics comprise syllabic structure characteristics of said audio message of said digital and analog signals, respectively.

32. Method of claim 39, wherein:

said syllabic structure characteristics are limited to those occurring below a predetermined upper frequency limit.

33. The method of claim 32, wherein:

said predetermined upper frequency limit is 50 Hertz.

34. The method of claim 30, including the steps of:

correlating said intrinsic characteristics of said portions of said current segment of said digital signal with said intrinsic characteristics of said analog signal by re-iterative subtractions; and

wherein lowest differences resulting from said re-iterative subtractions are used for determining correlation.

35. The method of claim 30, whrein:

said predetermined time difference is at least about three seconds.

36. The method of claim 30, wherein:

said predetermined time difference is in the range from about 6 seconds to about 10 seconds.

37. The method of claim 21, wherein said analog signal of the audio message comprises respective signals for enabling the audio message thereof to be reproduced with left (L) and right (R) sound components and whrein said digital signal of the audio message also comprises a plurality of respective signals for enabling the audio message thereof to be reproduced with left (L) and right (R) sound components, said method including the steps of:

adding together characteristics of respective signals of said analog signal for providing a first monaural signal containing audio message information from a plurality of the analog channel signals,

adding together characteristics of respective signals of said digital signal for providing a second monaural signal containing audio message information from a plurality of the digital channel signals; and

correlating intrinsic characteristics of said second monaural signal with intrinsic characteristics of said first monaural signal for synchronizing said digital signal comprising a plurality of channel signals with said analog signal comprising a plurality of channel signals.

38. The method of claim 37, including the steps of:

causing the audio message of said digital signal to be advanced in time relative to the audio message of said analog signal by at least a predetermined time interval,

temporarily storing continuously updated current segments of the respective signals of said digital signal,

said continuously updated current segments having an audio message time length at least equal to the current advance in time of the audio message of said digital signal relative to the audio message of said analog signal,

termporarily storing a continuously updated current segment of said second monaural signal,

said continuously updated current segment of said second monaural signal exactly relating with said continously updated segments of the respective signals of said digital signal,

selectively removing from temporary storage those portions of said continuosly updated segments of the respective channel signals of said digital signal having exactly relating portions of said second monaural signal whose intrinsic characteristics are correlating with the intrinsic characteristics of said first monaural signal, and

using the selectively removed portions of said continuously updated segments of the respective signals of said digital signal for providing the digital signal comprising a plurality of respective signals synchronized with said analog signal comprising a plurality of respective signals.

39. The method of claim 37, wherein:

said first monaural signal is a digital signal,

said second monaural signal is a digital signal, and

intrinsic characteristics of said digital second monaural signal are correlated with intrinsic characteristics of said digital first monaural signal.

40. The method of claim 38, wherein:

said first monaural signal is a digital signal,

said second monaural signal is a digital signal, and

intrinsic characteristics of said digital second monaural signal are correlated with intrinsic characteristics of said digital first monaural signal.

41. The method of claim 39, including the step of:

correlating intrinsic characteristics of said digital second monaural signal with intrinsic characteristics of said digital first monaural signal by re-iterative subtraction seeking portions of said digital second monaural signal yielding the lowest differences resulting from re-iterative subtraction, and

using those portions of said digital signal comprising a plurality of respective signals exactly corresponding with those portions of said digital second monaural signal yielding the lowest differences for providing the digital signal comprising a plurality of respective signals synchronized with said analog signal comprising a plurality of respective signals.

42. The method of claim 40, including the step of:

correlating intrinsic characteristics of said digital second monaural signal with intrinsic characteristics of said digital first monaural signal by re-iterative subtraction seeking those portions of said digital second monaural signal yielding the lowest differences resulting from re-iterative subtraction.

43. The method of synchronizing a second signal having an audio message at higher quality with a first signal having said audio message at lower quality and wherein said first signal is derived from a master recording, said method being capable of accommodating discontinuities in said first signal and comprising the steps of:

cuasing said second signal to be derived from the master recording,

causing the higher quality audio message of the second signal to precede in time the lower quality audio message of the first signal by at least a predetermined interval of time,

temporarily storing a continuously updated current segment of the higher quality second audio message,

said continuously updated current segment of the higher quality second audio message having a time duration at least equal to the current interval of time by which the higher quality second audio message signal is preceding the lower quality first audio message signal,

selectively removing from temporary storage those portions of said higher quality second audio message having intrinsic characteristics correlating with intrinsic characteristics of said lower quality first audio message, and

using the selectively removed portions of said higher quality second audio message for providing said second signal having the audio message at higher quality synchronized with the audio message at lower quality of said first signal.

44. The method of claim 43, including the steps of:

deriving a first sequence of indicia indicative of intrinsic characteristics of said lower quality audio message signal,

deriving a second sequence of indicia indicative of intrinsic characteristics of said higher quality audio message signal,

correlating intrinsic characteristics of said portions of said higher quality second audio message signal with intrinsic characteristics of said lower quality first audio message signal by correlating portions of said second sequence of indicia with said first sequence of indicia, and

selectively removing from temporary storage those portions of said higher quality second audio message corresponding with correlated portions of said second sequence of indicia.

45. The method of claim 44, wherein:

said first sequence of indicia is indicative of syllabic structures of said lower quality audio message signal, and

said second sequence of indicia is indicative of syllabic structures of said higher quality audio message signal.

46. The method of claim 45, wherein:

said first sequence of indicia is indicative of syllabic structures of an absolute value envelope of said lower quality audio message signal, and

said second sequence of indicia is indicative of syllabic structures of an absolute value envelope of said higher quality audio message signal.

47. The method of claim 46, wherein:

said first sequence of indicia is indicative of syllabic structures of an absolute value envelope of said lower quality audio message signal below a predetermined upper frequency limit, and

said second sequence of indicia is indicative of syllabic structures of an absolute value envelope of said higher quality audio message below said predetermined upper frequency limit.

48. The method of claim 47, wherein:

said predetermined upper frequency limit is 50 Hertz.

49. the method of claim 44, wherein:

said indicia of said first sequence and said indicia of said second sequence involve magnitude values,

said step of correlating portions of said second sequence of indicia with said first sequence of indicia comprise re-iterative subtractions of magnitude values, and

wherein lowest differences determine that said portions of said second sequence of indicia are correlating with said first sequence of indicia.

50. The method of claim 49, in which:

said current interval of time is at least about 3 seconds.

51. The method of claim 50, in which:

said current interval of time is in the range from about 6 seconds to about 10 seconds.

52. A system for synchronizing an audio message signal at higher quality with an audio message signal at lower quality, wherein said audio message at lower quality is derived from a master recording said system comprising:

a medium for providing said audio message at higher quality made from the master recording,

contorl means for causing said medium to provide the higher quality audio message signal preceding in time the lower quality audio message signal by at least a predetermined interval of time,

time-delay means for temporarily storing a continuously updated current segment of the higher quality audio message signal having a time-delay capability at least equal to said predetermined interval of time,

said control means including correlating means associated with said time-delay means for selectively removing from said time-delay means those portions of said higher quality audio message signal having intrinsic characteristics correlating with intrinsic characteristics of said lower quality audio message signal for providing said higher quality audio message signal synchronized with said lower quality audio message signal, and

whereby said system is capable of accommodating discontinuities in said lower quality audio message signal.

53. The system of claim 52, wherein:

said time-delay means includes a FIFO overwriting memory store.

54. The system of claim 52, wherein:

said time-delay means includes first delay means for temporarily storing said continuously updated current segment of the higher quality audio message signal, and said time-delay means includes a second delay means,

said second delay means has a time-delay capability at least equal to the time-delay capability of said first delay means,

said second delay means operates synchronously with said first delay means,

said system further comprising;

first processor means for deriving a first sequence of indicia indicative of intrinsic characteristics of said lower quality audio message signal,

second processor means for deriving a second sequence of indicia indicative of intrinsic characteristics of said higher quality audio message signal,

said second delay means being coupled to said second processor means for temporarily storing a continuously updated current segment of said second sequence of indicia relating to the continuously updated current segments of the higher quality audio message temporarily stored in said first delay means, and

said correlating means being associated with said second delay means for determining those portions of said second sequence of indicia which are correlated with said first sequence of indicia for causing said control means to remove from said second delay means those portions of the continuously updated temporarily stored current segment of the higher quality audio message relating to said correlated portions of said continuously updated temporarily stored second sequence of indicia.

55. The system of claim 54, wherein:

said first delay means comprises a first FIFO overwriting memory store, and

said second delay means comprises a second FIFO overwriting memory store less complex than said first FIFO overwriting memory store.

56. The system of claim 54, further comprising:

selection means for selecting most significant bits of digital words in the higher quality audio message signal, and wherein;

said selection means feeds selected most significant bits of the digital words in the higher quality audio message signal into said second delay means.

57. A system for providing a viewing audience with high quality sound synchronized with viewable images having an ordinart sound signal track associated with said viewable images for providing ordinary sound synchronized with said viewable images, said system comprising:

auxiliary means for providing a high quality signal for providing said high quality sound,

control means coupled to said auxiliary means for controlling said auxiliary means,

time-delay means coupled to said auxiliary means for temporarily storing a continuously updated current segment of said high quality signal, and

said control means including correlating means for correlating portions of said current segment with said ordinary sound signal track for providing a high quality sound signal synchronized with said ordinary sound signal track for providing said high quality sound synchronized with the viewable images,

whereby said system maintains synchronization of said high quality sound with said viewable images in spite of discontinuities in said ordinary sound signal track.

58. A system as claimed in claim 57, further comprising:

smooth transition fader means coupled to said time-delay means and coupled to said ordinary sound signal track,

said fader means being controlled by said control means for providing ordinary quality sound synchronized with the viewable images during momentary intervals wherein synchronization of the high quality sound is lost due to unusually lengthy discontinuities in said ordinary quality sound signal track, and

whereby the viewing audience hears effectively continuous sound appropriately synchronized with the viewable images throughout a presentation thereof.

59. A system as claimed in claim 58, wherein:

said control means causes said auxiliary means to operate at increased speed during such momentary intervals of loss of synchronization of high quality sound for causing said auxiliary means to supply to said time-delay means an appropriately updated current segment of said high quality sound signal.

60. A system as claimed in claim 58, wherein:

said control means causes said auxiliary means normally to operate at a predetermined time advance relative to said ordinary sound signal track, and

said control means causes said auxiliary means to reestabish said predetermined time advance when said synchronization of said high quality sound has been momentarily lost.