System and method for automatically detecting a set of fundamental frequencies simultaneously present in an audio signal

• US 6,140,568 A
• Filed: 11/05/1998
• Issued: 10/31/2000
• Est. Priority Date: 11/06/1997
• Status: Expired due to Term

• Current Assignee: Innovative Music Systems Incorporated
• Original Assignee: Innovative Music Systems Incorporated
• Inventors: Kohler, Joseph Louis
• Primary Examiner(s): Donels, Jeffrey
• Application Number:
  US09/186,818
• Time in Patent Office: 726 Days
• Field of Search: 84/616, 84/654, 84/681, 84/645
• US Class Current: 84/616
• CPC Class Codes: G10H 1/0066 using a MIDI interface
• CPC Class Codes: G10H 1/125 using a digital filter
• CPC Class Codes: G10H 2250/235 Fourier transform; Discrete Fourier Transform [DFT]; Fast Fourier Transform [FFT]
• CPC Class Codes: G10H 3/125 Extracting or recognising the pitch or fundamental frequency of the picked up signal

Abstract

A system and method for automatically detecting and identifying a plurality of frequencies simultaneously present in an audio signal, as well as the duration, amplitude, and phase of those frequencies, then filtering out harmonic components to determine which frequencies are fundamentals. The system includes a computer readable medium of instruction code that decomposes the signal into its component sine waves by computing and comparing correlations between the input signal and sine waves at various phase and amplitude combinations. The system also employs several optimization and error correction routines.

Claims

1. A method of identifying one or more fundamental frequencies simultaneously present in a complex signal, comprising the steps of:

receiving the complex signal;

decomposing the signal into sine wave components to determine all frequencies present in the signal;

setting and obtaining parameters used to detect fundamental frequencies; and

filtering out harmonic frequencies to determine the fundamental frequencies actually present in the signal.

2. The method of claim 1, wherein the step of receiving the complex signal includes dividing the signal into a series of sample windows which contain sample amplitudes.

3. The method of claim 2, further comprising an optimization step of ignoring sample windows in which the sample amplitudes do not meet a predetermined threshold.

4. The method of claim 2, wherein the step of decomposing includes obtaining best correlation scores by comparing reference frequencies to the complex signal, and comparing said best correlation scores to determine which reference frequencies are in the complex signal.

5. The method of claim 4, wherein the step of decomposing is carried out by comparing amplitudes resulting in the best correlation score at each reference frequency to amplitudes resulting in best correlation scores at adjacent reference frequencies to locate amplitude peaks.

6. The method of claim 4, wherein the step of decomposing includes the step of scaling the amplitudes of the detected frequencies according to said frequencies.

7. The method of claim 4, wherein the step of obtaining the best correlation scores comprises shifting the amplitude and phase of the reference frequencies until an amplitude/phase combination yielding the best correlation score out of all correlation scores for said combinations of each reference frequency is found.

8. The method of claim 4, further comprising the step of attenuating frequencies lower than current reference frequencies before correlation scores are obtained.

9. The method of claim 8, wherein said attenuation step comprises:

a high-pass filter having a cutoff frequency set to the current reference frequency; and

a low-pass filter having a cutoff frequency set to one half of the current reference frequency.

10. The method of claim 7, further comprising an optimization step of ignoring amplitude/phase combinations which yield correlation scores that are worse than previously obtained correlation scores.

11. The method of claim 7, wherein the step of obtaining correlation scores includes:

subtracting samples in the current reference frequency from corresponding samples in the sample window;

squaring each difference; and

summing the results.

12. The method of claim 11, wherein the current reference frequencies are stored and utilized as a sine wave table.

13. The method of claim 11, wherein the step of obtaining correlation scores includes the step of subtracting the reference frequencies from the complex signal.

14. The method of claim 1, wherein the complex signal represents one or more musical notes.

15. The method of claim 4, further comprising an optimization step in which frequencies that are not within a range of frequencies established by predetermined parameters are disregarded.

16. The method of claim 4, further comprising an optimization step in which frequencies that are not within the frequencies represented by a predetermined musical key are disregarded.

17. The method of claim 1, wherein the step of decomposing includes:

determining the duration of all frequencies;

determining the amplitude of all frequencies; and

determining the phase of all frequencies.

18. The method of claim 4, further comprising the step of comparing frequencies identified in one sample window with frequencies identified in adjacent sample windows to ascertain whether the windows contain substantially the same set of frequencies.

19. The method of claim 18, wherein the method of computing includes:

summing the lesser amplitudes of each pair of corresponding frequencies;

dividing by the sum of the greater amplitudes of each said pair; and

comparing the result to a predetermined threshold level.

20. The method of claim 18, further comprising the step of counting successive sample windows determined to contain substantially the same set of frequencies to obtain a duration component of said frequencies.

21. The method of claim 1, wherein the step of filtering out harmonic frequencies comprises examining amplitudes of frequencies that are multiples of each previously detected frequency and subtracting a portion of each amplitude that is not greater than an amplitude of the previous harmonic and adding that to the amplitude of the previously detected frequency.

22. The method of claim 21, wherein multiples of fundamental frequencies identified in the immediately preceding sample window are examined in a first iteration, and thereafter an entire set of reference frequencies are reprocessed to identify any additional fundamental frequencies not identified in the first iteration.

23. The method of claim 21, wherein the step of filtering out harmonics further comprises comparing amplitudes of frequencies that appear to be fundamental to amplitudes of fundamental frequencies that have previously been identified to determine if a current frequency is in fact a harmonic of one of any previously identified fundamental frequencies.

24. The method of claim 21, wherein the step of filtering out harmonics employs linear regression to predict expected amplitudes in a harmonic series to at least partially correct error.

25. The method of claim 21, wherein the step of filtering out harmonics further comprises disregarding any fundamental frequencies having an amplitude below a predetermined threshold.

26. The method of claim 2, further comprising the step of reducing error caused by transition of fundamental frequencies between sample windows to determine whether each detected frequency is actually present in the signal.

27. The method of claim 26, wherein the step of reducing error disregards what appears to be identified fundamental frequencies that fail to appear in a predetermined number of successive sample windows.

28. The method of claim 1, further comprising the step of representing detected fundamental frequencies, as well as durations and amplitudes of said detected fundamental frequencies, as MIDI (Musical Instrument Digital Interface) information.

29. The method of claim 28, further comprising the step of representing tempo, time signature, key signature, and instrument patch settings as MIDI (Musical Instrument Digital Interface) information.

30. An apparatus for automatically detecting one or more fundamental frequencies simultaneously present in a complex signal, said apparatus comprising:

means for receiving the complex signal;

means for decomposing the signal into sine wave components to determine all frequencies present in the signal;

means for setting and obtaining parameters used by said apparatus; and

means for filtering out harmonic frequencies to determine the fundamental frequencies actually present in the signal.

31. The apparatus of claim 30, wherein the means for receiving the complex signal includes means for dividing the signal into a series of sample windows which contain sample amplitudes.

32. The apparatus of claim 31, further comprising an optimization means for ignoring sample windows in which the sample amplitudes do not meet a predetermined threshold.

33. The apparatus of claim 31, wherein the means for decomposing includes means for comparing best correlation scores obtained by comparing reference frequencies to the complex signal to determine which reference frequencies are in the complex signal.

34. The apparatus of claim 33, wherein the means for decomposing compares amplitudes resulting in the best correlation score at each reference frequency to amplitudes resulting in best correlation scores at adjacent reference frequencies to locate amplitude peaks.

35. The apparatus of claim 33, wherein the means for decomposing includes a means for scaling the amplitudes of the detected frequencies according to said frequencies.

36. The apparatus of claim 33, wherein the means for obtaining the best correlation scores comprises shifting the amplitude and phase of the reference frequencies until an amplitude/phase combination yielding the best correlation score out of all correlation scores for said combinations of each reference frequency is found.

37. The apparatus of claim 33, further comprising means for attenuating frequencies lower than current reference frequencies before correlation scores are obtained.

38. The apparatus of claim 37, wherein said means for attenuating comprises:

a high-pass filter having a cutoff frequency set to the current reference frequency; and

a low-pass filter having a cutoff frequency set to one half of said current reference frequency.

39. The apparatus of claim 36, further comprising an optimization means for ignoring amplitude/phase combinations which yield correlation scores that are worse than previously obtained correlation scores.

40. The apparatus of claim 36, wherein the means for obtaining correlation scores employs:

means for subtracting each sample of the current reference frequency from a corresponding sample in the sample window;

means for squaring each difference; and

means for summing the results.

41. The apparatus of claim 40, wherein the current reference frequencies are stored and utilized as a sine wave table.

42. The apparatus of claim 40, wherein the means for obtaining correlation scores includes a means for subtracting the reference frequencies from the complex signal.

43. The apparatus of claim 30, wherein the complex signal represents one or more musical notes.

44. The apparatus of claim 33, further comprising an optimization means for disregarding frequencies that are not:

within a range of frequencies established by predetermined parameters.

45. The apparatus of claim 33, further comprising an optimization means for disregarding frequencies that are not within the frequencies represented by a predetermined musical key.

46. The apparatus of claim 30, wherein the means for decomposing comprises:

means for determining the duration of all frequencies;

means for determining the amplitude of all frequencies; and

means for determining the phase of all frequencies.

47. The apparatus of claim 33, further comprising means for comparing frequencies identified in one sample window with frequencies identified in adjacent sample windows to ascertain whether the windows contain substantially the same set of frequencies.

48. The apparatus of claim 47, wherein the means for comparing includes:

means for summing the lesser amplitudes of each pair of corresponding frequencies;

means for dividing by the sum of the greater amplitudes of each said pair; and

means for comparing the result to a predetermined threshold level.

49. The apparatus of claim 47, further comprising means for counting successive sample windows determined to contain substantially the same set of frequencies to obtain a duration component of said frequencies.

50. The apparatus of claim 30, wherein the means for filtering out harmonic frequencies comprises means for examining amplitudes of frequencies that are multiples of each previously detected frequency, means for subtracting a portion of each amplitude that is not greater than the amplitude of the previous harmonic, and means for adding the portion of each amplitude that is not greater than the amplitude of the previous harmonic to the amplitude of the previously detected frequency.

51. The apparatus of claim 50, wherein multiples of fundamental frequencies identified in the immediately preceding sample window are examined in a first iteration, and thereafter an entire set of reference frequencies are reprocessed to identify any additional fundamental frequencies not identified in the first iteration.

52. The apparatus of claim 50, wherein said means for filtering out harmonics further comprises means for comparing amplitudes of frequencies that appear to be fundamental to amplitudes of fundamental frequencies that have previously been identified to determine if a current frequency is in fact a harmonic of one of any previously identified fundamental frequencies.

53. The apparatus of claim 50, wherein said means for filtering out harmonics employs linear regression to predict expected amplitudes in a harmonic series to at least partially correct error.

54. The apparatus of claim 50, wherein said means for filtering out harmonics further comprises a means for disregarding any fundamental frequencies having an amplitude below a predetermined threshold.

55. The apparatus of claim 31, further comprising means for reducing error caused by transition of fundamental frequencies between sample windows to determine whether each detected frequency is actually present in the signal.

56. The apparatus of claim 55, wherein said means for reducing error disregards what appears to be identified fundamental frequencies that fail to appear in a predetermined number of successive sample windows.

57. The apparatus of claim 55, further comprising a means for representing detected frequencies, as well as durations and amplitudes thereof, as MIDI (Musical Instrument Digital Interface) information.

58. The apparatus of claim 30, further comprising a means for representing tempo, time signature, key signature, and instrument patch settings as MIDI (Musical Instrument Digital Interface) information.