Method for analyzing music using sounds instruments

• US 6,856,923 B2
• Filed: 12/03/2001
• Issued: 02/15/2005
• Est. Priority Date: 12/05/2000
• Status: Expired due to Fees

• Current Assignee: Amusetec Co., Ltd.
• Original Assignee: Amusetec Co., Ltd.
• Inventors: Jung, Doill
• Primary Examiner(s): Barlow, John
• Assistant Examiner(s): LE, JOHN H
• Application Number:
  US10/433,051
• Publication Number:
  US 20040044487A1
• Time in Patent Office: 1,170 Days
• Field of Search: 702/66, 702/71, 702/75, 702/112, 702/124, 702/193, 844/54, 844/62, 844/64, 844/65, 846/03, 846/08, 846/09, 846/16, 846/22, 846/26
• US Class Current: 702/71
• CPC Class Codes: G10H 1/0008 Associated control or indicating means
• CPC Class Codes: G10H 2210/056 for extraction or identification of individual instrumental parts, e.g. melody, chords, bass; Identification or separation of instrumental parts by their characteristic voices or timbres
• CPC Class Codes: G10H 2220/126 for graphical editing of individual notes, parts or phrases represented as variable length segments on a 2D or 3D representation, e.g. graphical edition of musical collage, remix files or pianoroll representations of MIDI-like files
• CPC Class Codes: G10H 2240/056 MIDI or other note-oriented file format
• CPC Class Codes: G10H 2250/235 Fourier transform; Discrete Fourier Transform [DFT]; Fast Fourier Transform [FFT]
• CPC Class Codes: G10H 3/125 Extracting or recognising the pitch or fundamental frequency of the picked up signal

Abstract

A method for analyzing digital-sounds using sound-information of instruments and/or score-information is provided. Particularly, sound-information of instruments which were used or which are being used to generate input digital-sounds is used. Alternatively, in addition to the sound-information, score-information which were used or which are being used to generate the input digital-sounds is also used. According to the method, sound-information including pitches and strengths of notes performed on instruments used to generate the input digital-sounds is stored in advance so that monophonic or polyphonic pitches performed on the instruments can be easily analyzed. Since the sound-information of instruments and the score-information are used together, the input digital-sounds can be accurately analyzed and output as quantitative data.

Claims

1. A method for analyzing digital-sounds using sound-information of musical-instruments, the method comprising the steps of:

(a) generating and storing sound-information of different musical instruments;

(b) selecting the sound-information of the particular instrument to be actually played from among the stored sound-information of different musical-instruments;

(c) receiving digital-sound-signals;

(d) decomposing the digital-sound-signals into frequency-components in units of frames;

(e) comparing the frequency-components of the digital-sound-signals with frequency-components of the selected sound-information of the particular instrument and analyzing the frequency-components of the digital-sound-signals to detect monophonic-pitches-information from the digital-sound-signals; and

(f) outputting the detected monophonic-pitches-information.

2. The method of claim 1, wherein the step (e) comprises detecting time-information of each frame, comparing the frequency-components of the digital-sound-signals with the frequency-components of the selected sound-information of the particular instrument and analyzing the frequency-components of the digital-sound-signals in units of frames, and detecting pitch-information, strength-information, and time-information of each of individual pitches contained in each of the frames.

3. The method of claim 1 or 2, wherein the step (e) comprises the steps of:

(e1) selecting the lowest peak frequency-components contained in a current frame of the digital-sound-signals;

(e2) detecting the sound-information containing the lowest peak frequency-components from the selected sound-information of the particular instrument;

(e3) detecting, as monophonic-pitches-information, the sound-information containing most similar peak frequency-components to those of the current-frame from among the detected sound-information in step (e2);

(e4) removing the frequency-components of the sound-information detected as the monophonic-pitches-information in step (e3) from the current-frame; and

(e5) repeating steps (e1) through (e4) when there are any peak frequency-components left in the current-frame.

4. The method of claim 2, wherein the step (e) further comprises determining whether the detected monophonic-pitches-information contains any new-pitch which is not included in a previous-frame, dividing a current-frame including the new-pitch into subframes if it is determined that the detected monophonic-pitches-information contains the new-pitch, finding a subframe including the new-pitch, and detecting pitch-information and strength-information of the new-pitch and time-information of the found subframe.

5. The method of claim 1, wherein the step (a) comprises periodically updating the sound-information of different musical instruments.

6. The method of claim 1, wherein the step (a) comprises storing each individual pitch which can be expressed by the sound-information in the form of wave data when storing the sound-information of different musical instruments in the form of samples of sounds having at least one strength, and extracting the frequency-components of the sound-information of different musical instruments from the wave data stored.

7. The method of claim 1, wherein the step (a) comprises storing each individual pitch which can be expressed by the sound-information in a form which can directly expressing the magnitudes of each frequency-components of the pitch when storing the sound-information of different musical instruments in the form of samples of sounds having at least one strength.

8. The method of claim 6 or 7, wherein the step (a) comprises separately storing sound-information of keyboard-instruments according to use/nonuse of pedals.

9. The method of claim 6 or 7, wherein the step (a) comprises separately storing sound-information of string-instruments by each string.

10. The method of claim 7, wherein the step (a) comprises performing Fourier transform on the sound-information of different musical instruments and storing the sound-information in a form in which the sound-information can be directly displayed.

11. The method of claim 7, wherein the step (a) comprises performing wavelet transform on the sound-information of different musical instruments and storing the sound-information in a form in which the sound-information can be directly displayed.

12. A method for analyzing digital-sounds using sound-information of musical-instruments and score-information, the method comprising the steps of:

(a) generating and storing sound-information of different musical instruments;

(b) generating and storing score-information of a score to be performed;

(c) selecting the sound-information of the particular instrument to be actually played and the score-information of the score to be actually performed from among the stored sound-information of different musical instruments and the stored score-information;

(d) receiving digital-sound-signals;

(e) decomposing the digital-sound-signals into frequency-components in units of frames;

(f) comparing the frequency-components of the digital-sound-signals with frequency-components of the selected sound-information of the particular instrument and the selected score-information, and analyzing the frequency-components of the digital-sound-signals to detect performance-error-information and monophonic-pitches-information from the digital-sound-signals; and

(g) outputting the detected monophonic-pitches-information.

13. The method of claim 12, wherein the step (f) comprises detecting time-information of each-frame, comparing the frequency-components of the digital-sound-signals with the frequency-components of the selected sound-information of the particular instrument and the selected score-information, analyzing the frequency-components of the digital-sound-signals in units of frames, and detecting pitch-information, strength-information, and time-information of each of individual pitches contained in each of the frames.

14. The method of claim 12 or 13, wherein the step (f) further comprises determining whether the detected monophonic-pitches-information contains any new-pitch which is not included in a previous frame, dividing a current frame including a new-pitch into subframes if it is determined that the detected monophonic-pitches-information contains the new-pitch, finding a subframe including the new-pitch, and detecting pitch-information and strength-information of the new-pitch and time-information of the found subframe.

15. The method of claim 12 or 13, wherein the step (f) comprises the steps of:

(f1) generating expected-performance-values of the current-frame referring to the score-information in real time; and

determining whether there is any note in the expected-performance-values which is not compared with the digital-sound-signals in the current-frame;

(f2) if it is determined that there is no note in the expected-performance-value which is not compared with the digital-sound-signals in the current-frame in step (f1), determining whether frequency-components of the digital-sound-signals in the current-frame correspond to performance-error-information, detecting performance-error-information and monophonic-pitches-information, and removing the frequency-components of the sound-information corresponding to the performance-error-information and the monophonic-pitches-information from the digital-sound-signals in the current-frame;

(f3) If it is determined that there is any note in the expected-performance-value which is not compared with the digital-sound-signals in the current-frame in step (f1), comparing the digital-sound-signals in the current-frame with the expected-performance-values and analyzing to detect monophonic-pitches-information from the digital-sound-signals in the current-frame, and removing the frequency-components of the sound-information detected as the monophonic-pitches-information from the digital-sound-signals in the current-frame; and

(f4) repeating steps (f1) through (f4) when there are any peak frequency-components left in the current-frame of the digital-sound-signals.

16. The method of claim 15, wherein the step (f2) comprises the steps of:

(f2_—

1) selecting the lowest peak frequency-components contained in the current-frame of the digital-sound-signals;

(f2_—

2) detecting the sound-information containing the lowest peak frequency-components from the selected sound-information of the particular instrument;

(f2_—

3) detecting, as performance-error-information, the sound-information containing most similar peak frequency-components to peak frequency-components of the current-frame from the detected sound information;

(f2_—

4) if it is determined that the current pitches of the performance-error-information are contained in next notes in the score-information, adding the current pitches of the performance-error-information to the expected-performance-value and moving the current pitches of the performance-error-information into the monophonic-pitches-information; and

(f2_—

5) removing the frequency-components of the sound-information detected as the performance-error-information or the monophonic-pitches-information from the digital-sounds in the current-frame.

17. The method of claim 16, wherein the step (f2_—

3) comprises detecting the pitch and strength of a corresponding performed note as the performance-error-information.

18. The method of claim 16, wherein the step (f3_—

3) comprises removing an expected-performance-value corresponding to the selected sound-information whose frequency-components are included in the digital-sound-signals at one or more time points but are not included in at least a predetermined number (N) of consecutive previous frames.

19. The method of claim 15, wherein the step (f3) comprises the steps of:

(f3_—

1) selecting the sound-information of the lowest peak frequency-components which is not compared with frequency-components contained in the current-frame of the digital-sound-signals from the sound-information corresponding to the expected-performance-value which has not undergone comparison;

(f3_—

2) if it is determined that the frequency-components of the selected sound-information are included in frequency-components contained in the current-frame of the digital-sound-signals, detecting the selected sound-information as monophonic-pitches-information and removing the frequency-components of the selected sound-information from the current-frame of the digital-sound-signals; and

(f3_—

3) if it is determined that the frequency-components of the selected sound-information are not included in the frequency-components contained in the current-frame of the digital-sound-signals, adjusting the expected-performance-value.

20. The method of claim 12, wherein the step (a) comprises periodically updating the sound-information of different musical instruments.

21. The method of claim 12, wherein the step (a) comprises storing each individual pitch which can be expressed by the sound-information in the form of wave data when storing the sound-information of different musical instruments in the form of samples of sounds having at least one strength.

22. The method of claim 12, wherein the step (a) comprises storing each individual pitch which can be expressed by the sound-information in a form which can directly expressing the magnitudes of each frequency-components of the pitch when storing the sound-information of different musical instruments in the form of samples of sounds having at least one strength.

23. The method of claim 21 or 22, wherein the step (a) comprises separately storing sound-information of keyboard-instruments according to use/nonuse of pedals.

24. The method of claim 21 or 22, wherein the step (a) comprises separately storing sound-information of string-instruments by each string.

25. The method of claim 22, wherein the step (a) comprises performing Fourier transform on the sound-information of different musical instruments and storing the sound-information in a form in which the sound-information can be directly displayed.

26. The method of claim 22, wherein the step (a) comprises performing wavelet transform on the sound-information of different musical instruments and storing the sound-information in a form in which the sound-information can be directly displayed.

27. The method of claim 12, further comprising the step of (h) estimating performance accuracy based on the performance-error-information detected in step (f).

28. The method of claim 12, further comprising the step of (i) adding the individual notes of the performance-error-information to the existing score-information based on the performance-error-information detected in step (f).

29. The method of claim 12, wherein the step (b) comprises generating and storing at least one kind of information selected from the group consisting of pitch-information, note-length-information, speed-information, tempo-information, note-strength-information, detailed performance-information including staccato, staccatissimo, and pralltriller, and discrimination-information for performance using two-hands or performance using a plurality of instruments, based on the score to be performed.