Real time music recognition and display system

• US 20050015258A1
• Filed: 07/16/2003
• Published: 01/20/2005
• Est. Priority Date: 07/16/2003
• Status: Active Grant

• Current Assignee: Iowa State University Research Foundation Incorporated (Iowa State University)
• Original Assignee: Iowa State University Research Foundation Incorporated (Iowa State University)
• Inventors: Zhao, Liang, Somani, Arun, Adhami, Raed, Sahai, Anil, Tao, Wu
• Granted Patent: US 7,323,629 B2
• Time in Patent Office: Days
• US Class Current: 704/278
• CPC Class Codes: G10H 1/0008 Associated control or indicating means
• CPC Class Codes: G10H 2240/165 Memory card, i.e. removable module or card for storing music data for an electrophonic musical instrument
• CPC Class Codes: G10H 2250/235 Fourier transform; Discrete Fourier Transform [DFT]; Fast Fourier Transform [FFT]
• CPC Class Codes: G10H 3/125 Extracting or recognising the pitch or fundamental frequency of the picked up signal

Abstract

Systems and methods for performing simple and quick real time single music note recognition algorithm based on fuzzy pattern matching are disclosed. In one aspect, the systems and methods use a 256-point FFT and fuzzy pattern identification and recognition method. The systems and methods can recognize a note as short as 0.125 seconds in a frequency range from 16 Hz to 4000 Hz, with 11.025 KHz sampling rate and 8-bit per sampling signal. The systems and methods may be used as part of a music tutor system that receives a played note, identifies the played note, and compares the played note with a reference note. An indication may be given as to whether the played note matched the reference note.

Claims

1. A computerized method for recognizing music, the method comprising:

receiving an input data representing a played note;

performing time alignment on the input data;

extracting features from the input data; and

comparing the extracted features to a dataset of saved note features to determine a matching note.

2. The method of claim 1, wherein the input data is analog data and further comprising performing an analog to digital conversion of the input data.

3. The method of claim 1, wherein performing time alignment include performing a FFT on the input data.

4. The method of claim 3, wherein the FFT comprises a 512 point FFT.

5. The method of claim 1, wherein the played note matches the current note if a set of note features for the played note match a set of note features for the current note.

6. The method of claim 5, wherein the set of note features includes a fundamental frequency.

7. The method of claim 5, wherein the set of note features includes note-duration.

8. The method of claim 5, wherein the set of note features includes at least one harmonic frequency.

9. The method of claim 8, wherein the set of note features includes at least 5 harmonic frequencies.

10. The method of claim 5, wherein the set of note features includes at least one peak location and at least one peak value.

11. The method of claim 10, wherein the comparing includes weighting the at least one peak location and the at least one peak value.

12. The method of claim 1, wherein performing time alignment includes determining a start point and an end point of a note in the input data.

13. The method of claim 12, wherein a sum of the square of the amplitude is used to determine the start point.

14. The method of claim 12, wherein the sum of the absolute amplitude is used to determine the start point.

15. A computerized method for providing a music tutor, the method comprising:

retrieving a set of musical data comprising one or more reference notes;

displaying at least a portion of the set of musical data, said portion including a current note from the one or more reference notes;

receiving a played note;

comparing the played note to the current note; and

displaying an indication of whether the played note matches the current note.

16. The computerized method of claim 15, wherein displaying an indication changes the color of the reference note in accordance with whether the played note matched the reference note.

17. The computerized method of claim 15, further comprising training a system to recognize a set of notes played by a musical instrument.

18. The computerized method of claim 15, further comprising composing the set of reference notes.

19. The computerized method of claim 15, wherein the reference notes are included on a flash card.

20. The computerized method of claim 15, wherein the reference notes are included on a musical segment.

21. The computerized method of claim 15 wherein displaying an indication of whether the played note matches the current note includes highlighting correctly played notes in a first highlight and highlighting incorrectly played notes in a second highlight.

22. The computerized method of claim 21, wherein the first highlight is a first color and the second highlight is a second color.

23. The computerized method of claim 21, wherein the first highlight is a first cross-hatching and the second highlight is a second cross-hatching.

24. A computerized system comprising:

a processor and a memory coupled to the processor;

an analog to digital (A/D) converter coupled to the processor;

a sound input device coupled to the A/D converter;

a database; and

a display;

wherein the analog to digital converter is operable to receive sound input from the sound input device and wherein the processor is operable to;

receive a set of data from the A/D converter, said data representing at least one note, and

identify the note based on matching the data representing at least one note to the set of database data, said identification occurring in near real-time.

25. The system of claim 24, wherein the A/D converter is included in a sound card.

26. The system of claim 24, wherein the sound input device is a microphone.

27. The system of claim 24, wherein the sound input device is a MIDI compatible device.

28. The system of claim 24, wherein the display is an LCD (Liquid Crystal Display).

29. The system of claim 24, wherein the processor is further operable to output a musical segment comprising at least one note to the display.

30. The system of claim 24, wherein the processor, the memory, the A/D converter and the display are incorporated on a single board computer.

31. The system of claim 24, wherein the processor, the memory, the A/D converter and the display are incorporated in a personal computer.

32. A computer-readable medium having computer-executable instructions for performing a method for recognizing music, the method comprising:

receiving an input data representing a played note;

performing time alignment on the input data;

extracting features from the input data; and

comparing the extracted features to a dataset of saved note features to determine a matching note.

33. The computer-readable medium of claim 32, wherein the input data is analog data and wherein the method further comprises performing an analog to digital conversion of the input data.

34. The computer-readable medium of claim 32, wherein performing time alignment include performing a FFT on the input data.

35. The computer-readable medium of claim 34, wherein the FFT comprises a 512 point FFT.

36. The computer-readable medium of claim 32, wherein the played note matches the current note if a set of note features for the played note match a set of note features for the current note.

37. The computer-readable medium of claim 36, wherein the set of note features includes a fundamental frequency.

38. The computer-readable medium of claim 36, wherein the set of note features includes note-duration.

39. The computer-readable medium of claim 36, wherein the set of note features includes at least one harmonic frequency.

40. The computer-readable medium of claim 39, wherein the set of note features includes at least 5 harmonic frequencies.

41. The computer-readable medium of claim 36, wherein the set of note features includes at least one peak location and at least one peak value.

42. The computer-readable medium of claim 41, wherein the comparing includes weighting the at least one peak location and the at least one peak value.

43. The computer-readable medium of claim 32, wherein performing time alignment includes determining a start point and an end point of a note in the input data.

44. The computer-readable medium of claim 43, wherein a sum of the square of the amplitude is used to determine the start point.

45. The computer-readable medium of claim 43, wherein the sum of the absolute amplitude is used to determine the start point.

46. A computer-readable medium having computer-executable instructions for performing a method for providing a music tutor, the method comprising:

retrieving a set of musical data comprising one or more reference notes;

displaying at least a portion of the set of musical data, said portion including a current note from the one or more reference notes;

receiving a played note;

comparing the played note to the current note; and

displaying an indication of whether the played note matches the current note.

47. The computer-readable medium of claim 46, wherein displaying an indication changes the color of the reference note in accordance with whether the played note matched the reference note.

48. The computer-readable medium of claim 46, further comprising training a system to recognize a set of notes played by a musical instrument.

49. The computer-readable medium of claim 46, further comprising composing the set of reference notes.

50. The computer-readable medium of claim 46, wherein the reference notes are included on a flash card.

51. The computer-readable medium of claim 46, wherein the reference notes are included on a musical segment.

52. The computer-readable medium of claim 46 wherein displaying an indication of whether the played note matches the current note includes highlighting correctly played notes in a first highlight and highlighting incorrectly played notes in a second highlight.

53. The computer-readable medium of claim 52, wherein the first highlight is a first color and the second highlight is a second color.

54. The computer-readable medium of claim 52, wherein the first highlight is a first cross-hatching and the second highlight is a second cross-hatching.

55. A computerized system comprising:

a database having a set of data representing at least one database note;

a sound input interface;

a pattern matching module coupled to the database and the sound input interface and operable to compare a set of data representing at least one played note with the set of data representing the at least one musical note and to identify the played note;

a compose segment module operable to receive the identified played note and to output the played note.

56. The computerized system of claim 55, wherein the compose segment module outputs the played note to a display.

57. The computerized system of claim 55, wherein the compose segment modules is operable to output the played note to a music file.

58. The computerized system of claim 57 wherein the music file is a flash card file.

59. The computerized system of claim 55, further comprising a playback module operable to read a music file and to display a set notes in the music file.

60. The computerized system of claim 59, wherein the set of notes are displayed as a flash card.

61. The computerized system of claim 59, wherein the playback module receives data representing at least one played note from the pattern matching module and compares the at least one played note to the set of notes in the flash card file.

62. The computerized system of claim 61, wherein the playback module identifies whether the at least one played note was played correctly.

63. The computerized system of claim 61, wherein the playback module maintains statistics on a number of correctly played notes.