Music displaying apparatus and computer-readable storage medium storing music displaying program

• US 7,829,777 B2
• Filed: 02/25/2008
• Issued: 11/09/2010
• Est. Priority Date: 12/28/2007
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: Fujita, Takahiko, Kyuma, Koichi, Ozaki, Yuichi
• Primary Examiner(s): Fletcher; Marlon T
• Application Number:
  US12/071,708
• Publication Number:
  US 20090165633A1
• Time in Patent Office: 988 Days
• Field of Search: None
• US Class Current: 84/477.00R
• CPC Class Codes: G10H 1/368 displaying animated or moving pictures synchronized with the music or audio part
• CPC Class Codes: G10H 2210/066 for pitch analysis as part of wider processing for musical purposes, e.g. transcription, musical performance evaluation; Pitch recognition, e.g. in polyphonic sounds; Estimation or use of missing fundamental
• CPC Class Codes: G10H 2210/071 for rhythm pattern analysis or rhythm style recognition rhythm pattern per se G10H2210/341
• CPC Class Codes: G10H 2210/091 for performance evaluation, i.e. judging, grading or scoring the musical qualities or faithfulness of a performance, e.g. with respect to pitch, tempo or other timings of a reference performance
• CPC Class Codes: G10H 2220/135 Musical aspects of games or videogames; Musical instrument-shaped game input interfaces game background music G10H2210/026; musical game scoring or performance evaluation G10H2210/091
• CPC Class Codes: G10H 2220/151 Musical difficulty level setting or selection
• CPC Class Codes: G10H 2240/081 Genre classification, i.e. descriptive metadata for classification or selection of musical pieces according to style analysis genre G10H2210/036
• CPC Class Codes: G10H 2240/105 User profile, i.e. data about the user, e.g. for user settings or user preferences
• CPC Class Codes: G10H 2240/135 Library retrieval index, i.e. using an indexing scheme to efficiently retrieve a music piece

Abstract

A music displaying apparatus stores in advance music piece related information concerning a music piece, and a plurality of comparison parameters which is associated with the music piece related information. The music displaying apparatus obtains voice data concerning singing of a user, analyzes the voice data to calculate a plurality of singing characteristic parameters which indicate a characteristic of the singing of the user. Next, the music displaying apparatus compares the plurality of singing characteristic parameters with the plurality of comparison parameters to calculate a similarity between the plurality of singing characteristic parameters and the plurality of comparison parameters. Then, the music displaying apparatus selects at least one piece of the music piece related information which is associated with a comparison parameter which has a high similarity with the singing characteristic parameter, and shows certain information based on the music piece related information.

Claims

1. A music displaying apparatus comprising:

a voice input device to obtain voice data concerning singing of a user;

singing characteristic analysis programmed logic circuitry for analyzing the voice data to calculate a plurality of singing characteristic parameters which indicate a characteristic of the singing of the user;

music piece related information storage medium for storing music piece related information concerning a music piece;

comparison parameter storage medium for storing a plurality of comparison parameters, which are to be compared with the plurality of singing characteristic parameters, so as to be associated with the music piece related information;

comparison programmed logic circuitry for comparing the plurality of singing characteristic parameters with the plurality of comparison parameters to calculate a similarity between the plurality of singing characteristic parameters and the plurality of comparison parameters;

selection programmed logic circuitry for selecting at least one piece of the music piece related information which is associated with a comparison parameter which has a high similarity with the singing characteristic parameter;

a display to display information based on the music piece related information selected by the selection programmed logic circuitry, wherein

the music piece related information includes music piece data for reproducing at least the music piece,

the comparison parameter includes a parameter which indicates a musical characteristic of the music piece so as to be associated with the music piece data,

the selection programmed logic circuitry selects at least one piece of the music piece data which is associated with the comparison parameter which has the high similarity with the singing characteristic parameter,

the display to display information of the music piece based on the music piece data selected by the selection programmed logic circuitry,

the music piece related information includes genre data which indicates a music genre, and

the comparison parameter includes a parameter which indicates a musical characteristic of the music genre,

music piece genre similarity data storage medium for storing music piece genre similarity data which indicates a similarity between the music piece and the music genre; and

voice genre similarity calculation programmed logic circuitry for calculating a similarity between the singing characteristic parameter and the music genre, wherein

the selection programmed logic circuitry selects the music piece data based on the similarity calculated by the voice genre similarity calculation programmed logic circuitry and the music piece genre similarity data stored by the music piece genre similarity data storage medium.

2. The music displaying apparatus according to claim 1, wherein

the music piece data includes musical score data for indicating musical instruments used for playing the music piece, a tempo of the music piece, and a key of the music piece, and

the music displaying apparatus further comprises music piece genre similarity calculation programmed logic circuitry for calculating a similarity between the music piece and the music genre based on the musical instruments, the tempo and the key which are included in the musical score data.

3. The music displaying apparatus according to claim 1, wherein each of the plurality of singing characteristic parameters and the plurality of comparison parameters includes a value obtained by evaluating one of accuracy of pitch concerning the singing of the user, variation in pitch, a periodical input of voice, and a singing range.

4. The music displaying apparatus according to claim 1, wherein

the music piece data includes musical score data which indicates musical instruments used for the music piece, a tempo of the music piece, a key of the music piece, a plurality of musical notes which constitute the music piece,

the singing characteristic analysis programmed logic circuitry includes voice volume/pitch data calculation programmed logic circuitry for calculating, from the voice data, voice volume value data which indicates a voice volume value, and pitch data which indicates a pitch, and

the singing characteristic analysis programmed logic circuitry compares at least one of the voice volume value data and the pitch data with the musical score data to calculate the singing characteristic parameter.

5. The music displaying apparatus according to claim 4, wherein the singing characteristic analysis programmed logic circuitry calculates the singing characteristic parameter based on an output value of a frequency component for a predetermined period from the voice volume value data.

6. The music displaying apparatus according to claim 4, wherein the singing characteristic analysis programmed logic circuitry calculates the singing characteristic parameter based on a difference between a start timing of each musical note of a melody part of a musical score indicated by the musical score data and an input timing of voice based on the voice volume value data.

7. The music displaying apparatus according to claim 4, wherein the singing characteristic analysis programmed logic circuitry calculates the singing characteristic parameter based on a difference between a pitch of a musical note of a musical score indicated by the musical score data and a pitch based on the pitch data.

8. The music displaying apparatus according to claim 4, wherein the singing characteristic analysis programmed logic circuitry calculates the singing characteristic parameter based on an amount of change in pitch for each time unit in the pitch data.

9. The music displaying apparatus according to claim 4, wherein the singing characteristic analysis programmed logic circuitry calculates, from the voice volume value data and the pitch data, the singing characteristic parameter based on a pitch having a maximum voice volume value among voices, a pitch of each of which is maintained for a predetermined time period or more.

10. The music displaying apparatus according to claim 4, wherein the singing characteristic analysis programmed logic circuitry calculates a quantity of high-frequency components included in the voice of the user from the voice data, and calculates the singing characteristic parameter based on a calculated result.

12. The music displaying apparatus according to claim 1, wherein

the music piece data includes musical score data for indicating musical instruments used for playing the music piece, a tempo of the music piece, and a key of the music piece,

the music displaying apparatus further comprises music piece parameter calculation programmed logic circuitry for calculating, from the musical score data, the comparison parameter for each music piece, and

the comparison parameter storage medium stores the comparison parameter calculated by the music piece parameter calculation programmed logic circuitry.

13. The music displaying apparatus according to claim 12, wherein the music piece parameter calculation programmed logic circuitry calculates, from the musical score data, the comparison parameter based on a difference in pitch between two adjacent musical notes, a position of a musical note within a beat, and a total time of musical notes having lengths equal to or larger than a predetermined threshold value.

11. A music displaying apparatus comprising:

a voice input device to obtain voice data concerning singing of a user;

singing characteristic analysis programmed logic circuitry for analyzing the voice data to calculate a plurality of singing characteristic parameters which indicate a characteristic of the singing of the user;

music piece related information storage medium for storing music piece related information concerning a music piece;

comparison parameter storage medium for storing a plurality of comparison parameters, which are to be compared with the plurality of singing characteristic parameters, so as to be associated with the music piece related information;

comparison programmed logic circuitry for comparing the plurality of singing characteristic parameters with the plurality of comparison parameters to calculate a similarity between the plurality of singing characteristic parameters and the plurality of comparison parameters;

selection programmed logic circuitry for selecting at least one piece of the music piece related information which is associated with a comparison parameter which has a high similarity with the singing characteristic parameter; and

a display to display information based on the music piece related information selected by the selection programmed logic circuitry, wherein

the music piece related information includes genre data which indicates at least a music genre,

the comparison parameter includes a parameter which indicates a musical characteristic of the music genre so as to be associated with the music genre,

the selection programmed logic circuitry selects the music genre which is associated with the comparison parameter which has the high similarity with the singing characteristic parameter, and

the display to display a name of the music genre as information based on the music piece related information.

14. A non-transitory computer-readable storage medium storing a music displaying program which causes a computer of a music displaying apparatus, which shows a music piece to a user, to perform a method comprising:

obtaining voice data concerning singing of the user;

analyzing the voice data to calculate a plurality of singing characteristic parameters which indicate a characteristic of the singing of the user;

storing music piece related information concerning a music piece;

storing a plurality of comparison parameters, which are operable to be compared with the plurality of singing characteristic parameters, so as to be associated with the music piece related information;

comparing the plurality of singing characteristic parameters with the plurality of comparison parameters to calculate a similarity between the plurality of singing characteristic parameters and the plurality of comparison parameters;

selecting selection results, the selection results including at least one piece of the music piece related information which is associated with a comparison parameter of the plurality of comparison parameters which has a high similarity with a singing characteristic parameter of the plurality of singing characteristic parameters; and

displaying resultant information based on the selection results, wherein

the music piece related information includes music piece data for reproducing at least the music piece,

the comparison parameter includes a parameter which indicates a musical characteristic of the music piece so as to be associated with the music piece data, and

the selection results including at least one piece of the music piece data which is associated with the comparison parameter which has a high similarity with the singing characteristic parameter,

the music piece related information includes genre data which indicates a music genre, and

the comparison parameter includes a musical characteristic parameter of the music genre,

storing music piece genre similarity data which indicates a similarity between the music piece and the music genre; and

calculating a similarity between the singing characteristic parameter and the music genre, wherein

the at least one piece of music piece data selection based on the similarity calculated between the signing characteristic parameter and the music genre and the music piece genre similarity data.

15. The computer-readable storage medium according to claim 14, wherein

the music piece data includes musical score data for indicating musical instruments used for playing the music piece, a tempo of the music piece, and a key of the music piece, and

the computer-readable storage medium stores the music displaying program which causes the computer of the music displaying apparatus to perform the method further comprising:

calculating a similarity between the music piece and the music genre based on the musical instruments, the tempo, and the key which are included in the musical score data.

16. The computer-readable storage medium according to claim 14, wherein each of the plurality of singing characteristic parameters and the plurality of comparison parameters includes a value of accuracy of pitch concerning the singing of the user, a variation in pitch, a periodical input of voice, and a singing range.

17. The computer-readable storage medium according to claim 14, wherein

the music piece data includes musical score data which indicates musical instruments used for the music piece, a tempo of the music piece, a key of the music piece, a plurality of musical notes which constitute the music piece,

the analyzing the voice data to calculate a plurality of singing characteristic parameters which indicate a characteristic of the singing of the user further comprises:

calculating, from the voice data, voice volume value data which indicates a voice volume value, and pitch data which indicates a pitch, and

comparing at least one of the voice volume value data and the pitch data with the musical score data to calculate the singing characteristic parameter.

18. The computer-readable storage medium according to claim 17, wherein the analyzing the voice data to calculate a plurality of singing characteristic parameters which indicate a characteristic of the singing of the user further comprises:

calculating the singing characteristic parameter based on an output value of a frequency component for a predetermined period from the voice volume value data.

19. The computer-readable storage medium according to claim 17, wherein the analyzing the voice data to calculate a plurality of singing characteristic parameters which indicate a characteristic of the singing of the user further comprises:

calculating the singing characteristic parameter based on a difference between a start timing of each musical note of a melody part of a musical score indicated by the musical score data and an input timing of voice based on the voice volume value data.

20. The computer-readable storage medium according to claim 17, wherein the analyzing the voice data to calculate a plurality of singing characteristic parameters which indicate a characteristic of the singing of the user further comprises:

calculating the singing characteristic parameter based on a difference between a pitch of a musical note of a musical score indicated by the musical score data and a pitch based on the pitch data.

21. The computer-readable storage medium according to claim 17, wherein the analyzing the voice data to calculate a plurality of singing characteristic parameters which indicate a characteristic of the singing of the user further comprises:

calculating the singing characteristic parameter based on an amount of change in pitch for each time unit in the pitch data.

22. The computer-readable storage medium according to claim 17, wherein the analyzing the voice data to calculate a plurality of singing characteristic parameters which indicate a characteristic of the singing of the user further comprises:

calculating from the voice volume value data and the pitch data, the singing characteristic parameter based on a pitch having a maximum voice volume value among voices, a pitch of each of which is maintained for a predetermined time period or more.

23. The computer-readable storage medium according to claim 17, wherein the analyzing the voice data to calculate a plurality of singing characteristic parameters which indicate a characteristic of the singing of the user further comprises:

calculating a quantity of high-frequency components included in the voice of the user from the voice data, and calculates the singing characteristic parameter based on a calculated result.

24. The computer-readable storage medium according to claim 14, wherein

the music piece data includes musical score data for indicating musical instruments used for playing the music piece, a tempo of the music piece, and a key of the music piece, and

the music displaying program further causes the computer of the music displaying apparatus to perform a method further comprising calculating, from the musical score data, the comparison parameter for each music piece.

25. The computer-readable storage medium according to claim 24, wherein calculating, from the musical score data, the comparison parameter for each music piece further comprises calculating, from the musical score data, the comparison parameter based on a difference in pitch between two adjacent musical notes, a position of a musical note within a beat, and a total time of musical notes having lengths equal to or larger than a predetermined threshold value.

26. A non-transitory computer-readable storage medium storing a music displaying program which causes a computer of a music displaying apparatus, which shows a music piece to a user, to perform a method comprising:

obtaining voice data concerning singing of the user;

analyzing the voice data to calculate a plurality of singing characteristic parameters which indicate a characteristic of the singing of the user;

storing music piece related information concerning a music piece;

storing a plurality of comparison parameters, which are operable to be compared with the plurality of singing characteristic parameters, so as to be associated with the music piece related information;

comparing the plurality of singing characteristic parameters with the plurality of comparison parameters to calculate a similarity between the plurality of singing characteristic parameters and the plurality of comparison parameters;

selecting selection results, the selection results including at least one piece of the music piece related information which is associated with a comparison parameter of the plurality of comparison parameters which has a high similarity with a singing characteristic parameter of the plurality of singing characteristic parameters; and

displaying resultant information based on the selection results, wherein

the music piece related information includes genre data which indicates at least a music genre,

the comparison parameter includes a parameter which indicates a musical characteristic of the music genre so as to be associated with the music genre, and

the selection results includes the music genre which is associated with the comparison parameter which has a high similarity with the singing characteristic parameter.

27. A method for correlating a music piece to a singing user of a computer music system, the method comprising:

obtaining voice data from the singing user;

analyzing the voice data to calculate a plurality of singing characteristic parameters which correspond to singing characteristics of the singing user;

storing music piece related information concerning a plurality of music pieces and a plurality of comparison parameters associated with each one of the plurality of music pieces;

comparing the plurality of singing characteristic parameters with the plurality of comparison parameters to calculate a similarity between the plurality of singing characteristic parameters and the plurality of comparison parameters;

selecting at least one music piece from the plurality of music pieces when the similarity between the plurality of comparison parameters to the plurality of singing characteristic parameters is high;

displaying results based on the at least one music piece selected, wherein

the music piece related information includes music piece data for reproducing at least the music piece,

the comparison parameter includes a parameter which indicates a musical characteristic of the music piece so as to be associated with the music piece data,

at least one piece of the music piece data which is associated with the comparison parameter which has the high similarity with the singing characteristic parameter is selected,

the information of the music piece based on the selected music piece data is displayed,

the music piece related information includes genre data which indicates a music genre, and

the comparison parameter includes a parameter which indicates a musical characteristic of the music genre,

storing music piece genre similarity data which indicates a similarity between the music piece and the music genre; and

calculating a similarity between the singing characteristic parameter and the music genre, wherein

the music piece data is selected based on the calculated similarity and the stored music piece genre similarity data.

28. A computer system operable to display music information that correlates to a singing user, the system comprising:

a voice input device to obtain voice data from the singing user;

computer writeable storage medium configured to store;

a representation of a plurality of music pieces;

a plurality of comparison parameters that are associated with each one of the plurality of music pieces;

music piece genre similarity data which indicates a similarity between the music piece and a music genre; and

a processor configured to;

analyze the voice data of the singing user and calculate a plurality of singing characteristic parameters that correlate to the singing characteristics of the singing user;

determine a degree of similarity between each one of the plurality of singing characteristic parameters to the plurality of comparison parameters of the plurality of music pieces;

select results, the results including at least one music piece from the plurality of music pieces where the degree of similarity is determined to be substantially high;

generate a display of the results for the singing user, wherein

the representation of the plurality of music pieces includes music piece data for reproducing at least the music piece,

the comparison parameters includes a parameter which indicates a musical characteristic of the music piece so as to be associated with the music piece data,

at least one piece of the music piece data which is associated with the comparison parameter which has the high similarity with the singing characteristic parameter is selected,

information of the music piece based on the selected music piece data is displayed,

the music piece related information includes genre data which indicates the music genre, and

the comparison parameter includes a parameter which indicates a musical characteristic of the music genre; and

calculate a similarity between the singing characteristic parameter and the music genre, wherein the music piece data is selected based on the calculated similarity and the stored music piece genre similarity data.

29. A method for correlating a music piece to a singing user of a computer music system, the method comprising:

obtaining voice data from the singing user;

analyzing the voice data to calculate a plurality of singing characteristic parameters which correspond to singing characteristics of the singing user;

storing music piece related information concerning a plurality of music pieces and a plurality of comparison parameters associated with each one of the plurality of music pieces;

comparing the plurality of singing characteristic parameters with the plurality of comparison parameters to calculate a similarity between the plurality of singing characteristic parameters and the plurality of comparison parameters;

selecting at least one music piece from the plurality of music pieces when the similarity between the plurality of comparison parameters to the plurality of singing characteristic parameters is high;

displaying results based on the at least one music piece selected, wherein

the music piece related information includes genre data which indicates at least a music genre,

the comparison parameter includes a parameter which indicates a musical characteristic of the music genre so as to be associated with the music genre,

the music genre which is associated with the comparison parameter which has the high similarity with the singing characteristic parameter is selected, and

a name of the music genre as information based on the music piece related information is displayed.

30. A computer system operable to display music information that correlates to a singing user, the system comprising:

a voice input device to obtain voice data from the singing user;

computer writeable storage medium configured to store;

music piece related information concerning a plurality of music pieces;

a plurality of comparison parameters that are associated with each one of the plurality of music pieces;

a processor configured to;

analyze the voice data of the singing user and calculate a plurality of singing characteristic parameters that correlate to the singing characteristics of the singing user;

determine a degree of similarity between each one of the plurality of singing characteristic parameters to the plurality of comparison parameters of the plurality of music pieces;

select results, the results including at least one music piece from the plurality of music pieces where the degree of similarity is determined to be substantially high; and

generate a display of the results for the singing user, wherein

the music piece related information includes genre data which indicates at least a music genre,

the comparison parameters include a parameter which indicates a musical characteristic of the music genre so as to be associated with the music genre,

the music genre which is associated with the comparison parameter which has the high similarity with the singing characteristic parameter is selected, and

a name of the music genre as information based on the music piece related information is displayed.