Apparatus for analyzing musical performance, performance analysis method, automatic playback method, and automatic player system

• US 10,580,393 B2
• Filed: 01/18/2019
• Issued: 03/03/2020
• Est. Priority Date: 07/22/2016
• Status: Active Grant

• Current Assignee: Yamaha Corporation
• Original Assignee: Yamaha Corporation
• Inventors: Maezawa, Akira
• Primary Examiner(s): Donels, Jeffrey
• Application Number:
  US16/252,086
• Publication Number:
  US 20190156806A1
• Time in Patent Office: 410 Days
• Field of Search: 84600
• CPC Class Codes: G10G 3/04 using electrical means
• CPC Class Codes: G10H 1/00 Details of electrophonic musical instruments
• CPC Class Codes: G10H 1/0008 Associated control or indicating means
• CPC Class Codes: G10H 1/361 Recording/reproducing of accompaniment for use with an external source, e.g. karaoke systems
• CPC Class Codes: G10H 1/40 Rhythm
• CPC Class Codes: G10H 2210/091 for performance evaluation, i.e. judging, grading or scoring the musical qualities or faithfulness of a performance, e.g. with respect to pitch, tempo or other timings of a reference performance
• CPC Class Codes: G10H 2220/201 for movement interpretation, i.e. capturing and recognizing a gesture or a specific kind of movement, e.g. to control a musical instrument
• CPC Class Codes: G10H 2240/325 Synchronizing two or more audio tracks or files according to musical features or musical timings

Abstract

An apparatus for analyzing musical performance includes a controller. The controller is configured to detect a cue gesture of a performer who plays a piece of music. The controller is also configured to calculate a distribution of likelihood of observation and estimate the playback position depending on the distribution of the likelihood of observation. The calculating of the distribution of the likelihood of observation includes decreasing the likelihood of observation during a period prior to a reference point specified on a time axis for the piece of music in a case where the cue gesture is detected.

Claims

1. An apparatus for analyzing performance of a piece of music, the apparatus comprising:

a controller that is configured to

detect a cue gesture of a performer who plays the piece of music;

calculate a distribution of likelihood of observation by analyzing an audio signal representative of a sound of the piece of music being played, wherein the likelihood of observation is an index showing a correspondence probability of a time point within the piece of music to a playback position; and

estimate the playback position depending on the distribution of the likelihood of observation, wherein

calculating the distribution of the likelihood of observation includes decreasing the likelihood of observation during a period prior to a reference point specified on a time axis for the piece of music in a case where the cue gesture is detected.

2. A computer-implemented performance analysis method, comprising:

detecting a cue gesture of a performer who plays a piece of music;

calculating a distribution of likelihood of observation by analyzing an audio signal representative of a sound of the piece of music being played, wherein the likelihood of observation is an index showing a correspondence probability of a time point within the piece of music to a playback position; and

estimating the playback position depending on the distribution of the likelihood of observation,

wherein calculating the distribution of the likelihood of observation includes decreasing the likelihood of observation during a period prior to a reference point specified on a time axis for the piece of music in a case where the cue gesture is detected.

3. The performance analysis method according to claim 2, wherein calculating the distribution of the likelihood of observation includes:

calculating from the audio signal a first likelihood value which is an index showing a correspondence probability of a time point within the piece of music to a playback position;

calculating a second likelihood value which is set to a first value in a state where no cue gesture is detected, or to a second value that is lower than the first value in a case where the cue gesture is detected; and

calculating the likelihood of observation by multiplying together the first likelihood value and the second likelihood value.

4. The performance analysis method according to claim 3, wherein the first value is 1, and the second value is 0.

5. A computer-implemented automatic playback method, comprising:

detecting a cue gesture of a performer who plays a piece of music,

estimating playback positions in the piece of music by analyzing an audio signal representative of a sound of the piece of music being played; and

causing an automatic player apparatus to execute automatic playback of the piece of music synchronous with the detected cue gesture and with progression of the playback positions,

wherein estimating each playback position includes;

calculating a distribution of likelihood of observation by analyzing the audio signal, wherein the likelihood of observation is an index showing a correspondence probability of a time point within the piece of music to a playback position; and

estimating the playback position depending on the distribution of the likelihood of observation, and

wherein calculating the distribution of the likelihood of observation includes decreasing the likelihood of observation during a period prior to a reference point specified on a time axis for the piece of music in a case where the cue gesture is detected.

6. An automatic playback method according to claim 5,

wherein calculating the distribution of the likelihood of observation includes:

calculating from the audio signal a first likelihood value which is an index showing a correspondence probability of a time point within the piece of music to a playback position;

calculating a second likelihood value which is set to a first value in a state where no cue gesture is detected, or to a second value that is below the first value in a case where the cue gesture is detected; and

calculating the likelihood of observation by multiplying together the first likelihood value and the second likelihood value.

7. The automatic playback method according to claim 5, wherein

the automatic player apparatus is caused to execute automatic playback in accordance with music data representative of content of playback of the piece of music, and

the reference point is specified by the music data.

8. The automatic playback method according to claim 5,

wherein a display device is caused to display an image representative of progress of the automatic playback.

9. An automatic player system comprising:

at least one processor configured to execute stored instructions to;

detect a cue gesture of a performer who plays a piece of music;

estimate playback positions in the piece of music by analyzing an audio signal representative of a sound of the piece of music being played; and

cause an automatic player apparatus to execute automatic playback of the piece of music synchronous with the detected cue gesture and with progression of the estimated playback positions,

wherein in estimating the playback positions, the at least one processor is configured to;

calculate a distribution of likelihood of observation by analyzing the audio signal, wherein the likelihood of observation is an index showing a correspondence probability of a time point within the piece of music to a playback position; and

estimate the playback position depending on the distribution of the likelihood of observation, and

wherein in calculating the distribution of likelihood of observation, the at least one processor is configured to decrease the likelihood of observation during a period prior to a reference point specified on a time axis for the piece of music in a case where the cue gesture is detected.