Musical performance-related information output device, system including musical performance-related information output device, and electronic musical instrument

• US 8,697,975 B2
• Filed: 07/29/2009
• Issued: 04/15/2014
• Est. Priority Date: 07/29/2008
• Status: Active Grant

• Current Assignee: Yamaha Corporation
• Original Assignee: Yamaha Corporation
• Inventors: Iwase, Hiroyuki, Sone, Takuro, Fukui, Mitsuru
• Primary Examiner(s): Warren, David S.
• Application Number:
  US12/935,463
• Publication Number:
  US 20110023691A1
• Time in Patent Office: 1,721 Days
• Field of Search: 84600-602, 846/45, 846/15
• US Class Current: 84/600
• CPC Class Codes: G10H 1/0066 using a MIDI interface
• CPC Class Codes: G10H 1/06 Circuits for establishing the harmonic content of tones , or other arrangements for changing the tone colour
• CPC Class Codes: G10H 1/40 Rhythm
• CPC Class Codes: G10H 2220/301 Fret-like switch array arrangements for guitar necks
• CPC Class Codes: G10H 2220/391 Angle sensing for musical purposes, using data from a gyroscope, gyrometer or other angular velocity or angular movement sensing device angles measured by an accelerometer or gravimeter G10H2220/395; angles calculated from 3D position sensing G10H2220/401; player body joint angle sensing G10H2220/321
• CPC Class Codes: G10H 2240/031 File merging MIDI, i.e. merging or mixing a MIDI-like file or stream with a non-MIDI file or stream, e.g. audio or video
• CPC Class Codes: G10H 2240/205 Synchronous transmission of an analog or digital signal, e.g. according to a specific intrinsic timing, or according to a separate clock
• CPC Class Codes: G10H 2240/215 Spread spectrum, i.e. transmission on a bandwidth considerably larger than the frequency content of the original information
• CPC Class Codes: G10H 2240/225 Frequency division multiplexing
• CPC Class Codes: G10H 3/188 for converting the signal to digital format transmission using a MIDI interface G10H1/0066

Abstract

Provided are a musical performance-related information output device and a musical performance system capable of superimposing musical performance-related information on an audio signal without damaging the general versatility of the audio signal. The musical performance-related information output device includes a musical performance-related information acquiring section that is adapted to acquire musical performance-related information related to a musical performance of a performer, a superimposing section that is adapted to superimpose the musical performance-related information on an analog audio signal such that a modulated component of the musical performance-related information is included in a band higher than the frequency component of the analog audio signal generated in accordance with the musical performance manipulation of the performer, and an output section that outputs the analog audio signal on which the superimposing section superimposes the musical performance-related information.

Claims

1. A musical performance-related information output device comprising:

a musical performance-related information acquiring section configured to acquire musical performance-related information related to a musical performance of a performer;

a superimposing section configured to superimpose the musical performance-related information on an analog audio signal generated in accordance with a musical performance manipulation of the performer so that a modulated component of the musical performance-related information is included in a band higher than a frequency component of the analog audio signal; and

an output section that outputs the analog audio signal on which the superimposing section superimposes the musical performance-related information.

2. The musical performance-related information output device according to claim 1, wherein the musical performance-related information acquiring section acquires musical performance information indicating the musical performance manipulation of the performer as the musical performance-related information.

3. The musical performance-related information output device according to claim 2, wherein the superimposing section includes:

a spread code generating section that generates a spread code having a predetermined period;

a modulating section that phase-modulates the spread code in each period on the basis of the musical performance information; and

a synthesizing section that synthesizes a modulated signal generated on the basis of the phase-modulated spread code and the analog audio signal in the frequency band higher than the frequency component of the analog audio signal, and outputs the resultant signal as a synthesized signal.

4. The musical performance-related information output device according to claim 2, further comprising:

a generating section configured to detect vibration generated in accordance with the musical performance manipulation and generate an analog audio signal based on the detected vibration,

wherein the superimposing section superimposes the musical performance information on the analog audio signal generated by the generating section.

5. The musical performance-related information output device according to claim 1, wherein the musical performance-related information acquiring section acquires tempo information indicating a musical performance tempo as the musical performance-related information.

6. The musical performance-related information output device according to claim 5, wherein the musical performance-related information acquiring section receives a reference tempo signal, which is the reference of the musical performance tempo, from an external source, and extracts the tempo information on the basis of the reference tempo signal.

7. The musical performance-related information output device according to claim 1, further comprising:

a reference clock generating section configured to generate a reference clock at predetermined intervals;

a time difference detecting section configured to detect the time difference between a timing of superimposing sequence data and the reference clock,

wherein the musical performance-related information acquiring section acquires information regarding the reference clock, the sequence data, the timing of superimposing the sequence data, and the time difference, as the musical performance-related information.

8. The musical performance-related information output device according to claim 7, further comprising:

a manipulation input section for inputting a manipulation of the performer; and

a data generating section configured to generate the sequence data in accordance with a manipulation input of the manipulation input section,

wherein the time difference detecting section detects the time difference between the timing of the manipulation input and the timing of superimposing the sequence data by a difference from the reference clock, and information regarding the time difference is represented by the difference.

9. The musical performance-related information output device according to claim 7, wherein the superimposing section includes:

a spread code generating section that generates a spread code having a predetermined period;

a modulating section that phase-modulates the spread code in each period on the basis of superimposing information; and

a synthesizing section that synthesizes a modulated signal generated on the basis of the phase-modulated spread code and the analog audio signal in the frequency band higher than the frequency component of the analog audio signal, and outputs the resultant signal as a synthesized signal.

10. The musical performance-related information output device according to claim 7, wherein tempo information in which a musical performance tempo of the performer is reflected is used as the reference clock.

15. An electronic musical instrument comprising the musical performance-related information output device according to claim 5.

16. The musical performance-related information output device according to claim 1, further comprising:

an input section for receiving an input of a manipulation for controlling an external apparatus; and

a control signal generating section configured to generate a control signal for controlling the external apparatus in accordance with the input of the manipulation received by the input section,

wherein the musical performance-related information acquiring section acquires the control signal as the musical performance-related information.

17. The musical performance-related information output device according to claim 16, wherein the superimposing section includes:

a spread code generating section that generates a spread code having a predetermined period;

a modulating section that phase-modulates the spread code in each period on the basis of the control signal; and

a synthesizing section that synthesizes a modulated signal generated on the basis of the phase-modulated spread code and the analog audio signal in the frequency band higher than the frequency component of the analog audio signal, and outputs the resultant signal as a synthesized signal.

18. The musical performance-related information output device according to claim 16, further comprising:

a musical performance information acquiring section configured to acquire musical performance information indicating content of the musical performance manipulation; and

a storage section for storing the musical performance information and the control signal in association with each other,

wherein the input section acquires the musical performance information as the input of the manipulation, and

wherein the control signal generating section generates the control signal with reference to the storage section by the musical performance information acquired by the input section.

19. The musical performance-related information output device according to claim 18, further comprising:

a pose sensor that detects the pose of the device to generate pose information,

wherein the storage section stores the pose information and the control signal in association with each other,

wherein the input section acquires the pose information as the input of the manipulation, and

wherein the control signal generating section generates the control signal with reference to the storage section by the pose information acquired by the input section.

20. The musical performance-related information output device according to claim 19, further comprising:

a measuring section configured to measure an elapsed time after the musical performance manipulation starts,

wherein the storage section further stores a reception period, for which the input of the manipulation is received, in association with the control signal, and

wherein the control signal generating section generates the control signal when the elapsed time measured by the measuring section is within the reception period.

21. The musical performance-related information output device according to claim 18, further comprising:

a control signal input section for inputting the control signal; and

a registering section configured to register the control signal input from the control signal input section and the manipulation input from the input section in the storage section in association with each other.

22. The musical performance-related information output device according to claim 16, further comprising:

an audio signal generating section configured to generate the analog audio signal based on vibration generated in accordance with the musical performance manipulation,

wherein the superimposing section superimposes the control signal on the analog audio signal generated by the audio signal generating section.

23. A sound processing system comprising:

the musical performance-related information output device according to claim 7; and

a decoding device configured to decode information superimposed on the analog audio signal,

wherein the decoding device includes a synchronizing section configured to receive the analog audio signal in advance, and to synchronize the analog audio signal and the sequence data on the basis of the reference clock decoded from the analog audio signal and information regarding the time difference.

24. The sound processing system according to claim 23, wherein:

the superimposing section of the musical performance-related information output device superimposes pseudo noise on the analog audio signal with a timing based on the reference clock to superimpose the reference clock, and

the decoding device includes a decoding section configured to calculate a correlation between the analog audio signal and the pseudo noise, and decode the reference clock on the basis of a peak-generated timing of the correlation.

25. An electronic musical instrument comprising the musical performance-related information output device according to claim 7.

11. A musical performance system comprising:

a musical performance-related information output device; and

a reproducing device,

wherein the musical performance-related information output device includes;

a musical performance-related information acquiring section configured to acquire musical performance-related information related to a musical performance of a performer;

a superimposing section configured to superimpose the musical performance-related information on an analog audio signal generated in accordance with a musical performance manipulation of the performer so that a modulated component of the musical performance-related information is included in a band higher than a frequency component of the analog audio signal; and

an output section that outputs the analog audio signal on which the superimposing section superimposes the musical performance-related information, and

wherein the reproducing device includes;

an input section for inputting the analog audio signal output from the output section of the musical performance-related information output device;

a decoding section configured to extract the musical performance-related information from the analog audio signal input to the input section and decode the musical performance-related information; and

a synchronous output section that synchronously outputs the analog audio signal and the musical performance information on the basis of the time required for superimposition and decoding of the musical performance information.

12. A sound processing system comprising:

a musical performance-related information output device; and

a decoding device configured to decode the tempo information,

wherein the musical performance-related information output device includes;

a musical performance-related information acquiring section configured to acquire musical performance-related information related to a musical performance of a performer;

a superimposing section configured to superimpose the musical performance-related information on an analog audio signal generated in accordance with a musical performance manipulation of the performer so that a modulated component of the musical performance-related information is included in a band higher than a frequency component of the analog audio signal; and

an output section that outputs the analog audio signal on which the superimposing section superimposes the musical performance-related information,

wherein the musical performance-related information acquiring section acquires tempo information indicating a musical performance tempo as the musical performance-related information,

wherein the superimposing section of the musical performance-related information output device superimposes pseudo noise on the analog audio signal with a timing based on the musical performance tempo to superimpose the tempo information, and

wherein the decoding device includes;

an input section for inputting the analog audio signal; and

a decoding section configured to calculate a correlation between the analog audio signal input to the input section and the pseudo noise, and to decode the tempo information on the basis of a peak-generated timing of the correlation.

13. The sound processing system according to claim 12, wherein:

the musical performance-related information acquiring section of the musical performance-related information output device extracts multiple kinds of tempo information in accordance with the respective timings of the musical performance tempo,

the superimposing section superimposes multiple kinds of pseudo noise to superimpose the multiple kinds of tempo information, and

the decoding section of the decoding device calculates correlations between the analog audio signal input to the input section and the respective multiple kinds of pseudo noise, and decodes the multiple kinds of tempo information on the basis of the peak-generated timings of the respective correlations.

14. The sound processing system according to claim 12 wherein the superimposing section of the musical performance-related information output device includes:

a spread code generating section that generates the pseudo noise as a spread code having a predetermined period;

a modulating section that phase-modulates the spread code in each period on the basis of the tempo information; and

a synthesizing section that synthesizes a modulated signal generated on the basis of the phase-modulated spread code and the analog audio signal in the frequency band higher than the frequency component of the analog audio signal, and outputs the resultant signal as a synthesized signal.

26. A method of outputting musical performance-related information using a musical performance-related information output device, the method comprising:

a musical performance-related information acquiring step of acquiring musical performance-related information related to a musical performance of a performer;

a superimposing step of superimposing the musical performance-related information on an analog audio signal generated in accordance with a musical performance manipulation of the performer so that a modulated component of the musical performance-related information is included in a band higher than a frequency component of the analog audio signal; and

an output step of outputting the analog audio signal on which the superimposing section superimposes the musical performance-related information.

27. A musical performance-related information output device comprising:

a musical performance-related information acquiring section configured to acquire musical performance-related information related to a musical performance of a performer;

a superimposing section configured to superimpose the musical performance-related information on an audio signal generated in accordance with a musical performance manipulation of the performer so that a modulated component of the musical performance-related information is included in a band; and

an output section configured to output the audio signal on which the superimposing section superimposes the musical performance-related information,

wherein the superimposing section includes;

a spread code generating section configured to generate a spread code having a predetermined period;

a modulating section configured to phase-modulate the spread code in each period based on the musical performance information; and

an adding section configured to mix the phase-modulated spread code into the audio signal.

28. The musical performance-related information output device according to claim 27, wherein the band which includes a modulated component of the musical performance-related information is higher than the frequency component of the audio signal.

29. The musical performance-related information output device according to claim 27, wherein the musical performance-related information indicates one of musical performance information, tempo information, control information, a reference clock, sequence data, a timing of superimposing the sequence data on the audio signal, or information indicative of a time difference between the timing of superimposing and the reference clock.

30. An audio reproducing device comprising:

an acquiring section configured to acquire an audio signal on which musical performance information is superimposed;

a decoding section configured to extract and decode the musical performance information from the audio signal; and

a synchronous output section configured to output the audio signal synchronously with the musical performance information on the basis of a time required for superimposition and decoding of the musical performance information.

31. The audio reproducing device according to claim 30, wherein the musical performance information is previously superimposed on the audio signal by phase-modulating a spread code in each given period.

32. The audio reproducing device according to claim 30, wherein a frequency component of the musical performance information is higher than a frequency component of the audio signal.

33. An audio decoding device comprising:

an acquiring section configured to acquire an audio signal on which musical performance-related information is superimposed;

a decoding section configured to calculate a correlation between the audio signal and a spread code, and to decode as data code the musical performance-related information from the audio signal; and

an outputting section configured to output the decoded musical performance-related information.

34. The audio decoding device according to claim 33, wherein the musical performance-related information is previously superimposed on the audio signal by phase-modulating the spread code in each given period.

35. The audio decoding device according to claim 33, wherein a frequency component of the musical performance-related information is higher than a frequency component of the audio signal.

36. The audio decoding device according to claim 33, wherein the musical performance-related information indicates one of musical performance information, tempo information, control information, a reference clock, sequence data, a timing of superimposing the sequence data on the audio signal, or information indicative of a time difference between the timing of superimposing and the reference clock.

37. The audio decoding device according to claim 33, wherein:

the musical performance-related information indicates a reference clock, sequence data and information indicative of a time difference between the reference clock and a timing of superimposing the sequence data on the audio signal, and

the decoding device further includes a synchronizing section configured to synchronize the audio signal with the sequence data on the basis of the reference clock and the information indicative of the time difference which are decoded from the audio signal.

38. A method of controlling a musical performance-related information output device, the method comprising:

an acquiring step of acquiring musical performance-related information related to a musical performance of a performer;

a superimposing step of superimposing the musical performance-related information on an audio signal generated in accordance with a musical performance manipulation of the performer so that a modulated component of the musical performance-related information is included in a band; and

an outputting step of outputting the audio signal on which the musical performance-related information is superimposed,

wherein the superimposing step comprises;

a generating step of generating a spread code having a predetermined period;

a phase-modulating step of phase-modulating the spread code in each period based on the musical performance information; and

a mixing step of mixing the phase-modulated spread code into the audio signal.

39. A non-transitory computer-readable storage medium storing a program executable by a computer to execute a method of controlling a musical performance-related information output device, the method comprising:

an acquiring step of acquiring musical performance-related information related to a musical performance of a performer;

a superimposing step of superimposing the musical performance-related information on an audio signal generated in accordance with a musical performance manipulation of the performer so that a modulated component of the musical performance-related information is included in a band; and

an outputting step of outputting the audio signal on which the musical performance-related information is superimposed,

wherein the superimposing step comprises;

a generating step of generating a spread code having a predetermined period;

a phase-modulating step of phase-modulating the spread code in each period based on the musical performance information; and

a mixing step of mixing the phase-modulated spread code into the audio signal.