Method and apparatus of synthesizing plucked string instruments using recurrent neural networks
First Claim
Patent Images
1. A method of synthesizing sounds produced by a plucked string comprising the steps of:
- generating an input waveform corresponding to an initial plucking of the string, wherein waveform values are assigned to a plurality of predetermined points on the string and interpolating further waveform values based on the assigned values;
initializing nodes of a neural network with values based on the input waveform;
iteratively changing the node values based on weights associated with the nodes;
outputting a waveform based on selected ones of the node values at a plurality of the iterations; and
generating sound based on a sequence of the output waveforms, the generated sound simulating the sound made by the plucked string.
2 Assignments
0 Petitions
Accused Products
Abstract
A “virtual string” is generated for synthesizing sound produced by plucked-string instruments using recurrent neural networks. The disclosed recurrent neural network, called a Scattering Recurrent Network (SRN), is based on the physics of waves traveling in the string. Vibration measured from a plucked string is used as the training data for the SRN. The trained SRN is a virtual model capable of generating tones similar to the tones generated by the physical string. As with a real string, the “virtual string” corresponding to the SRN responds differently to different types of string “plucking” motions.
-
Citations
9 Claims
-
1. A method of synthesizing sounds produced by a plucked string comprising the steps of:
-
generating an input waveform corresponding to an initial plucking of the string, wherein waveform values are assigned to a plurality of predetermined points on the string and interpolating further waveform values based on the assigned values;
initializing nodes of a neural network with values based on the input waveform;
iteratively changing the node values based on weights associated with the nodes;
outputting a waveform based on selected ones of the node values at a plurality of the iterations; and
generating sound based on a sequence of the output waveforms, the generated sound simulating the sound made by the plucked string. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
calculating values of arrival nodes in the neural network based on values of departure nodes in the neural network at a previous iteration;
calculating values of displacement nodes in the neural network based on the calculated values of the arrival nodes; and
calculating values of the departure nodes in the neural network based on the calculated values of the displacement nodes and arrival nodes.
-
-
6. The method of claim 5, wherein the sub-step of calculating the values of the arrival nodes in the neural network further includes the step of evaluating the equation:
-
7. The method of claim 5, wherein the sub-step of calculating the values of the displacement nodes in the neural network further includes the step of evaluating the equations:
-
netiy(t+1)=ri,i−
1·
φ
i,i−
1(t+1)+ri,i+1·
φ
i,i+1(t+1).
-
-
8. The method of claim 5, wherein the sub-step of calculating the values of the departure nodes in the neural network further includes the step of evaluating the equation:
-
9. A computer readable medium containing instructions for execution on a computer that synthesizes sounds produced by a plucked string, the instructions, when executed, causing the computer to perform the steps of:
-
generating an input waveform corresponding to an initial plucking of the string, wherein waveform values are assigned to a plurality of predetermined points on the string and interpolating further waveform values based on the assigned values;
initializing nodes of a neural network with values based on the input waveform;
iteratively changing the node values based on weights associated with the nodes;
outputting a waveform based on selected ones of the node values at a plurality of the iterations; and
generating sound based on a sequence of the output waveforms, the generated sound simulating the sound made by the plucked string.
-
Specification