Control structure for sound synthesis

• US 5,880,392 A
• Filed: 12/02/1996
• Issued: 03/09/1999
• Est. Priority Date: 10/23/1995
• Status: Expired due to Fees

• Current Assignee: Regents of the University of California (University of California)
• Original Assignee: Regents of the University of California (University of California)
• Inventors: Lee, Michael, Wessel, David
• Primary Examiner(s): Shoop, Jr., William M.
• Assistant Examiner(s): FLETCHER, MARLON T
• Application Number:
  US08/756,935
• Time in Patent Office: 827 Days
• Field of Search: 84/622-625, 84/659-661, 84/626-633, 84/662-665
• US Class Current: 84/659
• CPC Class Codes: G10H 2230/221 Spint saxophone, i.e. mimicking conical bore musical instruments with single reed mouthpiece, e.g. saxophones, electrophonic emulation or interfacing aspects therefor
• CPC Class Codes: G10H 2230/351 Spint bell, i.e. mimicking bells, e.g. cow-bells
• CPC Class Codes: G10H 2250/151 Fuzzy logic
• CPC Class Codes: G10H 2250/235 Fourier transform; Discrete Fourier Transform [DFT]; Fast Fourier Transform [FFT]
• CPC Class Codes: G10H 2250/311 Neural networks for electrophonic musical instruments or musical processing, e.g. for musical recognition or control, automatic composition or improvisation
• CPC Class Codes: G10H 2250/625 Interwave interpolation, i.e. interpolating between two different waveforms, e.g. timbre or pitch or giving one waveform the shape of another while preserving its frequency or vice versa
• CPC Class Codes: G10H 7/10 using coefficients or parameters stored in a memory, e.g. Fourier coefficients G10H7/12 takes precedence

Abstract

An improved control structure for music synthesis is provided in which: 1) the sound representation provided to the adaptive function mapper allows for a greatly increased degree of control over the sound produced; and 2) training of the adaptive function mapper is performed using an error measure, or error norm, that greatly facilitates learning while ensuring perceptual identity of the produced sound with the training example. In accordance with one embodiment of the invention, sound data is produced by applying to an adaptive function mapper control parameters including: at least one parameter selected from the set of time and timbre space coordinates; and at least one parameter selected from the set of pitch, Δpitch, articulation and dynamic. Using an adaptive function mapper, mapping is performed from the control parameters to synthesis parameters to be applied to a sound synthesizer. In accordance with another embodiment of the invention, an adaptive function mapper is trained to produce, in accordance with information stored in a mapping store, synthesis parameters to be applied to a sound synthesizer, by steps including: analyzing sounds to produce sound parameters describing the sounds; further analyzing the sound parameters to produce control parameters; applying the control parameters to the adaptive function mapper, the adaptive function mapper in response producing trial synthesis parameters comparable to the sound parameters; deriving from the sound parameters and the trial synthesis parameters an error measure in accordance with a perceptual error norm in which at least some error contributions are weighted in approximate degree to which they are perceived by the human ear during synthesis; and adapting the information stored in the mapping store in accordance with the error measure.

Claims

1. A method of producing sound data for a desired sound having a temporal progression, comprising the steps of:

applying to an adaptive function mapper control parameters including;

a time parameter, said time parameter specifying from among an ordered sequence of points a particular point within said temporal progression; and

at least one parameter selected from the set of timbre space coordinates, pitch, Δ

pitch, articulation and dynamic; and

using the adaptive function mapper, mapping from the control parameters to synthesis parameters to be applied to a sound synthesizer.

2. The method of claim 1, wherein the adaptive function mapper effects a several-fold multiplication in the number of parameters from the control parameters to the synthesis parameters.

3. The method of claim 1, wherein the time parameter is canonical time, derived by warping real time with respect to a plurality of sound samples to a common time base.

4. The method of claim 1, wherein the timbre space coordinates are coordinates within a perceptually uniform timbre space.

5. The method of claim 1, wherein the adaptive function mapper is a neural network.

6. The method of claim 1, wherein the adaptive function mapper is a simulated neural network.

7. The method of claim 1, wherein the synthesis parameters are to be applied to an additive sound synthesizer and include amplitudes of partials.

8. The method of claim 7, wherein the synthesis parameters are to be applied to an inverse FFT additive sound synthesizer and include, in addition to the amplitudes of partials, noise parameters.

9. The method of claim 8, wherein the noise parameters include broadband noise parameters and narrowband noise parameters.

10. A method of training an adaptive function mapper to produce, in accordance with information stored in a mapping store, synthesis parameters to be applied to a sound synthesizer, the method comprising the steps of:

analyzing sounds to produce sound parameters describing the sounds;

further analyzing the sound parameters to produce control parameters;

applying the control parameters to the adaptive function mapper, the adaptive function mapper in response producing trial synthesis parameters comparable to the sound parameters;

deriving from the sound parameters and the trial synthesis parameters an error measure in accordance with a perceptual error norm in which at least some error contributions are weighted in approximate degree to which an analytical model of human auditory perception predicts that they will be perceived by the human ear during synthesis; and

adapting the information stored in the mapping store in accordance with the error measure.

11. The method of claim 10, wherein the error contributions are weighted in accordance with a monotonically increasing, frequency-dependent weighting function.

12. The method of claim 11, wherein the error contributions are further weighted in accordance with a monotonically increasing function of a time derivative of the error measure.

13. The method of claim 10, wherein the control parameters include:

at least one parameter selected from the set of time and timbre space coordinates; and

at least one parameter selected from the set of pitch, Δ

pitch, articulation and dynamic.

14. The method of claim 13, wherein the time parameter is canonical time, derived by warping real time with respect to a plurality of sound samples to a common time base.

15. The method of claim 13, wherein the timbre space coordinates are coordinates within a perceptually uniform timbre space.

16. The method of claim 10, wherein the adaptive function mapper effects a several-fold multiplication in the number of parameters from the control parameters to the synthesis parameters.

17. The method of claim 10, wherein the adaptive function mapper is a neural network.

18. The method of claim 10, wherein the adaptive function mapper is a simulated neural network.

19. The method of claim 10, wherein the synthesis parameters are to be applied to an additive sound synthesizer and include amplitudes of partials.

20. The method of claim 19, wherein the synthesis parameters are to be applied to an inverse FFT additive sound synthesizer and include, in addition to the amplitudes of partials, noise parameters.

21. The method of claim 20, wherein the noise parameters include broadband noise parameters and narrowband noise parameters.

22. An apparatus for producing sound data for a desired sound having a temporal progression, comprising:

an adaptive function mapper;

means for applying to the adaptive function mapper control parameters including;

at least one parameter selected from the set of time and timbre space coordinates; and

at least one parameter selected from the set of pitch, Δ

pitch, articulation and dynamic; and

the adaptive function mapper comprising means for mapping from the control parameters to synthesis parameters to be applied to a sound synthesizer;

wherein said time parameter specifies from among an ordered sequence of points a particular point within said temporal progression.

23. An apparatus for producing synthesis parameters to be applied to a sound synthesizer, the apparatus comprising:

an adaptive function mapper;

a mapping store coupled to the adaptive function mapper

means for analyzing sounds to produce sound parameters describing the sounds;

means for further analyzing the sound parameters to produce control parameters;

means for applying the control parameters to the adaptive function mapper, the adaptive function mapper in response producing trial synthesis parameters comparable to the sound parameters;

means for deriving from the sound parameters and the trial synthesis parameters an error measure in accordance with a perceptual error norm in which at least some error contributions are weighted in approximate degree to which an analytical model of human auditory perception predicts that they will be perceived by the human ear during synthesis; and

means for adapting the information stored in the mapping store in accordance with the error measure.

24. A method of producing sound data, comprising the steps of:

applying to an adaptive function mapper a trigger input signal; and

using the adaptive function mapper, producing synthesis parameters to be applied to a sound synthesizer, said synthesis parameters including a time constant representative of a decaying exponential function of time.

25. The method of claim 24, wherein the adaptive function mapper effects a several-fold multiplication in the number of parameters from the control parameters to the synthesis parameters.

26. The method of claim 24, wherein the timbre space coordinates are coordinates within a perceptually uniform timbre space.

27. The method of claim 24, wherein the adaptive function mapper is a neural network.

28. The method of claim 24, wherein the adaptive function mapper is a simulated neural network.

29. The method of claim 24, wherein the synthesis parameters are to be applied to an additive sound synthesizer and include amplitudes of partials.

30. The method of claim 24, wherein amplitudes of partials and time constants are produced in one-to-one correspondence.