Derivation of control signals from real-time overtone measurements

• US 7,960,640 B2
• Filed: 09/30/2003
• Issued: 06/14/2011
• Est. Priority Date: 05/15/1998
• Status: Expired due to Fees

• Current Assignee: NRI R&D Patent Licensing, LLC (Advanced Touchscreen and Gesture Technologies, LLC)
• Original Assignee: Lester F. Ludwig
• Inventors: Ludwig, Lester F.
• Primary Examiner(s): Fletcher; Marlon T
• Application Number:
  US10/676,926
• Publication Number:
  US 20040069128A1
• Time in Patent Office: 2,814 Days
• Field of Search: None
• US Class Current: 84/735
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G10H 1/00 Details of electrophonic musical instruments
• CPC Class Codes: G10H 1/0058 Transmission between separate instruments or between individual components of a musical system G10H1/0083 takes precedence
• CPC Class Codes: G10H 1/0066 using a MIDI interface
• CPC Class Codes: G10H 1/348 Switches actuated by parts of the body other than fingers
• CPC Class Codes: G10H 1/46 Volume control
• CPC Class Codes: G10H 2220/521 Hall effect transducers or similar magnetic field sensing semiconductor devices, e.g. for string vibration sensing or key movement sensing
• CPC Class Codes: G10H 2220/525 Piezoelectric transducers for vibration sensing or vibration excitation in the audio range; Piezoelectric strain sensing, e.g. as key velocity sensor; Piezoelectric actuators, e.g. key actuation in response to a control voltage
• CPC Class Codes: G10H 2230/095 Spint zither, i.e. mimicking any neckless stringed instrument in which the strings do not extend beyond the sounding board
• CPC Class Codes: G10H 2230/101 Spint koto, i.e. mimicking any traditional asian-style plucked zither with movable bridges
• CPC Class Codes: G10H 2230/115 Spint sitar, i.e. mimicking any long-necked plucked string instrument with a large number of additional non-playable sympathetic resonating strings or an additional gourd-like resonating chamber
• CPC Class Codes: G10H 2230/145 Spint guitar keyboard, i.e. mimicking a combination of a guitar-like instrument, with or without strings, and a piano-like keyboard, e.g. with white and black keys arranged like on a piano
• CPC Class Codes: G10H 2230/271 Spint gong, i.e. mimicking circular flat, nippled or bowl-shaped metallic percussion instruments G10H2230/321 takes precedence
• CPC Class Codes: G10H 3/18 using a string, e.g. electric guitar mechanical features G10D1/085
• CPC Class Codes: G10H 3/26 : using electric feedback
• CPC Class Codes: Y10S 84/10 : Feedback

Abstract

A system for control signal generation using detected dynamic characteristics of components of an incoming electronic signal. Fixed or adjustable bandpass filters are coupled to signal parameter measurement elements. Each filter isolates a particular overtone component from the incoming electronic signal for isolated signal parameter measurement. Pitch and amplitude of a plurality of overtones may be individually measured. A control signal processor generates one or more outgoing control signals based upon one or more isolated overtone parameter signals. The control signal processor may use these to generate individual associated output signals, subject to optional mathematical operations and warpings, or combine measured parameters of several overtones via mathematical operations to other output signals. The invention may be used to enhance audio-to-MIDI converters, or as an add-on to a traditional fundamental-pitch based audio-to-MIDI converters, giving musicians and vocalists valuable new levels of timbre control over audio synthesis and signal processing.

Claims

1. A system for control signal generation using detected dynamic characteristics of frequency components of an incoming electronic signal, said incoming electronic signal comprising a fundamental frequency component and at least one overtone component of a higher frequency than said fundamental frequency component, said fundamental frequency component and said at least one overtone component comprising an amplitude parameter and a pitch parameter, said system comprising:

at least one bandpass filter adapted to isolate said at least one overtone component from said incoming electronic signal to produce an isolated overtone signal;

a separate signal parameter measurement element operatively coupled with each filter of said at least one bandpass filter, wherein said signal parameter measurement element provides amplitude measurement of said isolated overtone signal resulting in an isolated overtone parameter signal; and

a parameter signal processing unit for receiving said isolated overtone parameter signal, said parameter signal processing unit generating an outgoing control signal based upon said isolated overtone parameter signal.

2. The system according to claim 1, wherein said isolated overtone parameter signal comprises an amplitude parameter.

3. The system according to claim 1, wherein said signal parameter measurement element further provides pitch measurement resulting in said isolated overtone parameter signal comprising a pitch parameter.

4. The system according to claim 1, wherein said incoming electronic signal is generated by a vibration-sensing transducer in response to vibrations of a vibrating element.

5. The system according to claim 1, wherein said incoming electronic signal further includes a plurality of overtone components, wherein each of said plurality of overtone components have a higher frequency than said fundamental frequency component, said fundamental frequency component and each of said plurality of overtone components comprising an amplitude parameter and a pitch parameter, said system further comprising:

a filter bank comprising a plurality of said bandpass filters, wherein each bandpass filter of said plurality of bandpass filters is adapted to isolate a particular overtone component of said plurality of overtone components to generate an isolated overtone signal, said filter bank providing a plurality of isolated overtone signals generated by said plurality of bandpass filters, wherein;

said separate signal parameter measurement element is operatively coupled with each of said plurality of bandpass filters comprising said filter bank, wherein each of said plurality of signal parameter measurement elements is adapted to provide amplitude measurement of a particular isolated overtone signal of said plurality of isolated overtone signals to generate an isolated overtone parameter signal, and wherein;

said parameter signal processing unit is adapted to receive said isolated overtone parameter signal from each of said plurality of signal parameter measurement elements, said parameter signal processing unit generating at least one outgoing control signal based upon one or more of said plurality of isolated overtone parameter signals.

6. The system according to claim 1, wherein said incoming electronic signal is generated by a vibration-sensing transducer in response to a vibrating string.

7. The system according to claim 1, wherein said incoming electronic signal is generated by a vibration-sensing transducer in response to a human voice.

8. The system according to claim 7, wherein said human voice comprises at least one sung vowel.

9. The system according to claim 1, wherein said incoming electronic signal is generated by a vibration-sensing transducer in response to a vibrating reed.

10. The system according to claim 1, wherein said incoming electronic signal is generated by a vibration-sensing transducer in response to a vibrating column of air.

11. The system according to claim 1, wherein said incoming electronic signal is generated by a vibration-sensing transducer in response to a vibrating element of a percussion instrument.

12. The system according to claim 1, wherein said parameter processing unit generates said outgoing control signal based upon a ratio of a parameter of said isolated overtone parameter signal relative to a parameter of said fundamental frequency of said incoming electronic signal.

13. The system according to claim 5, wherein said parameter processing unit generates said outgoing control signal based upon a ratio of a first isolated overtone parameter signal associated with a first isolated overtone signal of said plurality of said isolated overtone signals relative to a second isolated overtone parameter signal associated with a second isolated overtone signal of said plurality of said isolated overtone signals.

14. The system according to claim 1, wherein said outgoing control signal comprises a signal of MIDI format.

15. The system according to claim 1, wherein said outgoing control signal comprises a signal of MIDI note format.

16. The system according to claim 1, wherein said outgoing control signal comprises a signal of MIDI continuous controller format.

17. The system according to claim 1, wherein said bandpass filter is a controllable bandpass filter.

18. The system according to claim 1, wherein said outgoing control signal is communicated to an external system comprising a synthesizer.

19. The system according to claim 1, wherein said outgoing control signal is communicated to an external system comprising a signal processor controlled by said outgoing control signal.

20. The system according to claim 1, wherein said incoming electronic signal is generated by a vibration-sensing transducer in response to plucking of a vibrating element.

21. The system according to claim 1, wherein said incoming electronic signal is generated by a vibration-sensing transducer in response to bowing of a vibrating element.

22. The system according to claim 1, wherein said incoming electronic signal is generated by a vibration-sensing transducer in response to vibrations of a vibrating element, wherein

said incoming electronic signal is further amplified by an amplifier to produce an amplified drive signal, and wherein

said amplified drive signal drives at least one drive transducer to stimulate said vibrating element.

23. The system according to claim 22, wherein said at least one drive transducer is a piezo transducer.

24. The system according to claim 22, wherein said vibration-sensing transducer is a piezo transducer.

25. The system according to claim 1, wherein variations of said isolated overtone parameter signal are tracked over time.

26. The system according to claim 1, wherein said isolated overtone parameter signal is further processed by nonlinear warping.

27. The system according to claim 1, wherein said signal parameter measurement element comprises a nonlinear warping characteristic.

28. The system according to claim 26, wherein said nonlinear warping is performed by said parameter processing unit.

29. The system according to claim 26, wherein said isolated overtone parameter signal comprises nonlinear warping exhibiting substantially logarithmic behavior.

30. The system according to claim 26, wherein said isolated overtone parameter signal comprises nonlinear warping exhibiting piecewise linear behavior.

31. The system according to claim 1, wherein said parameter processing unit generates said outgoing control signal by mathematical operations applied to a plurality of parameters of said isolated overtone parameter signal.

32. The system according to claim 31, wherein said mathematical operations include averaging a plurality of parameters of said isolated overtone parameter signal.

33. The system according to claim 31, wherein said mathematical operations include sums-of-squares calculations of a plurality of parameters of said isolated overtone parameter signal.

34. The system according to claim 5, wherein there is a plurality of said outgoing control signals, said plurality of said outgoing control signals communicated to an external synthesizer, wherein said external synthesizer generates a synthesized audio signal based upon and said outgoing control signal, wherein said synthesized audio signal complements a harmonic balance of said incoming electronic signal.

35. The system according to claim 5, wherein there is a plurality of said outgoing control signals, said plurality of said outgoing control signals communicated to an external synthesizer, where said external synthesizer generates a synthesized audio signal based upon said outgoing control signal, wherein said synthesized audio signal mimics a harmonic balance of said incoming electronic signal.

36. The system according to claim 1, wherein said parameter processing unit generates said outgoing control signal by assigning said isolated overtone parameter signal to a particular type of outgoing control signal.

37. The system according to claim 1, wherein said parameter processing unit combines a plurality of parameters of said isolated overtone parameter signal to generate a mathematically combined signal, and wherein said parameter processing unit generates said outgoing control signal by assigning said mathematically combined signal to a particular type of outgoing control signal.

38. The system according to claim 1, wherein said parameter processing unit processes a parameter of said isolated overtone parameter signal to generate a processed signal, and wherein said parameter processing unit generates said outgoing control signal by assigning said processed signal to a particular type of outgoing control signal.

39. The system according to claim 1, wherein said parameter processing unit processes a plurality of parameters of said isolated overtone parameter signal to generate a mathematically combined signal, and wherein said parameter processing unit generates said outgoing control signal by assigning said mathematically combined signal to a particular type of outgoing control signal.

40. The system according to claim 1, said system further comprising:

a model-based overtone series calculator coupled with said bandpass filter, wherein said calculator provides filter control signals for centering said bandpass filter.

41. The system according to claim 40, wherein said calculator further provides overtone center line frequency information to one or more of said at least one bandpass filters.

42. A method for control signal generation using detected dynamic characteristics of frequency components of an incoming electronic signal, said incoming electronic signal comprising a fundamental frequency component and at least one overtone component of a higher frequency than said fundamental frequency component, said fundamental frequency component and said at least one overtone component comprising an amplitude parameter and a pitch parameter, said method comprising:

isolating said at least one overtone component from said incoming electronic signal using at least one bandpass filter, wherein said isolating results in the production of an isolated overtone signal;

measuring amplitude of said isolated overtone signal using a separate signal parameter measurement element operatively coupled with each filter of said at least one bandpass filter, wherein said measuring results in an isolated overtone parameter signal;

receiving said isolated overtone parameter signal at a parameter signal processing unit; and

generating an outgoing control signal at said parameter signal processing unit, wherein said outgoing control signal is generated based upon said isolated overtone parameter signal.

43. The method according to claim 42, wherein said isolated overtone parameter signal comprises an amplitude parameter.

44. The method according to claim 42, wherein said isolated overtone parameter signal comprises a pitch parameter.

45. The method according to claim 42, said method further comprising:

generating said incoming electronic signal using a vibration-sensing transducer, wherein said incoming electronic signal is generated in response to vibrations present in a vibrating element.

46. The method according to claim 42, wherein said incoming electronic signal further includes a plurality of overtone components, wherein each of said plurality of overtone components have a higher frequency than said fundamental frequency component, said fundamental frequency component and each of said plurality of overtone components comprising an amplitude parameter and a pitch parameter, said method further comprising:

isolating each overtone component of said plurality of overtone components of said incoming electronic signal using a corresponding plurality of bandpass filters, wherein said isolating of said plurality of overtone components generate a corresponding plurality of isolated overtone signals;

directing each isolated overtone signal of said plurality of isolated overtone signals to a separate signal parameter measurement element;

measuring amplitude of said plurality of isolated overtone signals to generate a corresponding plurality of isolated overtone parameter signals;

receiving said plurality of isolated overtone parameter signals at a parameter signal processing unit; and

generating at least one outgoing control signal for each parameter signal of said plurality of isolated overtone parameter signals.

47. The method according to claim 42, said method further comprising:

generating said incoming electronic signal using a vibration-sensing transducer, wherein said incoming electronic signal is generated in response to vibration present in a vibrating string.

48. The method according to claim 42, said method further comprising:

generating said incoming electronic signal using a vibration-sensing transducer, wherein said incoming electronic signal is generated in response to a human voice.

49. The method according to claim 48, wherein said human voice comprises at least one sung vowel.

50. The method according to claim 42, said method further comprising:

generating said incoming electronic signal using a vibration-sensing transducer, wherein said incoming electronic signal is generated in response to a vibrating reed.

51. The method according to claim 42, said method further comprising:

generating said incoming electronic signal using a vibration-sensing transducer, wherein said incoming electronic signal is generated in response to a vibrating column of air.

52. The method according to claim 42, said method further comprising:

generating said incoming electronic signal using a vibration-sensing transducer, wherein said incoming electronic signal is generated in response to a vibrating element of a percussion instrument.

53. The method according to claim 42, wherein said parameter processing unit generates said outgoing control signal based upon a ratio of a parameter of said isolated overtone parameter signal relative to a parameter of said fundamental frequency of said incoming electronic signal.

54. The method according to claim 46, wherein said parameter processing unit generates said outgoing control signal based upon a ratio of a first isolated overtone parameter signal associated with a first isolated overtone signal of said plurality of said isolated overtone signals relative to a second isolated overtone parameter signal associated with a second isolated overtone signal of said plurality of said isolated overtone signals.

55. The method according to claim 42, wherein said outgoing control signal comprises a signal of MIDI format.

56. The method according to claim 42, wherein said outgoing control signal comprises a signal of MIDI note format.

57. The method according to claim 42, wherein said outgoing control signal comprises a signal of MIDI continuous controller format.

58. The method according to claim 42, wherein said bandpass filter is a controllable bandpass filter.

59. The method according to claim 42, wherein said outgoing control signal is communicated to an external system comprising a synthesizer.

60. The method according to claim 42, wherein said outgoing control signal is communicated to an external system comprising a signal processor controlled by said outgoing control signal.

61. The method according to claim 42, said method further comprising:

generating said incoming electronic signal using a vibration-sensing transducer, wherein said incoming electronic signal is generated in response to plucking of said vibrating element.

62. The method according to claim 42, said method further comprising:

generating said incoming electronic signal using a vibration-sensing transducer, wherein said incoming electronic signal is generated in response to bowing of said vibrating element.

63. The method according to claim 42, said method further comprising:

generating said incoming electronic signal using a vibration-sensing transducer, wherein said incoming electronic signal is generated in response to vibration present in a vibrating element;

amplifying further said incoming electronic signal to produce an amplified drive signal; and

stimulating said vibrating element using at least one drive transducer driven by said amplified drive signal.

64. The method according to claim 63, wherein said at least one drive transducer is a piezo transducer.

65. The method according to claim 63, wherein said vibration-sensing transducer is a piezo transducer.

66. The method according to claim 42, wherein variations of said isolated overtone parameter signal are tracked over time.

67. The method according to claim 42, wherein said isolated overtone parameter signal is further processed by nonlinear warping.

68. The method according to claim 42, wherein said signal parameter measurement element comprises a nonlinear warping characteristic.

69. The method according to claim 67, wherein said nonlinear warping is performed by said parameter processing unit.

70. The method according to claim 67, wherein said isolated overtone parameter signal comprises nonlinear warping exhibiting substantially logarithmic behavior.

71. The method according to claim 67, wherein said isolated overtone parameter signal comprises nonlinear warping exhibiting piecewise linear behavior.

72. The method according to claim 42, wherein said parameter processing unit generates said outgoing control signal by mathematical operations applied to a plurality of parameters of said isolated overtone parameter signal.

73. The method according to claim 72, wherein said mathematical operations include averaging a plurality of parameters of said isolated overtone parameter signal.

74. The method according to claim 72, wherein said mathematical operations include sums-of-squares calculations of a plurality of parameters of said isolated overtone parameter signal.

75. The method according to claim 46, wherein there is a plurality of said outgoing control signals, said plurality of said outgoing control signals communicated to an external synthesizer, where said external synthesizer generates a synthesized audio signal based upon said outgoing control signal, wherein said synthesized audio signal complements a harmonic balance of said incoming electronic signal.

76. The method according to claim 73, wherein there is a plurality of said outgoing control signals, said plurality of said outgoing control signals communicated to an external synthesizer, where said external synthesizer generates a synthesized audio signal based upon said outgoing control signal, wherein said synthesized audio signal mimics a harmonic balance of said incoming electronic signal.

77. The method according to claim 42, wherein said parameter processing unit generates said outgoing control signal by assigning said isolated overtone parameter signal to a particular type of outgoing control signal.

78. The method according to claim 42, wherein said parameter processing unit combines a plurality of parameters of said isolated overtone parameter signal to generate a mathematically combined signal, and wherein said parameter processing unit generates said outgoing control signal by assigning said mathematically combined signal to a particular type of outgoing control signal.

79. The method according to claim 42, wherein said parameter processing unit processes a parameter of said isolated overtone parameter signal to generate a processed signal, and wherein said parameter processing unit generates said outgoing control signal by assigning said processed signal to a particular type of outgoing control signal.

80. The method according to claim 42, wherein said parameter processing unit processes a plurality of parameters of said isolated overtone parameter signal to generate a mathematically combined signal, and wherein said parameter processing unit generates said outgoing control signal by assigning said mathematically combined signal to a particular type of outgoing control signal.

81. The method according to claim 42, said method further comprising:

coupling a model-based overtone series calculator with said bandpass filter, wherein said calculator provides filter control signals for centering said bandpass filter.

82. The method according to claim 81, wherein said calculator further provides overtone center line frequency information to one or more of said at least one bandpass filter.

83. A method for control signal generation using detected dynamic characteristics of frequency components of an incoming electronic signal, said incoming electronic signal comprising a fundamental frequency component and at least one overtone component of a higher frequency than said fundamental frequency component, said fundamental frequency component and said at least one overtone component comprising an amplitude parameter and a pitch parameter, said method comprising:

isolating said at least one overtone component from said incoming electronic signal to provide an isolated overtone signal;

measuring amplitude of said isolated overtone signal to generate an isolated overtone parameter signal comprising said amplitude; and

generating an outgoing control signal based upon said isolated overtone parameter signal, wherein said outgoing control signal is adapted to control an external system.

84. The method according to claim 83, wherein said isolated overtone parameter signal comprises an amplitude parameter.

85. The method according to claim 83, wherein said isolated overtone parameter signal comprises a pitch parameter.

86. The method according to claim 83, wherein said incoming electronic signal further includes a plurality of overtone components, wherein each of said plurality of overtone components have a higher frequency than said fundamental frequency component, said fundamental frequency component and each of said plurality of overtone components comprising an amplitude parameter and a pitch parameter, said method further comprising:

isolating each overtone component of said plurality of overtone components of said incoming electronic signal using a corresponding plurality of bandpass filters, wherein said isolating of said plurality of overtone components generate a corresponding plurality of isolated overtone signals;

directing each isolated overtone signal of said plurality of isolated overtone signals to a separate signal parameter measurement element;

measuring amplitude of said plurality of isolated overtone signals to generate a corresponding plurality of isolated overtone parameter signals;

receiving said plurality of isolated overtone parameter signals at a parameter signal processing unit; and

generating a separate outgoing control signal for each parameter signal of said plurality of isolated overtone parameter signals.

87. The method according to claim 83, wherein said parameter processing unit generates said outgoing control signal based upon a ratio of a parameter of said isolated overtone parameter signal relative to a parameter of said fundamental frequency of said incoming electronic signal.

88. The method according to claim 83, wherein said parameter processing unit generates said outgoing control signal based upon a ratio of a first parameter of said isolated overtone parameter signal relative to a second parameter of said isolated overtone parameter signal.

89. The method according to claim 83, wherein said outgoing control signal comprises a signal of MIDI format.

90. The method according to claim 83, wherein said outgoing control signal is communicated to an external system comprising a synthesizer.

91. The method according to claim 83, wherein said outgoing control signal is communicated to an external system comprising a signal processor controlled by said outgoing control signal.

92. The method according to claim 83, wherein variations of said isolated overtone parameter signal are tracked over time.

93. The method according to claim 83, wherein said isolated overtone parameter signal is further processed by nonlinear warping.

94. The method according to claim 83, wherein said parameter processing unit generates said outgoing control signal by mathematical operations applied to a plurality of parameters of said isolated overtone parameter signal.

95. The method according to claim 83, said method further comprising:

coupling a model-based overtone series calculator with said bandpass filter, wherein said calculator provides filter control signals for centering said bandpass filter.

96. The method according to claim 95, wherein said calculator further provides overtone center line frequency information to one or more of said at least one bandpass filter.