COMPUTER INTERFACE FOR POLYPHONIC STRINGED INSTRUMENTS

• US 20110303075A1
• Filed: 06/21/2011
• Published: 12/15/2011
• Est. Priority Date: 07/10/2008
• Status: Active Grant

• Current Assignee: Kesumo, LLC
• Original Assignee: Stringport, LLC
• Inventors: McMillen, Keith, Shaver, Chris
• Granted Patent: US 8,581,086 B2
• Time in Patent Office: Days
• US Class Current: 84/645
• CPC Class Codes: G10H 1/125 using a digital filter
• CPC Class Codes: G10H 1/342 for guitar-like instruments with or without strings and with a neck on which switches or string-fret contacts are used to detect the notes being played electric guitars in which the tones are generated by the vibration of strings G10H3/18
• CPC Class Codes: G10H 2210/031 Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
• CPC Class Codes: G10H 2210/155 Musical effects
• CPC Class Codes: G10H 2210/311 Distortion, i.e. desired non-linear audio processing to change the tone colour, e.g. by adding harmonics or deliberately distorting the amplitude of an audio waveform
• CPC Class Codes: G10H 2220/155 User input interfaces for electrophonic musical instruments
• CPC Class Codes: G10H 2220/161 with 2D or x/y surface coordinates sensing
• CPC Class Codes: G10H 2220/361 Mouth control in general, i.e. breath, mouth, teeth, tongue or lip-controlled input devices or sensors detecting, e.g. lip position, lip vibration, air pressure, air velocity, air flow or air jet angle
• CPC Class Codes: G10H 2220/395 Acceleration sensing or accelerometer use, e.g. 3D movement computation by integration of accelerometer data, angle sensing with respect to the vertical, i.e. gravity sensing
• CPC Class Codes: G10H 2240/285 USB, i.e. either using a USB plug as power supply or using the USB protocol to exchange data
• CPC Class Codes: G10H 2240/311 MIDI transmission G10H2240/056 takes precedence
• CPC Class Codes: G10H 2250/311 Neural networks for electrophonic musical instruments or musical processing, e.g. for musical recognition or control, automatic composition or improvisation
• CPC Class Codes: G10H 2250/501 Formant frequency shifting, sliding formants
• CPC Class Codes: G10H 2250/615 Waveform editing, i.e. setting or modifying parameters for waveform synthesis
• CPC Class Codes: G10H 3/186 Means for processing the signal picked up from the strings filtering G10H1/12
• CPC Class Codes: G10H 3/188 for converting the signal to digital format transmission using a MIDI interface G10H1/0066

Abstract

An interface device is described that allows the audio signals from a polyphonic stringed instrument to be introduced into a personal computer environment for feature extraction and signal processing.

Claims

1. A computer-implemented method for processing audio signals for a stringed instrument, comprising:

receiving a serial data stream with a serial data interface of a computing device, the serial data stream encoding a plurality of digital audio signals, each digital audio signal representing one of a plurality of strings of the stringed instrument;

extracting the encoded digital audio signals from the serial data stream using the computing device; and

processing each of the extracted digital audio signals with the computing device, thereby generating a plurality of processed digital audio signals, each of the processed digital audio signals corresponding to one of the plurality of strings of the stringed instrument.

2. The method of claim 1 wherein processing the extracted digital audio signals comprises one or more of dynamics processing, equalization, pitch-shifting, filtering, frequency modulation, amplitude modulation, delay, reverberation, distortion, wave shaping, driving wave tables, stimulating resonances, gating, or limiting.

3. The method of claim 1 wherein processing each of the extracted digital audio signals comprises processing each of the extracted digital audio signals using a plurality of processing modules that simultaneously process each extracted digital audio signal in both a time domain and a frequency domain.

4. The method of claim 1 further comprising recording each of the digital audio signals for subsequent processing.

5. The method of claim 1 further comprising generating graphical representations corresponding to each of the digital audio signals, the graphical representations representing one or more of pitch, dynamics, or timbre for the corresponding string of the string instrument.

6. The method of claim 1 further comprising controlling a special effect using information extracted from the serial digital stream.

7. The method of claim 1 further comprising generating performance event data representing performance events for each of the plurality of strings with reference to the corresponding extracted digital audio signals, the performance events relating to specific types of interaction with the strings by a musician, wherein processing of the extracted digital audio signals is done with reference to the performance event data.

8. The method of claim 7 wherein generating the performance event data comprises detecting event candidates with reference to peaks associated with the extracted digital audio signals, and classifying the event candidates into one or more of a plurality of event classifications.

9. The method of claim 8 further comprising identifying a beginning of a note with reference to classification of the event candidates.

10. The method of claim 8 wherein classifying the event candidates into the one or more event classifications is done using a neural network.

11. The method of claim 1 further comprising extracting a plurality of audio signal characteristics from the extracted digital audio signals, wherein processing of the extracted digital audio signals is done with reference to the audio signal characteristics.

12. The method of claim 11 wherein the audio signal characteristics include any of a continuous pitch of a fundamental harmonic of a string, an amplitude, a centroid, brightness, even/odd harmonic balance, noise, spectral shape, or complete spectrum.

13. The method of claim 1 further comprising generating a plurality of acoustic instrument messages with reference to the extracted digital audio signals, each acoustic instrument message summarizing spectral information corresponding to a particular one of the plurality of strings of the stringed instrument.

14. A computer program product for processing audio signals for a stringed instrument, the computer program product comprising at least one computer-readable storage medium having computer program instructions stored therein configured to enable at least one computing device to:

receive a serial data stream with a serial data interface of a computing device, the serial data stream encoding a plurality of digital audio signals, each digital audio signal representing one of a plurality of strings of the stringed instrument;

extract the encoded digital audio signals from the serial data stream using the computing device; and

process each of the extracted digital audio signals with the computing device, thereby generating a plurality of processed digital audio signals, each of the processed digital audio signals corresponding to one of the plurality of strings of the stringed instrument.