AUTOMATIC LABELING AND CONTROL OF AUDIO ALGORITHMS BY AUDIO RECOGNITION

• US 20110075851A1
• Filed: 09/28/2010
• Published: 03/31/2011
• Est. Priority Date: 09/28/2009
• Status: Active Grant

• Current Assignee: Native Instruments
• Original Assignee: iZotope, Inc. (Soundwide GmbH)
• Inventors: LeBoeuf, Jay, Pope, Stephen
• Granted Patent: US 9,031,243 B2
• Time in Patent Office: Days
• US Class Current: 381/56
• CPC Class Codes: G10L 25/51 for comparison or discrimination
• CPC Class Codes: H04R 29/00 Monitoring arrangements; Testing arrangements for hearing aids H04R25/30; detection of loudspeaker connection H04R5/04; sound-field adaptation dependent on speaker detection H04S7/308

Abstract

Controlling a multimedia software application using high-level metadata features and symbolic object labels derived from an audio source, wherein a first-pass of low-level signal analysis is performed, followed by a stage of statistical and perceptual processing, followed by a symbolic machine-learning or data-mining processing component is disclosed. This multi-stage analysis system delivers high-level metadata features, sound object identifiers, stream labels or other symbolic metadata to the application scripts or programs, which use the data to configure processing chains, or map it to other media. Embodiments of the invention can be incorporated into multimedia content players, musical instruments, recording studio equipment, installed and live sound equipment, broadcast equipment, metadata-generation applications, software-as-a-service applications, search engines, and mobile devices.

Claims

1. A method for multi-stage audio signal analysis, the method comprising:

performing a first-stage processing operation on an audio signal, the first stage processing operation including a windowed signal analysis that derives a raw feature vector;

performing a second stage statistical processing operation on the raw feature vector to derive a reduced feature vector;

performing a third stage processing operation on the reduced feature vector to derive at least one sound object label that refers to the original audio signal; and

mapping the at least one sound object label into a stream of control events sent to a sound-object-driven, multimedia-aware software application, wherein any of the processing operations of the first through third stages are configurable and scriptable.

2. The method of claim 1, wherein the audio signal is a file.

3. The method of claim 1, wherein the audio signal is a stream.

4. The method of claim 1, wherein the first stage processing operation is selected from the group consisting of amplitude-detection, FFT, MFCC, LPC, wavelet analysis, spectral measures, and stereo/spatial feature extraction.

5. The method of claim 1, wherein the second stage processing operation is selected from the group consisting of statistical averaging, mean/variance calculation, statistical moments, Gaussian mixture models, principal component analysis (PCA), independent subspace analysis (ISA), hidden Markhov models, tempo-tracking, pitch-tracking, peak/partial-tracking, onset detection, segmentation, and/or bark/sone mapping.

6. The method of claim 1, wherein the third stage processing operation is selected from the group consisting of support vector machines (SVN), neural networks (NN), partitioning/clustering, constraint satisfaction, stream labeling, rule-based expert systems, classification according to instrument, genre, artist, etc., musical transcription, and/or sound object source separation.