Automatic recognition and matching of tempo and phase of pieces of music, and an interactive music player based thereon

• US 7,615,702 B2
• Filed: 01/07/2002
• Issued: 11/10/2009
• Est. Priority Date: 01/13/2001
• Status: Active Grant

• Current Assignee: Native Instruments GmbH (Soundwide GmbH)
• Original Assignee: Native Instruments GmbH (Soundwide GmbH)
• Inventors: Kurz, Michael, Holl, Thomas, Haver, Daniel, Diepstraten, Toine, Becker, Friedemann
• Primary Examiner(s): Fletcher; Marlon T
• Application Number:
  US10/251,000
• Publication Number:
  US 20040069123A1
• Time in Patent Office: 2,864 Days
• Field of Search: None
• US Class Current: 84/612
• CPC Class Codes: G10H 1/00 Details of electrophonic musical instruments
• CPC Class Codes: G10H 2210/076 for extraction of timing, tempo; Beat detection
• CPC Class Codes: G10H 2240/061 MP3, i.e. MPEG-1 or MPEG-2 Audio Layer III, lossy audio compression
• CPC Class Codes: G10H 2240/325 Synchronizing two or more audio tracks or files according to musical features or musical timings

Abstract

A method of matching the tempo and phase in pieces of music which allows the conjunction of the pieces of music to form a continuous stream of music. The interactive music player which digitally executes the method of matching the tempo and phase in pieces of music is also disclosed.

Claims

1. A method for detecting a tempo and phase of a piece of music available in digital format, comprising:

a. approximating the tempo (A) of the piece of music by means of a statistical evaluation (STAT) of the time intervals (Ti) between rhythm-relevant beat information in the digital audio data (Ei);

b. approximating the phase (P) of the piece of music with reference to the position of the beats in the digital audio data within the time grid of a reference oscillator (MCLK) oscillating at a frequency proportional to the tempo established;

c. successively correcting the established tempo (A) and phase (P) of the piece of music with reference to a possible phase shift of the reference oscillator (MCLK) relative to the digital audio data by evaluating the resulting systematic phase shift and regulating the frequency of the reference oscillator in proportion to the phase shift established.

2. The method for detecting the tempo and phase of a piece of music available in digital format according to claim 1, wherein rhythm-relevant information (Ti) is obtained by band-pass filtering (F1, F2) of the basic digital audio data in various frequency ranges.

3. The method for detecting the tempo and phase of a piece of music available in digital format according to claim 1, wherein rhythmic intervals in the audio data are transformed, if necessary by raising their frequency by a power of 2, into a pre-defined frequency octave (OKT), where they provide time intervals (T1io. . . T3io) for establishing the tempo.

4. The method for detecting the tempo and phase of a piece of music available in digital format according to claim 3, wherein the frequency transformation (OKT) is preceded by a grouping of rhythmic intervals (Ti) into pairs (T2i) or groups of three (T3i), by addition of their time values.

5. The method for detecting the tempo and phase of a piece of music available in digital format according to claim 2, wherein the quantity of data obtained for time intervals (BPM1, BPM2) in the rhythm-relevant beat information (N) is investigated for accumulation points, and the approximation of tempo is carried out on the basis of information referring to an accumulation maximum.

6. The method for detecting the tempo and phase of piece of music available in digital format according to claim 1, wherein, for the approximation of the phase (P) of the piece of music, the phase of the reference oscillator (MCLK) is selected in such a manner that the maximum possible agreement is set between the rhythm-relevant beat information in the digital audio data and the zero passes of the reference oscillator (MCLK).

7. The method for detecting the tempo and phase of a piece of music available in digital format according to claim 1, wherein a successive correction (2,3,4,5) of the established tempo and phase of the piece of music takes place at regular intervals over such short time intervals that resulting correction movements or correction shifts remain below the threshold of audibility.

8. The method for detecting the tempo and phase of a piece of music available in digital format according to claim 1, wherein all successive corrections of the established tempo and phase of the piece of music are accumulated (4) over time and further corrections are made on this basis with constantly increasing precision,

a. wherein, in particular, successive corrections are continued until the error in the established tempo falls below a predetermined tolerable error threshold value of less than 0.1%.

9. A method for synchronizing at least two pieces of music available in digital format, comprising:

a. completing establishment of the tempo and phase of the first piece of music by;

i. approximating the tempo (A) of the first piece of music by means of a statistical evaluation (STAT) of the time intervals (Ti) between rhythm-relevant beat information in the digital audio data (Ei);

ii. approximating the phase (P) of the first piece of music with reference to the position of the beats in the digital audio data within the time grid of a reference oscillator (MCLK) oscillating at a frequency proportional to the tempo established;

iii. successively correcting the established tempo (A) and phase (P) of the first piece of music with reference to a possible phase shift of the reference oscillator (MCLK) relative to the digital audio data by evaluating the resulting systematic phase shift and regulating the frequency of the reference oscillator in proportion to the phase shift established;

b. approximating the tempo and phase of a second piece of music in the same manner as the first piece of music; and

c. matching of the playback rate and the playback phase of the second piece of music by successive adaptation of the frequency and phase of the reference oscillator allocated to the second piece of music to the frequency and phase of the reference oscillator allocated to the first piece of music.

10. The method for synchronising at least two pieces of music available in digital format according to claim 9, wherein the playback rate and playback phase of the other piece of music is matched on the basis of a possible phase shift in the reference oscillator allocated to this other piece of music relative to the reference oscillator of the first piece of music, and the resulting systematic phase shift is evaluated and the frequency of the reference oscillator allocated to the other piece of music is regulated in proportion to the phase shift established.

11. A music player, wherein at least two pieces of music available in digital format can be synchronized in real-time by:

a. completing establishment of the tempo and phase of a first piece of the at least two pieces of music by;

i. approximating the tempo (A) of the first piece of music by means of a statistical evaluation (STAT) of the time intervals (Ti) between rhythm-relevant beat information in the digital audio data (Ei);

ii. approximating the phase (P) of the first piece of music with reference to the position of the beats in the digital audio data within the time grid of a reference oscillator (MCLK) oscillating at a frequency proportional to the tempo established;

iii. successively correcting the established tempo (A) and phase (P) of the first piece of music with reference to a possible phase shift of the reference oscillator (MCLK) relative to the digital audio data by evaluating the resulting systematic phase shift and regulating the frequency of the reference oscillator in proportion to the phase shift established;

b. approximating the tempo and phase of a second piece of the at least two pieces of music in the same manner as the first piece of music; and

c. matching of the playback rate and the playback phase of the second piece of music by successive adaptation of the frequency and phase of the reference oscillator allocated to the second piece of music to the frequency and phase of the reference oscillator allocated to the first piece of music

wherein rhythm-relevant beat information (Ti) from a predetermined past time relative to a current playing position of the pieces of music are used, in each case, as a basis for establishing the tempo in real-time.

12. The music player according to claim 11, wherein synchronized pieces of music can be automatically sorted and played back as a complete work with a unified rhythm.

13. The music player according to claim 11, wherein, for one or more of the synchronized pieces of music, the length of a playback loop extending over one or more beats of the piece of music can be defined and played back in real-time in a beat-synchronized manner, on the basis of the tempo information established for the relevant piece of music.

14. The music player according to claim 11, wherein, for one or more of the synchronized pieces of music, beat-synchronized jump marks can be defined in real-time and moved by whole-number multiples of beats within this piece of music on the basis of the phase information established for the relevant piece of music, wherein each audio-data stream played back can be manipulated in real-time by signal processing means, including by filtering devices or audio effects.

15. The music player according to claim 11, wherein real-time interventions over the time sequence can be stored as digital control information (MIX_DATA) relating to a mixing procedure for several pieces of music and/or additional signal processing.

16. The music player according to claim 11, wherein mixing procedures for pieces of music and/or interactive interventions in pieces of music using audio-signal processing means can be stored, independently of digital audio information for the pieces of music, in the form of digital control information (MIX_DATA) for the purpose of reproduction as a new complete work.

17. The music player according to claim 16, wherein the stored digital control information has a format, which provides information for identifying the pieces of music processed and the time sequence of playback positions and status information for the control elements of the music player allocated to each of the pieces of music.

18. The music player according to claim 11, which is realized with an appropriately programmed computer system fitted with audio interfaces.