Automatic extraction of musical portions of an audio stream

• US 20050126369A1
• Filed: 12/12/2003
• Published: 06/16/2005
• Est. Priority Date: 12/12/2003
• Status: Active Grant

• Current Assignee: Nokia Corporation
• Original Assignee: Nokia Corporation
• Inventors: Kirkeby, Ole, Huopaniemi, Jyri, Sorsa, Timo
• Granted Patent: US 7,179,980 B2
• Time in Patent Office: Days
• US Class Current: 84/615
• CPC Class Codes: G10H 1/00 Details of electrophonic musical instruments
• CPC Class Codes: G10H 2210/046 for differentiation between music and non-music signals, based on the identification of musical parameters, e.g. based on tempo detection
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04H 60/27 Arrangements for recording or accumulating broadcast information or broadcast-related information
• CPC Class Codes: H04H 60/37 for identifying segments of broadcast information, e.g. scenes or extracting programme ID
• CPC Class Codes: H04H 60/48 for recognising items expressed in broadcast information
• CPC Class Codes: H04H 60/58 of audio determination or detection of speech characteristics in general G10L25/00; speech recognition in general G10L15/00

Abstract

Music and non-music portions in an audio stream are identified. The audio stream is digitized and segmented into frames. Selected frames are passed through a filter bank which includes filters having bandwidths approximately proportional to their center frequencies. The spectral flux for each selected frame is calculated and smoothed. Frames having a smoothed spectral flux below a threshold value are associated with music, and frames having a smoothed spectral flux above a threshold value are associated with non-music.

Claims

1. A method for selectively recording music portions of an audio stream, comprising:

receiving an audio stream having music and non-music portions;

segmenting the audio stream into successive frames;

passing each of a plurality of the frames through a filter bank, the filter bank including filters with bandwidths approximately proportional to their center frequencies;

computing a modified spectral flux value for at least a subset of the plurality of frames;

identifying a start frame, the start frame being a frame of the plurality having a modified spectral flux value below a threshold value;

identifying a stop frame, the stop frame being a frame of the plurality having a modified spectral flux value above the threshold value; and

recording a portion of the audio stream bounded by the start and stop frames.

2. The method of claim 1, wherein said passing each of the plurality of frames through a filter bank comprises passing each of the plurality of frames through five Infinite Impulse Response (IIR) filters.

3. The method of claim 2, wherein the five IIR filters comprise a low pass filter, a band pass filter centered at approximately 450 Hz, a band pass filter centered at approximately 900 Hz, a band pass filter centered at approximately 1500 Hz, and a high pass filter.

4. The method of claim 1, further comprising:

determining whether time elapsed between the start frame and the stop frame exceeds a minimum duration, and wherein;

said recording step is performed upon determining that the elapsed time exceeds the minimum duration.

5. The method of claim 1, further comprising:

determining whether a time elapsed since the start frame exceeds a spectral flux threshold reset time; and

resetting the spectral flux threshold value upon determining the elapsed time exceeds the reset time, wherein;

said identifying a stop frame comprises identifying a frame of the plurality having a modified spectral flux value above the reset spectral flux threshold value.

6. The method of claim 1, further comprising:

determining whether a time elapsed since the start frame exceeds a maximum duration, and wherein;

said identifying a stop frame and said recording steps are performed based on the elapsed time not exceeding the maximum duration.

7. The method of claim 1, further comprising receiving a recording control signal initiated from a remotely located mobile terminal, and wherein:

said identifying a start frame comprises locating the start frame in a buffered portion of the audio stream, and

said recording comprises copying the portion of the audio stream received since the start frame into another memory.

8. The method of claim 1, further comprising:

digitizing the audio stream, and wherein;

said segmenting step comprises segmenting the digitized audio stream into successive digitized frames,

said passing step comprises passing each of a plurality of digitized frames through multiple filters, and

said computing step comprises computing a modified spectral flux value for at least a subset of the plurality of digitized frames.

9. The method of claim 1, wherein a modified spectral flux comprises a value of spectral flux averaged over K previous frames, where K is an integer.

10. A method for selectively recording music portions of a radio broadcast, comprising:

receiving a radio broadcast consisting essentially of an audio transmission;

calculating a value of a feature for each of a plurality of frames of the audio transmission;

identifying a start point, the start point being a frame in the audio transmission having a feature value bearing a first relation to a threshold value for the feature;

identifying a stop point, the stop point being a frame in the audio transmission having a feature value bearing a second relation to the threshold value for the feature; and

recording a portion of the audio transmission bounded by the start and stop points.

11. The method of claim 10, wherein the feature comprises a modified spectral flux, and further comprising:

passing each of the plurality of frames through a filter bank, the filter bank including filters with bandwidths approximately proportional to their center frequencies, the filtered frames being used for calculation of a modified spectral flux value for at least a subset of the plurality of frames.

12. The method of claim 10, further comprising receiving a recording control signal initiated from a remotely located mobile terminal, and wherein:

said identifying a start point comprises locating, within a buffered portion of the audio transmission, a frame having a feature value bearing the first relation to the threshold value, and

said recording comprises copying the portion of the audio transmission received since the start point into another memory.

13. A machine-readable medium having machine-executable instructions for performing steps comprising:

receiving an audio stream having music and non-music portions;

segmenting the audio stream into successive frames;

passing each of a plurality of the frames through a filter bank, the filter bank including filters with bandwidths approximately proportional to their center frequencies;

computing a modified spectral flux value for at least a subset of the plurality of frames;

identifying a start frame, the start frame being a frame of the plurality having a modified spectral flux value below a threshold value;

identifying a stop frame, the stop frame being a frame of the plurality having a modified spectral flux value above the threshold value; and

recording a portion of the audio stream bounded by the start and stop frames.

14. The machine-readable medium of claim 13, wherein said passing each of the plurality of frames through a filter bank comprises passing each of the plurality of frames through five Infinite Impulse Response (IIR) filters.

15. The machine-readable medium of claim 14, wherein the five IIR filters comprise a low pass filter, a band pass filter centered at approximately 450 Hz, a band pass filter centered at approximately 900 Hz, a band pass filter centered at approximately 1500 Hz, and a high pass filter.

16. The machine-readable medium of claim 13, comprising further instructions for performing steps comprising:

determining whether time elapsed between the start frame and the stop frame exceeds a minimum duration, and wherein;

said recording step is performed upon determining that the elapsed time exceeds the minimum duration.

17. The machine-readable medium of claim 13, comprising further instructions for performing steps comprising:

determining whether a time elapsed since the start frame exceeds a spectral flux threshold reset time; and

resetting the spectral flux threshold value upon determining the elapsed time exceeds the reset time, wherein;

said identifying a stop frame comprises identifying a frame of the plurality having a modified spectral flux value above the reset spectral flux threshold value.

18. The machine-readable medium of claim 13, comprising further instructions for performing steps comprising:

determining whether a time elapsed since the start frame exceeds a maximum duration, and wherein;

said identifying a stop frame and said recording steps are performed based on the elapsed time not exceeding the maximum duration.

19. The machine-readable medium of claim 13, comprising further instructions for performing steps comprising receiving a recording control signal initiated from a remotely located mobile terminal, and wherein:

said identifying a start frame comprises locating the start frame in a buffered portion of the audio stream, and

said recording comprises copying the portion of the audio stream received since the start frame into another memory.

20. The machine-readable medium of claim 13, wherein a modified spectral flux comprises a value of spectral flux averaged over K previous frames, where K is an integer.

21. A machine-readable medium having machine-executable instructions for performing steps comprising:

receiving a radio broadcast consisting essentially of an audio transmission;

calculating a value of a feature for each of a plurality of frames of the audio transmission;

identifying a start point, the start point being a frame in the audio transmission having a feature value bearing a first relation to a threshold value for the feature;

identifying a stop point, the stop point being a frame in the audio transmission having a feature value bearing a second relation to the threshold value for the feature; and

recording a portion of the audio transmission bounded by the start and stop points.

22. The machine-readable medium of claim 21, wherein the feature comprises a modified spectral flux, and comprising further instructions for performing steps comprising:

passing each of the plurality of frames through a filter bank, the filter bank including filters with bandwidths approximately proportional to their center frequencies, the filtered frames being used for calculation of a modified spectral flux value for at least a subset of the plurality of frames.

23. The machine-readable medium of claim 21, comprising further instructions for performing steps comprising receiving a recording control signal initiated from a remotely located mobile terminal, and wherein:

said identifying a start point comprises locating, within a buffered portion of the audio transmission, a frame having a feature value bearing the first relation to the threshold value, and

said recording comprises copying the portion of the audio transmission received since the start point into another memory.

24. A recording unit for recording broadcast programming, comprising:

a receiver for tuning to broadcast radio frequencies and receiving broadcast programming;

a memory having instructions stored therein; and

a processor coupled to the receiver and to the memory and configured execute the instructions so as to;

receive an audio stream having music and non-music portions,

segment the audio stream into successive frames,

pass each of a plurality of the frames through a filter bank, the filter bank including filters with bandwidths approximately proportional to their center frequencies,

compute a modified spectral flux value for at least a subset of the plurality of frames,

identify a start frame, the start frame being a frame of the plurality having a modified spectral flux value below a threshold value,

identify a stop frame, the stop frame being a frame of the plurality having a modified spectral flux value above the threshold value, and

record a portion of the audio stream bounded by the start and stop frames.

25. The recording unit of claim 24, wherein said passing each of the plurality of frames through a filter bank comprises passing each of the plurality of frames through five Infinite Impulse Response (IIR) filters.

26. The recording unit of claim 25, wherein the five IIR filters comprise a low pass filter, a band pass filter centered at approximately 450 Hz, a band pass filter centered at approximately 900 Hz, a band pass filter centered at approximately 1500 Hz, and a high pass filter.

27. The recording unit of claim 24, wherein the processor is further configured to execute the instructions so as to:

determine whether time elapsed between the start frame and the stop frame exceeds a minimum duration, and wherein;

said recording step is performed upon determining that the elapsed time exceeds the minimum duration.

28. The recording unit of claim 24, wherein the processor is further configured to execute the instructions so as to:

determine whether time elapsed since the start frame exceeds a spectral flux threshold reset time, and

reset the spectral flux threshold value upon determining the elapsed time exceeds the reset time, and wherein;

said identifying a stop frame comprises identifying a frame of the plurality having a modified spectral flux value above the reset spectral flux threshold value.

29. The recording unit of claim 24, wherein the processor is further configured to execute the instructions so as to:

determine whether time elapsed since the start frame exceeds a maximum duration, and wherein;

said identifying a stop frame and said recording steps are performed based on the elapsed time not exceeding the maximum duration.

30. The recording unit of claim 24, wherein the recording unit is a remote recording unit, wherein the processor is further configured to execute the instructions so as to receive a recording control signal initiated from a remotely located mobile terminal, and wherein:

said identifying a start frame comprises locating the start frame in a buffered portion of the audio stream, and

said recording comprises copying the portion of the audio stream received since the start frame into another memory.

31. A recording unit for remotely recording broadcast programming, comprising:

a receiver for tuning to broadcast radio frequencies and receiving broadcast programming;

a memory having instructions stored therein; and

a processor coupled to the receiver and to the memory and configured execute the instructions so as to;

receive a radio broadcast consisting essentially of an audio transmission,

calculate a value of a feature for each of a plurality of frames of the audio transmission,

identify a start point in the audio transmission, the start point being a frame in the audio transmission having a feature value bearing a first relation to a threshold value for the feature,

identify a stop point in the audio transmission, the stop point being a frame in the audio transmission having a feature value bearing a second relation to the threshold value for the feature, and

record a portion of the audio transmission bounded by the start and stop points.

32. The recording unit of claim 31, wherein the feature comprises a modified spectral flux, and wherein the processor is further configured to execute the instructions so as to:

pass each of the plurality of frames through a filter bank, the filter bank including filters with bandwidths approximately proportional to their center frequencies, the filtered frames being used for calculation of a modified spectral flux value for at least a subset of the plurality of frames.

33. The recording unit of claim 31, wherein the recording unit is a remote recording unit, wherein the processor is further configured to execute the instructions so as to receive a recording control signal initiated from a remotely located mobile terminal, and wherein:

said identifying a start point comprises locating, within a buffered portion of the audio transmission, a start point having a feature value bearing the first relation to the threshold value, and

said recording comprises copying the portion of the audio transmission received since the start point into another memory.

34. A recording unit for remotely recording broadcast programming, comprising:

a receiver for tuning to broadcast radio frequencies and receiving broadcast programming;

a buffer memory;

a storage memory having instructions stored therein;

a network interface; and

a processor coupled to the receiver, to the network interface and to the memories and configured execute the instructions so as to;

receive an audio stream having music and non-music portions,

segment the audio stream into successive frames,

pass each of a plurality of the frames through a low pass Infinite Impulse Response (IIR) filter, a band pass IIR filter centered at approximately 450 Hz, a band pass IIR filter centered at approximately 900 Hz, a band pass IIR filter centered at approximately 1500 Hz, and a high pass IIR filter,

compute a modified spectral flux value for each of the plurality of frames based on the output of the filters,

receive, via the network interface, a recording control signal initiated from a remotely located mobile terminal,

upon receipt of the recording control signal, identify a start frame in a portion of the audio stream stored in the buffer memory, the start frame being a frame of the plurality having a modified spectral flux value below a threshold value,

identify a stop frame, the stop frame being a frame of the plurality having a modified spectral flux value above the threshold value,

upon determining that the time elapsed between the start and stop frames exceeds a minimum value, store in the storage memory the part of the audio stream bounded by the start and stop frames, said storing including copying from the buffer memory a part of the audio stream buffered after the start frame.