System and method for classification of audio or audio/video signals based on musical content

• US 5,712,953 A
• Filed: 06/28/1995
• Issued: 01/27/1998
• Est. Priority Date: 06/28/1995
• Status: Expired due to Term

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Electronic Data Systems Corporation (Perspecta, Inc.)
• Inventors: Langs, Steven E.
• Primary Examiner(s): MacDonald, Allen R.
• Assistant Examiner(s): Chawan, Vijay B.
• Application Number:
  US08/508,519
• Time in Patent Office: 944 Days
• Field of Search: 395/2.23, 395/2.24, 395/2.17, 395/2.19
• US Class Current: 704/214
• CPC Class Codes: G10H 2210/046 for differentiation between music and non-music signals, based on the identification of musical parameters, e.g. based on tempo detection
• CPC Class Codes: G10H 2250/245 Hartley transform,; Discrete Hartley transform [DHT]; Fast Hartley transform [FHT]
• CPC Class Codes: G10H 2250/265 Blackman Harris window
• CPC Class Codes: G10H 2250/281 Hamming window
• CPC Class Codes: G10H 3/125 Extracting or recognising the pitch or fundamental frequency of the picked up signal
• CPC Class Codes: G10L 2025/783 based on threshold decision
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10

Abstract

An automated system and method for classifying audio or audio/video signals as music or non-music is provided. A spectrum module receives at least one digitized audio signal from a source and generates representations of the power distribution of the audio signal with respect to frequency and time. A first moment module calculates, for each time instant, a first moment of the distribution representation with respect to frequency and in turn generates a representation of a time series of first moment values.

A degree of variation module in turn calculates a measure of degree of variation with respect to time of the values of the time series and produces a representation of the first moment time series variation measuring values. Lastly, a module classifies the representation by detecting patterns of low variation, which correspond to the presence of musical content in the original digitized audio signal, and patterns of high variation, which correspond to the absence of musical content in the original digitized audio signal.

Claims

1. An automated processing system for classifying audio signals as music or non-music, comprising:

a source of at least one digitized audio signal;

a spectrum module for receiving said at least one digitized audio signal and for generating representations of spectral power distribution with respect to frequency and time of said audio signal;

a first moment module for receiving said generated representations from said spectrum module, for calculating for each time instant first moment of said distribution representation with respect to frequency, and for generating a representation of time series of first moment values;

a degree of variation module for receiving said representation of time series of first moment values from said first moment module, for calculating a measure of degree of variation with respect to time of said values of said time series, thereby producing a representation of first moment time series variation measuring values; and

a module for receiving said representation of said first moment time series variation measuring values and for classifying said received representation by detecting patterns of low variation, which correspond to the presence of musical content in said at least one digitized audio signal, and patterns of high variation, which correspond to the absence of musical content in said at least one digitized audio signal.

2. The automated processing system of claim 1, wherein said audio signals are audio signals which have been separated for automated processing from audio/video signals.

3. The automated processing system of claim 1, wherein said spectrum module further comprises a window module for receiving said at least one digitized audio signal, for extracting sample vectors from said signal, and for multiplying said sample vectors with a sampled window function before generating said representations of power distribution with respect to frequency and time of said audio signal.

4. The automated processing system of claim 1, wherein said spectrum module further comprises a floor module for attenuating to zero all values of said generated representations of power distribution with respect to frequency and time which are less than a floor value before they are provided to said first moment module.

5. The automated processing system of claim 1, wherein said degree of variation module further comprises a moving average module for receiving said representation of said first moment time series variation measuring values, calculating a moving average of said variation measuring values, before providing same to said module for receiving said representation of said first moment time series variation measuring values and for classifying said received representation.

6. The automated processing system of claim 1, wherein said measure of degree of variation with respect to time of said values of said time series is the second derivative of said time series of first moment values.

7. The automated processing system of claim 1, wherein said module for classifying said received representation further comprises a threshold module for thresholding said time series of variation measuring values, for producing a time series of logical values indicating whether said variation measuring values exceeded a predetermined threshold, before detecting patterns of said time series of logical values which correspond to presence or absence of musical content in said at least one digitized audio signal.

8. The automated processing system of claim 7, wherein said module for classifying said received representation further comprises a voting module for counting the number of each type of said logical values received, and for classifying said at least one digitized audio signal according to a state variable which holds said voting module'"'"'s current evaluation of the presence or absence of musical content, wherein said state variable is changed to an opposite evaluation by a preponderance of logical values opposing said current evaluation having occurred since a previous state change, and wherein a level preponderance required for a state change is established by a predetermined time-varying threshold level.

9. The automated processing system of claim 1, further comprising an application for receiving output from said module for classifying said received representation by detecting patterns, and for indexing said at least one digitized audio signal based on said output.

10. The automated processing system of claim 1, further comprising applications for receiving output from said module for classifying said received representation by detecting, and for filtering said at least one digitized audio signal based on said output.

11. The automated processing system of claim 1, further comprising applications for receiving output from said module for classifying said received representation by detecting, and for managing said at least one digitized audio signal based on said output.

12. An automated method for classifying audio or audio/video signals as music or non-music, comprising the steps of:

a. receiving at least one digitized audio signal;

b. generating representations of spectral power distribution with respect to frequency and time of said audio signal;

c. calculating for each time instant first moment of said distribution representation with respect to frequency, and for generating a representation of time series of first moment values;

d. calculating a measure of degree of variation with respect to time of said values of said time series, thereby producing a representation of first moment time series variation measuring values; and

e. classifying said received representation by detecting patterns of low variation, which correspond to the presence of musical content in said at least one digitized audio signal, and patterns of high variation, which correspond to the absence of musical content in said at least one digitized audio signal.

13. The automated method for classifying of claim 12, wherein said audio signals are audio signals which have been separated for automated processing from audio/video signals.

14. The automated method for classifying of claim 12, after said step of receiving said at least one digitized audio signal and before said step of generating said representations of power distribution with respect to frequency and time of said audio signal, further comprising the steps of:

extracting sample vectors from said signal; and

multiplying said sample vectors with a sampled window function.

15. The automated method for classifying of claim 12, further comprising the step of attenuating to zero all values of said generated representations of power distribution with respect to frequency and time which are less than a floor value before said step of calculating for each time instant first moment of said distribution representation.

16. The automated method for classifying of claim 12, further comprising the step of calculating a moving average of said variation measuring values before said step of classifying.

17. The automated method for classifying of claim 12, further comprising the step of calculating the second derivative of said time series of first moment values as said measure of degree of variation with respect to time of the values of said time series to thereby produce said representation of first moment time series variation measuring values.

18. The automated method for classifying of claim 12, wherein said step of classifying further comprises the step of thresholding said time series of variation measuring values, for producing a time series of logical values indicating whether said variation measuring values exceeded a predetermined threshold, before detecting patterns of said time series of logical values which correspond to presence or absence of musical content in said at least one digitized audio signal.

19. The automated method for classifying of claim 18, wherein said step of classifying further comprises the steps of:

counting the number of each type of said logical values received; and

classifying said at least one digitized audio signal according to a state variable which holds a current evaluation of the presence or absence of musical content, wherein said state variable is changed to an opposite evaluation by a preponderance of logical values opposing said current evaluation having occurred since a previous state change, and wherein a level preponderance required for a state change is determined by a predetermined time-varying threshold level.