Apparatus and method for classifying an audio signal

• US 20050131688A1
• Filed: 11/10/2004
• Published: 06/16/2005
• Est. Priority Date: 11/12/2003
• Status: Abandoned Application

• Current Assignee: Sony Deutschland GmbH (Sony Group Corp.)
• Original Assignee: Sony Deutschland GmbH (Sony Group Corp.)
• Inventors: Kompe, Ralf, Kemp, Thomas, Marasek, Krzysztof, Lam, Yin Hay, Tato, Raquel, Goronzy, Silke
• Application Number:
  US10/985,295
• Publication Number:
  US 20050131688A1
• Time in Patent Office: Days
• US Class Current: 704/240
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10
• CPC Class Codes: G11B 2220/20 Disc-shaped record carriers

Abstract

An apparatus for classifying audio signals comprises audio signal clipping means for partitioning audio signals into audio clips, and class discrimination means for discriminating the audio clips provided by the audio signal clipping means into predetermined audio classes based on predetermined audio class classifying rules, by analysing acoustic characteristics of the audio signals comprised in the audio clips, wherein a predetermined audio class classifying rule is provided for each audio class, and each audio class represents a respective kind of audio signals comprised in the corresponding audio clip. The determination process to find acceptable audio class classifying rules for each audio class according to the prior art is depending on both the used raw audio signals and the personal experience of the person conducting the determination process. Thus, the determination process usually is very difficult, time consuming and subjective. Furthermore, there is a high risk that not all possible peculiarities of the different programmes and the different categories the audio signal can belong to is sufficiently accounted for. This problem is solved in the inventive apparatus for classifying audio signals by class discrimination means calculating an audio class confidence value for each audio class assigned to an audio clip, wherein the audio class confidence value indicates the likelihood the respective audio class characterises the respective kind of audio signals comprised in the respective audio clip correctly. Furthermore, the class discrimination means use acoustic characteristics of audio clips of audio classes having a high audio class confidence value to train the respective audio class classifying rule.

Claims

1. Apparatus for classifying audio signals comprising:

audio signal clipping means for partitioning audio signals into audio clips; and

class discrimination means for discriminating the audio clips provided by the audio signal clipping means into predetermined audio classes based on predetermined audio class classifying rules by analysing acoustic characteristics of the audio signals comprised in the audio clips, wherein a predetermined audio class classifying rule is provided for each audio class, and each audio class represents a respective kind of audio signals comprised in the corresponding audio clip;

characterised in that

the class discrimination means calculates an audio class confidence value for each audio class assigned to an audio clip, wherein the audio class confidence value indicates the likelihood the respective audio class characterises the respective kind of audio signals comprised in the respective audio clip correctly; and

the class discrimination means uses acoustic characteristics of audio clips of audio classes having a high audio class confidence value to train the respective audio class classifying rule.

2. Apparatus for classifying audio signals according to claim 1,

characterised in that the classifying apparatus further comprises

segmentation means for segmenting classified audio signals into individual sequences of cohesive audio clips based on predetermined content classifying rules by analysing a sequence of audio classes of cohesive audio clips provided by the class discrimination means, wherein each sequence of cohesive audio clips segmented by the segmentation means corresponds to a content included in the audio signals;

wherein

the segmentation means calculates a content confidence value for each content assigned to a sequence of cohesive audio clips, wherein the content confidence value indicates the likelihood the respective content characterises the respective sequence of cohesive audio clips correctly; and

the segmentation means uses sequences of cohesive audio clips having a high content confidence value to train the respective content classifying rule.

3. Apparatus for classifying audio signals according to claim 1,

characterised in that

the classifying rules comprise Neuronal Networks;

and

weights used in the Neuronal Networks are updated to train the Neuronal Networks.

4. Apparatus for classifying audio signals according to claim 1,

characterised in that

the classifying rules comprise Gaussian Mixture Models;

and

parameters for maximum likelihood linear regression transformation and/or Maximum a Posteriori used in the Gaussian Mixture Models are adjusted to train the Gaussian Mixture Models.

5. Apparatus for classifying audio signals according to claim 1,

characterised in that

the classifying rules comprise decision trees;

and

questions related to event duration at each leaf node used in the decision trees are adjusted to train the decision trees.

6. Apparatus for classifying audio signals according to claim 1,

characterised in that

the classifying rules comprise hidden Markov models;

and

prior probabilities of a particular audio class given a number of last audio classes and/or transition probabilities used in the hidden Markov models are adjusted to train the hidden Markov models.

7. Apparatus for classifying audio signals according to claim 1,

characterised in that the classifying apparatus further comprises:

first user input means for manual segmentation of the audio signals into individual sequences of cohesive audio clips and manual allocation of a corresponding content;

wherein the segmentation means uses manually segmented audio signals to train the respective content classifying rules.

8. Apparatus for classifying audio signals according to claim 1,

characterised in that the classifying apparatus further comprises:

second user input means for manual discrimination of the audio clips into corresponding audio classes;

wherein the class discrimination means uses said manually discriminated audio clips to train the respective audio class classifying rules.

9. Apparatus for classifying audio signals according to claim 1,

characterised in that

the acoustic characteristics comprise bandwidth and/or zero cross rate and/or volume and/or sub-band energy rate and/or mel-cepstral components and/or frequency centroid and/or subband energies and/or pitch period of the respective audio signals.

10. Apparatus for classifying audio signals according to claim 1,

characterised in that

a predetermined audio class classifying rule is provided for each silence, speech, music, cheering and clapping.

11. Apparatus for classifying audio signals according to claim 1,

characterised in that

the audio signals are part of a video data file, the video data file being composed of at least an audio signal and a picture signal.

12. Apparatus for classifying audio signals according to claim 1,

characterised in that

the segmentation means identifies a sequence of commercials in the audio signals by analysing the contents of the audio signals and uses a sequence of cohesive audio clips preceding and/or following the sequence of commercials to train the respective content classifying rules.

13. Method for classifying audio signals comprising the following steps:

partitioning audio signals into audio clips; and

discriminating the audio clips into predetermined audio classes based on predetermined audio class classifying rules by analysing acoustic characteristics of the audio signals comprised in the audio clips, wherein a predetermined audio class classifying rule is provided for each audio class and each audio class represents a respective kind of audio signals comprised in the corresponding audio clip;

characterised in that the method further comprises the steps of;

calculating an audio class confidence value for each audio class assigned to an audio clip, wherein the audio class confidence value indicates the likelihood the respective audio class characterises the respective kind of audio signals comprised in the respective audio clip correctly; and

using acoustic characteristics of audio clips of audio classes having a high audio class confidence value to train the respective audio class classifying rules.

14. Method for classifying audio signals according to claim 13,

characterised in that the method further comprises the steps of:

segmenting the classified audio signals into individual sequences of cohesive audio clips based on predetermined content classifying rules by analysing a sequence of audio classes of cohesive audio clips, wherein each sequence of cohesive audio clips corresponds to a content included in the audio signals;

calculating a content confidence value for each content assigned to a sequence of cohesive audio clips, wherein the content confidence value indicates the likelihood, the respective content characterises the respective sequence of cohesive audio clips correctly; and

using sequences of cohesive audio clips having a high content confidence value to train the respective content classifying rules.

15. Method for classifying audio signals according to claim 13,

characterised in that the method further comprises the steps of:

using Neuronal Networks as classifying rules; and

updating weights used in the Neuronal Networks to train the Neuronal Networks.

16. Method for classifying audio signals according to claim 13,

characterised in that the method further comprises the steps of:

using Gaussian Mixture Models as classifying rules; and

adapting parameters for maximum likelihood linear regression transformation and/or Maximum a Posteriori used in the Gaussian Mixture Models to train the Gaussian Mixture Models.

17. Method for classifying audio signals according to claim 13,

characterised in that the method further comprises the steps of:

using decision trees as classifying rules; and

adapting questions related to event duration at each leaf node used in the decision trees to train the decision trees.

18. Method for classifying audio signals according to claim 13,

characterised in that the method further comprises the steps of:

using hidden Markov models as classifying rules; and

adapting prior probabilities of a particular audio class given a number of last audio classes and/or transition probabilities used in the hidden Markov models to train the hidden Markov models.

19. Method for classifying audio signals according to claim 13,

characterised in that the method further comprises the step of:

using audio signals which are segmented manually into individual sequences of cohesive audio clips and allocated manually to a corresponding content to train the respective content classifying rules.

20. Method for classifying audio signals according to claim 13,

characterised in that the method further comprises the step of:

using audio clips which are discriminated manually into corresponding audio classes to train the respective audio class classifying rules.

21. Method for classifying audio signals according to claim 13,

characterised in that the method further comprises the steps of:

identifying a sequence of commercials in the audio signals by analysing the contents of the audio signals; and

using a sequence of cohesive audio clips preceding and/or following the sequence of commercials to train the respective content classifying rules.

22. Software product comprising a series of state elements which are adapted to be processed by a data processing means of a mobile terminal such, that a method according to claim 13 may be executed thereon.