Method for producing reference segments describing voice modules and method for modeling voice units of a spoken test model

• US 7,398,208 B2
• Filed: 10/01/2002
• Issued: 07/08/2008
• Est. Priority Date: 10/11/2001
• Status: Active Grant

• Current Assignee: Cerence Operating Company (Cerence Inc.)
• Original Assignee: Siemens AG
• Inventors: Kämmerer, Bernhard
• Primary Examiner(s): Lerner; Martin
• Application Number:
  US10/492,302
• Publication Number:
  US 20040249639A1
• Time in Patent Office: 2,107 Days
• Field of Search: 704/236, 704/238, 704/241, 704/243, 704/245, 704/246, 704/249, 704/250, 704/251, 704/254, 704/255, 704/256, 704/256.1, 704/256.2, 704/256.4
• US Class Current: 704/241
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/12 using dynamic programming techniques, e.g. dynamic time warping [DTW]
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units
• CPC Class Codes: G10L 2015/0631 Creating reference templates; Clustering

Abstract

A method models voice units and produces reference segments for modeling voice units. The reference segments describe voice modules by characteristic vectors, the characteristic vectors being stored in the order in which they are found in a training voice signal. Alternative characteristic vectors are associated with each characteristic vector. The reference segments for describing the voice modules are combined during the modeling of larger voice units. In the event of identification, the respectively best adapted characteristic vector alternatives are used to determined the distance between a test utterance and the larger vocal units.

Claims

1. A method for producing reference segments describing speech modules, for a voice recognition system, comprising:

phonetically segmenting a spoken training voice signal into speech modules in accordance with a predefined transcription;

subdividing each speech module into a sequence of time windows;

analyzing the spoken training voice signal in each time window to obtain a characteristic vector for each time window and obtain a training model from a sequence of characteristic vectors corresponding to the sequence of time windows, each speech module having a plurality of training models corresponding to a plurality of different pronunciations for the speech module;

forming an average time structure for each speech module, the average time structure being formed by comparing the plurality of training modules for the speech module, the average time structure containing information regarding an average pronunciation speed and style, the average time structure having a plurality of time windows, the average time structure being formed by mapping the characteristic vectors of the different training models onto the time windows of the average time structure such that each time window of the average time structure contains a plurality of characteristic vectors, the characteristic vectors being mapped using a non-linear mapping; and

saving the plurality of time windows for the average time structure as a reference segment.

2. A method for producing reference segments for a voice recognition system, comprising:

phonetically segmenting a training voice signal into speech modules in accordance with a predefined transcription;

analyzing the training voice signal in predetermined time windows in order to obtain at least one characteristic vector for each time window, as a result of which training models are formed which in each case contain characteristic vectors in the time sequence of the training voice signal;

determining an average time structure, which is an average of change duration and time sequence characteristics, for each speech module;

assigning the characteristic vectors to the average time structure by a temporally non-linear mapping to produce a reference segment; and

storing the reference segment.

3. The method according to claim 2, wherein

the training voice signal is segmented into speech modules to separate phonemes, diphthongs, diphones, triphones or syllables.

4. The method according to claim 2, wherein

the characteristic vectors of the training models represent spectral characteristics, autocorrelation characteristics, LPC characteristics, MFCC characteristics or CC characteristics.

5. The method according to claim 2, wherein

the average time sequence is obtained by performing non-linear mappings of the training models on the speech module to one another and by averaging the mappings.

6. The method according to claim 2, further comprising clustering the characteristic vectors of the time windows.

7. The method according to claim 6, wherein

the number of characteristic vectors per time window are limited to a particular number.

8. The method according to claim 3, wherein

the characteristic vectors of the training models represent spectral characteristics, autocorrelation characteristics, LPC characteristics, MFCC characteristics or CC characteristics.

9. The method according to claim 8, wherein

the average time sequence is obtained by performing non-linear mappings of the training models on the speech module to one another and by averaging the mappings.

10. The method according to claim 9, further comprising clustering the characteristic vectors of the time windows.

11. The method according to claim 10, wherein

the number of characteristic vectors per time window are limited to a particular number.

12. The method according to claim 11, wherein

the number of characteristic vectors corresponds to a variance in the characteristic vectors for the training models, such that if there is a greater variance, more characteristic vectors are used.

13. A method for producing reference segments for a voice recognition system, comprising:

segmenting a training voice signal into speech modules in accordance with a predefined transcription;

analyzing the training voice signal in predetermined time windows in order to obtain at least one characteristic vector for each time window, as a result of which training models are formed which in each case contain characteristic vectors in the time sequence of the training voice signal;

determining an average time structure, which is an average of change duration and time sequence characteristics, for each speech module;

assigning the characteristic vectors to the average time structure by a temporally non-linear mapping to produce a reference segment;

storing the reference segment; and

clustering the characteristic vectors of the time windows, wherein

the number of characteristic vectors corresponds to a variance in the characteristic vectors for the training models, such that if there is a greater variance, more characteristic vectors are used.

14. A method for modeling speech units of a spoken test model in a voice recognition system, comprising:

producing reference segments describing speech modules for a voice recognition system, comprising;

phonetically segmenting a spoken training voice signal into speech modules in accordance with a predefined transcription;

subdividing each speech module into a sequence of time windows;

analyzing the spoken training voice signal in each time window to obtain a characteristic vector for each time window and obtain a training model from a sequence of characteristic vectors corresponding to the sequence of time windows, each speech module having a plurality of training models corresponding to a plurality of different pronunciations for the speech module;

forming an average time structure for each speech module, the average time structure being formed by comparing the plurality of training modules for the speech module, the average time structure containing information regarding an average pronunciation speed and style, the average time structure having a plurality of time windows, the average time structure being formed by mapping the characteristic vectors of the different training models onto the time windows of the average time structure such that each time window of the average time structure contains a plurality of characteristic vectors, the characteristic vectors being mapped using a non-linear mapping; and

saving the plurality of time windows for the average time structure as a reference segment;

forming a plurality of reference models, each reference model being formed by combining a plurality of reference segments, each reference model representing a speech unit;

performing a non-linear comparison of the reference models with the test model and determining in each case a distance between the reference model and the test model; and

selecting the reference model having the smallest distance from the test model, whereby the speech unit represented by the reference segments is assigned to the test model.

15. The method according to claim 14, wherein

each reference model represents a word to be recognized.

16. The method according to claim 15, wherein

each reference model is formed from a concatenation of the reference segments in accordance with the transcription.

17. The method according to claim 16, wherein

the non-linear comparison is effected by a non-linear time adjustment of the test model to the reference models for the words to be recognized.

18. The method according to claim 17, wherein

the non-linear time adjustment is restricted to a defined working range.

19. The method according to claim 18, wherein

each reference segment has a characteristic vector,

the test model has a characteristic vector,

in performing the non-linear comparison, a distance is determined between the characteristic vector of the test model and each of the characteristic vectors of the reference segment, and

the distance is determined to be the minimum of the distances between the characteristic vector of the test model and the characteristic vectors of the reference segments.

20. The method according to claim 19, wherein

distortion is limited in the non-linear mapping.

21. The method according to claim 14, wherein

the non-linear comparison is effected by a non-linear time adjustment of the test model to the reference models for the words to be recognized.

22. The method according to claim 21, wherein

the non-linear time adjustment is restricted to a defined working range.

23. The method according to claim 14, wherein

each reference segment has a characteristic vector,

the test model has a characteristic vector,

in performing the non-linear comparison, a distance is determined between the characteristic vector of the test model and each of the characteristic vectors of the reference segment, and

the distance is determined to be the minimum of the distances between the characteristic vector of the test model and the characteristic vectors of the reference segments.

24. The method according to claim 14, wherein

distortion is limited in the non-linear mapping.