MAPPING BETWEEN SPEECH SIGNAL AND TRANSCRIPT

• US 20190108831A1
• Filed: 10/10/2017
• Published: 04/11/2019
• Est. Priority Date: 10/10/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: FUKUDA, TAKASHI, ITOH, NOBUYASU
• Granted Patent: US 10,650,803 B2
• Time in Patent Office: Days
• CPC Class Codes: G10L 15/06 Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice G10L15/14 takes precedence
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10

Abstract

A method, a computer program product, and a computer system for mapping between a speech signal and a transcript of the speech signal. The computer system segments the speech signal to obtain one or more segmented speech signals and the transcript of the speech signal to obtain one or more segmented transcripts of the speech signal. The computer system generates estimated phone sequences and reference phone sequences, calculates costs of correspondences between the estimated phone sequences and the reference phone sequences, determines a series of the estimated phone sequences with a smallest cost, selects a partial series of the estimated phone sequences from the series of the estimated phone sequences, and generates mapping data which includes the partial series of the estimated phone sequences and a corresponding series of the reference phone sequences.

Claims

1. A computer-implemented method for mapping between a speech signal and a transcript of the speech signal, the method comprising:

obtaining the speech signal and the transcript of the speech signal;

segmenting the speech signal to obtain one or more segmented speech signals;

segmenting the transcript of the speech signal to obtain one or more segmented transcripts of the speech signal;

performing automatic speech recognition of the one or more segmented speech signals to obtain recognized texts;

converting the recognized texts into estimated phone sequences;

converting the one or more segmented transcripts of the speech signal into reference phone sequences;

calculating costs of correspondences between the estimated phone sequences and the reference phone sequences;

determining a series of the estimated phone sequences, the series of the estimated phone sequences being with a smallest cost;

selecting a partial series of the estimated phone sequences, from the series of the estimated phone sequences; and

generating mapping data which includes the partial series of the estimated phone sequences and a corresponding series of the reference phone sequences, wherein the corresponding series corresponds to the partial series of the estimated phone sequences.

2. The computer-implemented method of claim 1, further comprising:

training an acoustic model, using the mapping data.

3. The computer-implemented method of claim 1, wherein the segmenting the speech signal includes using voice activity detection (VAD).

4. The computer-implemented method of claim 3, wherein the segmenting the speech signal to obtain one or more segmented speech signals includes using the VAD with different parameter values.

5. The computer-implemented method of claim 1, wherein the segmenting the transcript of the speech signal includes segmenting the transcript by referring to punctuations.

6. The computer-implemented method of claim 1, wherein the costs are distances between the estimated phone sequences and the reference phone sequences.

7. A computer program product for mapping between a speech signal and a transcript of the speech signal, the computer program product comprising a computer readable storage medium having program code embodied therewith, the program code executable to:

obtain the speech signal and the transcript of the speech signal;

segment the speech signal to obtain one or more segmented speech signals;

segment the transcript of the speech signal to obtain one or more segmented transcripts of the speech signal;

perform automatic speech recognition of the one or more segmented speech signals to obtain recognized texts;

convert the recognized texts into estimated phone sequences;

convert the one or more segmented transcripts of the speech signal into reference phone sequences;

calculate costs of correspondences between the estimated phone sequences and the reference phone sequences;

determine a series of the estimated phone sequences, the series of the estimated phone sequences being with a smallest cost;

select a partial series of the estimated phone sequences, from the series of the estimated phone sequences; and

generate mapping data which includes the partial series of the estimated phone sequences and a corresponding series of the reference phone sequences, wherein the corresponding series corresponds to the partial series of the estimated phone sequences.

8. The computer program product of claim 7, further comprising the program code executable to:

train an acoustic model, using the mapping data.

9. The computer program product of claim 7, wherein to segment the speech signal includes using voice activity detection (VAD).

10. The computer program product of claim 9, wherein to segment the speech signal to obtain one or more segmented speech signals includes using the VAD with different parameter values.

11. The computer program product of claim 7, wherein to segment the transcript of the speech signal includes segmenting the transcript by referring to punctuations.

12. The computer program product of claim 7, wherein the costs are distances between the estimated phone sequences and the reference phone sequences.

13. A computer system for mapping between a speech signal and a transcript of the speech signal, the computer system comprising:

one or more processors, one or more computer readable tangible storage devices, and program instructions stored on at least one of the one or more computer readable tangible storage devices for execution by at least one of the one or more processors, the program instructions executable to;

obtain the speech signal and the transcript of the speech signal;

segment the speech signal to obtain one or more segmented speech signals;

segment the transcript of the speech signal to obtain one or more segmented transcripts of the speech signal;

perform automatic speech recognition of the one or more segmented speech signals to obtain recognized texts;

convert the recognized texts into estimated phone sequences;

convert the one or more segmented transcripts of the speech signal into reference phone sequences;

calculate costs of correspondences between the estimated phone sequences and the reference phone sequences;

determine a series of the estimated phone sequences, the series of the estimated phone sequences being with a smallest cost;

select a partial series of the estimated phone sequences, from the series of the estimated phone sequences; and

generate mapping data which includes the partial series of the estimated phone sequences and a corresponding series of the reference phone sequences, wherein the corresponding series corresponds to the partial series of the estimated phone sequences.

14. The computer system of claim 13, further comprising the program instructions executable to:

train an acoustic model, using the mapping data.

15. The computer system of claim 13, wherein to segment the speech signal includes using voice activity detection (VAD).

16. The computer system of claim 15, wherein to segment the speech signal to obtain one or more segmented speech signals includes using the VAD with different parameter values.

17. The computer system of claim 13, wherein to segment the transcript of the speech signal includes segmenting the transcript by referring to punctuations

18. The computer system of claim 13, wherein the costs are distances between the estimated phone sequences and the reference phone sequences.