METHOD FOR GENERATING CLOSED CAPTIONS

• US 20070118374A1
• Filed: 10/25/2006
• Published: 05/24/2007
• Est. Priority Date: 11/23/2005
• Status: Abandoned Application

• Current Assignee: General Electric Company
• Original Assignee: General Electric Company
• Inventors: Lizzi, John, Zinser, Richard, Goldfarb, Helena, Chai, Wei, Hoebel, Louis, Wise, Gerald, Abraham, Anil
• Application Number:
  US11/552,533
• Publication Number:
  US 20070118374A1
• Time in Patent Office: Days
• US Class Current: 704/235
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 21/06 Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids G10L15/26 takes precedence

Abstract

A method for detecting and modifying breath pauses in a speech input signal includes detecting breath pauses in a speech input signal; modifying the breath pauses by replacing the breath pauses with a predetermined input and/or attenuating the breath pauses; and outputting an output speech signal. A computer program for carrying out the method is also presented.

Claims

1. A method for detecting and modifying breath pauses in a speech input signal, the method comprising:

detecting breath pauses in a speech input signal;

modifying the breath pauses by replacing the breath pauses with a predetermined input and/or attenuating the breath pauses; and

outputting an output speech signal.

2. The method of claim 1, wherein the predetermined input is at least one of uniform noise and Gaussian noise and wherein detecting breath pauses comprises determining at least one of a normalized zero crossing count, a relative root-mean-square signal level, and one or more spectral power ratios.

3. The method of claim 2, wherein detecting breath pauses further comprises determining each of the normalized zero crossing count, the relative root-mean-square signal level, the spectral power ratio(s) and a non-linear combination of each of the normalized zero crossing count, the relative root-mean-square signal level and the one or more spectral power ratios.

4. The method of claim 3, wherein detecting breath pauses further comprises determining a contribution of +1, 0, −

1 or −

2 for each of the normalized zero crossing count, the relative root-mean-square signal level, the one or more spectral power ratios and the non-linear combination and wherein detecting breath pauses further comprises determining a pscore by combining each the contributions for each of the normalized zero crossing count, the relative root-mean-square signal level, the one or more spectral power ratios and the non-linear combination.

5. The method of claim 4, wherein detecting breath pauses further comprises determining the pscore over a predetermined number of audio frames and wherein detecting breath pauses still further comprises summing each pscore for each particular frame over the predetermined number of audio frames to determine a composite detection score.

6. The method of claim 5, wherein the composite detection score is determined for each of the normalized zero crossing count (NZCC), the relative root-mean-square (RRMS) signal level, the spectral power ratio and the non-linear combination based on the below:

NZCC;

if (0.09 <

ZCN <

0.22) pscore++;

RRMS;

if (RRMS <

0.085) pscore++;

else if (RRMS >

0.1) pscore−

−

;

Spectral if (lo-hi <

     5) AND (hiwide-lowide >

−

250) pscore−

−

;

Ratios;

if (lo-hi <

−

50) pscore−

−

;

if (lo-mid >

     200) AND (lo-hi <

     120) pscore−

−

;

if (hiwide-lowide >

−

100) pscore −

= 2;

Non-linear if the NZCC and RRMS criteria had positive contributions, Comb;

and the spectral ratio net contribution was zero, pscore++;

where;

ZCN is found by dividing a number of times a signal changes polarity within a frame by a length of the frame in samples; and

RRMS is found using the logic;

if (rms>

prms) prms=rms;

prms*=DECAY_COEFF;

rrms=rms/prms;

where rms is the current frame'"'"'s RMS value, PRMS is the peak-hold AR average RMS, DECAY_COEFF is a positive number less than 1.0.

7. The method of claim 6, wherein:

the method further comprises high pass filtering the speech input signal prior to detecting breath pauses; and

wherein determining spectral ratios comprise using a 4-term Blackman-Harris window, a 1024-point FFT, and N filter ratio calculators, where N=a predetermined number of spectral power ratios.

8. The method of claim 1, wherein detecting breath pauses further comprises detecting plosives.

9. The method of claim 8, wherein detecting plosives comprises either determining:

(rms_half2/rms_half1>

5) OR (rms_current_frame/rms_last_frame>

5);

(NZCC has positive pscore contribution) OR (the composite detection score>

3); and

(the composite detection score<

20);

or

determining;

(current_frame_pscore<

0) AND (composite detection score<

20); and

(the composite detection score>

=9) OR (last_frame_pscore>

=3).

10. The method of claim 1, wherein modifying the breath pauses comprises selecting one of four modes, the modes selectable comprise no alteration of the speech input signal;

attenuation of the speech input signal;

the replacement of a breath pause with Gaussian noise; and

the replacement of a breath pause with uniform noise.

11. The method of claim 1, wherein modifying breath pauses comprises extending a breath pause.

12. The method of claim 1, further comprising detecting zero-valued samples in a processed segment output from the breath detection unit.

13. The method of claim 12, wherein detecting zero-valued samples comprises counting a number of zero-valued samples and comparing the number to a predetermined threshold, where the number of zero-valued samples is above the threshold, further comprising adding uniform or Gaussian noise to the output speech signal.

14. The method of claim 1 being employed with a method for generating closed captions from an audio signal, the method for generating closed captions comprising:

correcting one additional predetermined undesirable attribute from the audio signal and outputting one or more speech segments;

generating from the one or more speech segments one or more text transcripts;

providing at least one pre-selected modification to the text transcripts; and

broadcasting the text transcripts corresponding to the speech segments as closed captions.

15. The method of claim 14, further comprising performing real-time system configuration.

16. The method of claim 15, further comprising:

identifying specific speakers associated with the speech segments; and

providing an appropriate individual speaker model.

17. The method of claim 16, wherein the one or more predetermined undesirable attributes comprises at least one of voice activity detection and crosstalk elimination.

18. The method of claim 17, wherein the at least one pre-selected modification to the text transcripts comprises at least one of context, error correction, vulgarity cleansing, and smoothing and interleaving of captions.

19. A computer program embodied on a computer readable medium and configured for detecting and modifying breath pauses in a speech input signal, the computer program comprising the steps of:

detecting breath pauses in a speech input signal;

modifying the breath pauses by replacing the breath pauses with a predetermined input and/or attenuating the breath pauses; and

outputting an output speech signal.

20. The computer program of claim 19, wherein the predetermined input is at least one of uniform noise and Gaussian noise and wherein detecting breath pauses comprises determining at least one of a normalized zero crossing count, a relative root-mean-square signal level, and one or more spectral power ratios.

21. The computer program of claim 20, wherein detecting breath pauses further comprises determining a contribution of +1, 0, −

1 or −

2 for each of the normalized zero crossing count, the relative root-mean-square signal level, the one or more spectral power ratios and the non-linear combination and wherein detecting breath pauses further comprises determining a pscore by combining each the contributions for each of the normalized zero crossing count, the relative root-mean-square signal level, the one or more spectral power ratios and the non-linear combination.

22. The computer program of claim 21, wherein detecting breath pauses further comprises determining the pscore over a predetermined number of audio frames and wherein detecting breath pauses still further comprises summing each pscore for each particular frame over the predetermined number of audio frames to determine a composite detection score.

23. The computer program of claim 22, further comprising filtering the speech input signal prior to detecting breath pauses;

and

wherein the composite detection score is determined for each of the normalized zero crossing count (NZCC), the relative root-mean-square (RRMS) signal level, the spectral power ratio and the non-linear combination based on the below;

NZCC;

if (0.09 <

ZCN <

0.22)pscore++;

RRMS;

if (RRMS <

0.085)pscore++;

else if (RRMS >

0.1)pscore−

−

;

Spectralif (lo-hi <

     5) AND (hiwide-lowide >

−

250)pscore−

−

;

Ratios;

if (lo-hi <

−

50)pscore−

−

;

if (lo-mid >

     200) AND (lo-hi <

     120)pscore−

−

;

if (hiwide-lowide >

−

100)pscore −

= 2;

Non-linearif the NZCC and RRMS criteria had positive contributions,Comb;

and the spectral ratio net contribution was zero, pscore++;

where;

ZCN is found by dividing a number of times a signal changes polarity within a frame by a length of the frame in samples; and

RRMS is found using the logic;

if (rms>

prms) prms=rms;

prms*=DECAY_COEFF;

rrms=rms/prms;

where rms is the current frame'"'"'s RMS value, PRMS is the peak-hold AR average RMS, DECAY_COEFF is a positive number less than 1.0.

24. The computer program of claim 19, wherein detecting breath pauses further comprises detecting plosives.

25. The computer program of claim 24, wherein detecting plosives comprises either determining:

(rms_half2/rms_half1>

5) OR (rms_current_frame/rms_last_frame>

5);

(NZCC has positive pscore contribution) OR (the composite detection score>

3); and

(the composite detection score<

20);

or

determining;

(current_frame_pscore<

0) AND (composite detection score<

20); and

(the composite detection score>

=9) OR (last_frame_pscore>

=3).

26. The computer program of claim 19, wherein modifying the breath pauses comprises selecting one of four modes, the modes selectable comprise no alteration of the speech input signal;

attenuation of the speech input signal;

replacing a breath pause with Gaussian noise; and

replacing a breath pause with uniform noise.

27. The computer program of claim 19, wherein modifying breath pauses comprises extending a breath pause.

28. The computer program of claim 19, further comprising detecting zero-valued samples in a processed segment output from the breath detection unit and wherein detecting zero-valued samples comprises counting a number of zero-valued samples and comparing the number to a predetermined threshold, where the number of zero-valued samples is above the threshold, further comprising adding uniform or Gaussian noise to the output speech signal.

29. The computer program of claim 19 being employed with a computer program for generating closed captions from an audio signal, the computer program for generating closed captions comprising:

correcting one additional predetermined undesirable attribute from the audio signal and outputting one or more speech segments;

generating from the one or more speech segments one or more text transcripts;

providing at least one pre-selected modification to the text transcripts; and

broadcasting the text transcripts corresponding to the speech segments as closed captions.

30. The computer program of claim 29, further comprising performing real-time system configuration.

31. The computer program of claim 30, further comprising:

identifying specific speakers associated with the speech segments; and

providing an appropriate individual speaker model.

32. The computer program of claim 31, wherein the one or more predetermined undesirable attributes comprises at least one of voice activity detection and crosstalk elimination.

33. The computer program of claim 32, wherein the at least one pre-selected modification to the text transcripts comprises at least one of context, error correction, vulgarity cleansing, and smoothing and interleaving of captions.