SYSTEMS AND METHODS FOR AUTOMATIC REPAIR OF SPEECH RECOGNITION ENGINE OUTPUT

• US 20180018960A1
• Filed: 07/13/2017
• Published: 01/18/2018
• Est. Priority Date: 07/13/2016
• Status: Active Grant

• Current Assignee: AA R&D LLC, TATA Consultancy Services Limited (Tata Sons Pvt Ltd.)
• Original Assignee: TATA Consultancy Services Limited (Tata Sons Pvt Ltd.)
• Inventors: ANANTARAM, Chandrasekhar, KOPPARAPU, Sunil Kumar, PATEL, Chiragkumar Rameshbhai, MITTAL, Aditya
• Granted Patent: US 10,410,622 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 40/232 Orthographic correction, e.g. spell checking or vowelisation
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 21/00 Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility G10L19/00 takes precedence

Abstract

Text output of speech recognition engines tend to be erroneous when spoken data has domain specific terms. The present disclosure facilitates automatic correction of errors in speech to text conversion using abstractions of evolutionary development and artificial development. The words in a speech recognition engine text output are treated as a set of injured genes in a biological cell that need repair which are then repaired and form genotypes that are then repaired to phenotypes through a series of repair steps based on a matching, mapping and linguistic repair through a fitness criteria. A basic genetic level repair involves phonetic MATCHING function together with a FITNESS function to select the best among the matching genes. A second genetic level repair involves a contextual MAPPING function for repairing remaining ‘injured’ genes of the speech recognition engine output. Finally, a genotype to phenotype repair involves using linguistic rules and semantic rules of the domain.

Claims

1. A processor implemented method (200) comprising:

receiving a text output from a general purpose automatic speech recognition (GP-ASR) engine (202);

identifying in real-time, a current environment of a speaker associated with the text output based on one or more environment variables associated thereof (204); and

automatically and selectively correcting the output of the GP-ASR by replacing one or more erroneously recognized terms in the text output with one or more best-fit terms based on at least a fitness function and a pre-determined threshold associated with the identified current environment (206).

2. The processor implemented method of claim 1 further comprising:

receiving speech input associated with the speaker and information pertaining to the current environment associated thereof; and

deriving the one or more environment variables from the received speech input and the information pertaining to the current environment.

3. The processor implemented method of claim 2, wherein the one or more environment variables are based on one or more of (i) location information associated with the speaker;

(ii) movement information associated with the speaker;

(iii) olfactory information;

(iv) proximity information; and

(v) image information.

4. The processor implemented method of claim 1, wherein the identified current environment is classified as at least one of a broad classification and a granular classification.

5. The processor implemented method of claim 1, wherein the pre-determined threshold is empirical.

6. The processor implemented method of claim 1, wherein the step of automatically and selectively correcting the output of the GP-ASR comprises performing multi-stage correction steps on the received text output, the multi-stage correction steps comprising one or more of:

domain ontology based correction, wherein a phonetic-level match, an edit-distance-level-match or a combination thereof is performed to obtain a matching stage output (206a);

contextual correction of the matching stage output, wherein a contextual phonetic-level match, an edit-distance-level-match or a combination thereof is performed to obtain a mapping stage output (206b); and

linguistic correction of the mapping stage output, wherein semantic and linguistic repair rules based on the identified current environment are applied on the mapping stage output to obtain a domain specific repaired output corresponding to the output of the GP-ASR (206c).

7. The processor implemented method of claim 6, wherein steps 206a through 206c are iteratively repeated based on the real-time current environment.

8. The processor implemented method of claim 6, wherein the fitness function for domain ontology based correction is a weighted cost function that matches the text output with a first set of candidate terms from the domain ontology based on a sliding window mechanism;

and replaces the one or more erroneously recognized terms in the text output with one or more best-fit terms from the first set of candidate terms from the domain ontology based on the pre-determined threshold associated with the identified current environment.

9. The processor implemented method of claim 6, wherein the fitness function for contextual correction is a weighted cost function that maps the matching stage output with a second set of candidate terms from the domain ontology based on a sliding window mechanism;

and replaces the one or more erroneously recognized terms in the matching stage output with one or more best-fit terms from the second set of candidate terms from the domain ontology based on the pre-determined threshold associated with the identified current environment.

10. The processor implemented method of claim 6, wherein the fitness function for the linguistic correction is a function of the mapping stage output and a set of semantic and linguistic repair rules.

11. A system (100) comprising:

one or more data storage devices (102) operatively coupled to one or more hardware processors (104) and configured to store instructions configured for execution by the one or more hardware processors to;

receive a text output from a general purpose automatic speech recognition (GP-ASR) engine;

identify in real-time, a current environment of a speaker associated with the text output based on one or more environment variables associated thereof;

the identified current environment being classified as at least one of a broad classification and a granular classification; and

automatically and selectively correct the output of the GP-ASR by replacing one or more erroneously recognized terms in the text output with one or more best-fit terms based on at least a fitness function and a pre-determined threshold associated with the identified current environment.

12. The system of claim 11, wherein the one or more hardware processors are further configured to:

receive speech input associated with the speaker and information pertaining to the current environment associated thereof; and

derive the one or more environment variables from the received speech input and the information pertaining to the current environment.

13. The system of claim 11, wherein the one or more environment variables are based on one or more of (i) location information associated with the speaker;

(ii) movement information associated with the speaker;

(iii) olfactory information;

(iv) proximity information; and

(v) image information.

14. The system of claim 11, wherein the pre-determined threshold is empirical.

15. The system of claim 11, wherein the one or more hardware processors are further configured to automatically and selectively correct the output of the GP-ASR by performing multi-stage correction steps on the received output, the multi-stage correction steps comprising one or more of:

domain ontology based correction, wherein a phonetic-level match, an edit-distance-level-match or a combination thereof is performed to obtain a matching stage output;

contextual correction of the matching stage output, wherein a contextual phonetic-level match, an edit-distance-level-match or a combination thereof is performed to obtain a mapping stage output; and

linguistic correction of the mapping stage output, wherein semantic and linguistic repair rules based on the identified current environment are applied on the mapping stage output to obtain a domain specific repaired output corresponding to the output of the GP-ASR.

16. The system of claim 15, wherein the one or more hardware processors are further configured to perform the domain ontology based correction, wherein the fitness function is a weighted cost function that matches the text output with a first set of candidate terms from the domain ontology based on a sliding window mechanism;

and replaces the one or more erroneously recognized terms in the text output with one or more best-fit terms from the first set of candidate terms from the domain ontology based on the pre-determined threshold associated with the identified current environment.

17. The system of claim 15, wherein the one or more hardware processors are further configured to perform the contextual correction, wherein the fitness function is a weighted cost function that maps the matching stage output with a second set of candidate terms from the domain ontology based on a sliding window mechanism;

and replaces the one or more erroneously recognized terms in the matching stage output with one or more best-fit terms from the second set of candidate terms from the domain ontology based on the pre-determined threshold associated with the identified current environment.

18. The system of claim 15, wherein the one or more hardware processors are further configured to perform the linguistic correction, wherein the fitness function is a function of the mapping stage output and a set of semantic and linguistic repair rules.

19. A computer program product comprising a non-transitory computer readable medium having a computer readable program embodied therein, wherein the computer readable program, when executed on a computing device, causes the computing device to:

receive a text output from a general purpose automatic speech recognition (GP-ASR) engine;

identify in real-time, a current environment of a speaker associated with the text output based on one or more environment variables associated thereof; and

automatically and selectively correct the output of the GP-ASR by replacing one or more erroneously recognized terms in the text output with one or more best-fit terms based on at least a fitness function and a pre-determined threshold associated with the identified current environment.