System and method for robust access and entry to large structured data using voice form-filling

• US 9,165,554 B2
• Filed: 12/04/2014
• Issued: 10/20/2015
• Est. Priority Date: 08/26/2005
• Status: Expired due to Fees

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: AT&T Intellectual Property II LP (AT&T, Inc.)
• Inventors: Allauzen, Cyril Georges Luc, Parthasarathy, Sarangarajan
• Primary Examiner(s): Desir, Pierre-Louis
• Assistant Examiner(s): KOVACEK, DAVID M
• Application Number:
  US14/560,219
• Publication Number:
  US 20150088510A1
• Time in Patent Office: 320 Days
• Field of Search: 704 9- 10, 704236-257, 704E17001-E17008, 704E17013-E17014, 704E15014-E15024, 704/E15.038
• US Class Current: 1/1
• CPC Class Codes: G06F 40/174 Form filling; Merging
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/06 Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice G10L15/14 takes precedence
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units
• CPC Class Codes: G10L 2015/081 Search algorithms, e.g. Baum-Welch or Viterbi
• CPC Class Codes: G10L 2015/088 Word spotting

Abstract

A method, apparatus and machine-readable medium are provided. A phonotactic grammar is utilized to perform speech recognition on received speech and to generate a phoneme lattice. A document shortlist is generated based on using the phoneme lattice to query an index. A grammar is generated from the document shortlist. Data for each of at least one input field is identified based on the received speech and the generated grammar.

Claims

1. A method comprising:

performing, via a processor, a first pass of entries in a database to generate a shortlist of the entries, the first pass comprising;

utilizing a phonotactic grammar to perform speech recognition on received speech and to generate a phoneme lattice, wherein the utilizing ignores inter-field constraints of the phonotactic grammar; and

generating the shortlist of the entries in the database based on a normalization of the phoneme lattice; and

performing a second pass on the shortlist of the entries in the database to obtain a final result, the second pass comprising;

generating a grammar from the shortlist of the entries in the database, to yield a generated grammar having inter-field constraints; and

identifying data from the database for each of the input field based on the received speech, the generated grammar, and the inter-field constraints.

2. The method of claim 1, further comprising:

generating an index to the entries in the database based on a plurality of training phoneme lattices and factors of interest from valid entries in the database, wherein the factors of interest comprise trigrams;

wherein the identifying of the data from the database is further based on the index.

3. The method of claim 2, wherein the index is divided into two sub-indices.

4. The method of claim 2, wherein the factors of interest further comprise N-grams based on the valid entries in the database.

5. The method of claim 1, wherein the generated grammar comprises common variations of the input field.

6. The method of claim 1, wherein the phonotactic grammar is an N-gram phonotactic grammar.

7. The method of claim 6, wherein the N-gram phonotactic grammar is unsmoothed, recognizing only N-grams which have been seen in data used to train the N-gram phonotactic grammar.

8. The method of claim 1, wherein the generated grammar is used to:

perform one of another speech recognition on the received speech, constrain recognition for each of the input field, and rescore a result of performing the speech recognition.

9. The method of claim 1, wherein generating the shortlist of the entries in the database further comprises:

utilizing only the factors of interest to generate a cost-normalized query based on a normalized lattice; and

selecting the N-best lowest indices from indices of an index containing the factors of interest and the cost-normalized query.

10. A system comprising:

a processor; and

a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising;

performing a first pass of entries in a database to generate a shortlist of the entries, the first pass comprising;

utilizing a phonotactic grammar to perform speech recognition on received speech and to generate a phoneme lattice, wherein the utilizing ignores inter-field constraints of the phonotactic grammar; and

generating the shortlist of the entries in the database based on a normalization of the phoneme lattice; and

performing a second pass on the shortlist of the entries in the database to obtain a final result, the second pass comprising;

generating a grammar from the shortlist of the entries in the database, to yield a generated grammar having inter-field constraints; and

identifying data from the database for each of the input field based on the received speech, the generated grammar, and the inter-field constraints.

11. The system of claim 10, the computer-readable storage medium having additional instructions stored which, when executed by the processor, result in operations comprising:

generating an index to the entries in the database based on a plurality of training phoneme lattices and factors of interest from valid entries in the database, wherein the factors of interest comprise trigrams;

wherein the identifying of the data from the database is further based on the index.

12. The system of claim 11, wherein the index is divided into two sub-indices.

13. The system of claim 11, wherein the factors of interest further comprise N-grams based on the valid entries in the database.

14. The system of claim 10, wherein the generated grammar comprises common variations of the input field.

15. The system of claim 10, wherein the phonotactic grammar is an N-gram phonotactic grammar.

16. The system of claim 15, wherein the N-gram phonotactic grammar is unsmoothed, recognizing only N-grams which have been seen in data used to train the N-gram phonotactic grammar.

17. The system of claim 10, wherein the generated grammar is used to:

perform one of another speech recognition on the received speech, constrain recognition for each of the input field, and rescore a result of performing the speech recognition.

18. The system of claim 10, wherein generating the shortlist of the entries in the database further comprises:

utilizing only the factors of interest to generate a cost-normalized query based on a normalized lattice; and

selecting the N-best lowest indices from indices of an index containing the factors of interest and the cost-normalized query.

19. A computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:

performing first pass of entries in a database to generate a shortlist of the entries, the first pass comprising;

utilizing a phonotactic grammar to perform speech recognition on received speech and to generate a phoneme lattice, wherein the utilizing ignores inter-field constraints of the phonotactic grammar; and

generating the shortlist of the entries in the database based on a normalization of the phoneme lattice; and

performing a second pass on the shortlist of the entries in the database to obtain a final result, the second pass comprising;

generating a grammar from the shortlist of the entries in the database, to yield a generated grammar having inter-field constraints; and

identifying data from the database for each of the input field based on the received speech, the generated grammar, and the inter-field constraints.

20. The computer-readable storage device of claim 19, having additional instructions stored which, when executed by the computing device, result in operations comprising:

generating an index to the entries in the database based on a plurality of training phoneme lattices and factors of interest from valid entries in the database, wherein the factors of interest comprise trigrams;

wherein the identifying of the data from the database is further based on the index.