System and method for robust access and entry to large structured data using voice form-filling

• US 9,824,682 B2
• Filed: 10/19/2015
• Issued: 11/21/2017
• Est. Priority Date: 08/26/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Allauzen, Cyril Georges Luc, Parthasarathy, Sarangarajan
• Primary Examiner(s): He, Jialong
• Application Number:
  US14/886,650
• Publication Number:
  US 20160042732A1
• Time in Patent Office: 764 Days
• Field of Search: 704 9- 10, 704236-257, 704E17001-E17008, 704E17013-E17014, 704E15014-E15024, 704E15038
• CPC Class Codes: G06F 40/174 Form filling; Merging
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/06 Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice G10L15/14 takes precedence
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units
• CPC Class Codes: G10L 2015/081 Search algorithms, e.g. Baum-Welch or Viterbi
• CPC Class Codes: G10L 2015/088 Word spotting

Abstract

A method, apparatus and machine-readable medium are provided. A phonotactic grammar is utilized to perform speech recognition on received speech and to generate a phoneme lattice. A document shortlist is generated based on using the phoneme lattice to query an index. A grammar is generated from the document shortlist. Data for each of at least one input field is identified based on the received speech and the generated grammar.

Claims

1. A method comprising:

receiving a speech;

recognizing the speech using a phonotactic grammar to generate a phone lattice;

removing silence and filler words from the phone lattice, to yield a revised phone lattice;

normalizing, via a processor, costs in the revised phone lattice such that a cost of a best path is set to zero;

generating a cost-normalized query using factors of interest, wherein an index of words is indexed by the factors of interest;

generating, via the processor and by performing a first pass of entries in a database, a shortlist of recognized speech possibilities using the revised phone lattice, the index of words, and indices contained in the cost-normalized query;

performing a second pass on the shortlist of recognized speech possibilities using a grammar generated from the entries in the database to obtain a final result; and

providing a response to the speech based on the final result.

2. The method of claim 1, wherein the index of words is generated based on a plurality of training phoneme lattices and factors of interest from valid entries in a database, wherein the factors of interest comprise trigrams.

3. The method of claim 2, wherein the factors of interest further comprise N-grams based on the valid entries in the database.

4. The method of claim 1, wherein the phonotactic grammar is unsmoothed and is used to recognize only N-grams which have been seen in data used to train the phonotactic grammar.

5. A system comprising:

a processor; and

a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising;

receiving a speech;

recognizing the speech using a phonotactic grammar to generate a phone lattice;

removing silence and filler words from the phone lattice, to yield a revised phone lattice;

normalizing costs in the revised phone lattice such that a cost of a best path is set to zero;

generating a cost-normalized query using factors of interest, wherein an index of words is indexed by the factors of interest;

generating, by performing a first pass of entries in a database, a shortlist of recognized speech possibilities using the revised phone lattice, the index of words, and indices contained in the cost-normalized query;

performing a second pass on the shortlist of recognized speech possibilities using a grammar generated from the entries in the database to obtain a final result; and

providing a response to the speech based on the final result.

6. The system of claim 5, wherein the index of words is generated based on a plurality of training phoneme lattices and factors of interest from valid entries in a database, wherein the factors of interest comprise trigrams.

7. The system of claim 6, wherein the factors of interest further comprise N-grams based on the valid entries in the database.

8. The system of claim 5, wherein the phonotactic grammar is unsmoothed and is used to recognize only N-grams which have been seen in data used to train the phonotactic grammar.

9. A computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:

receiving a speech;

recognizing the speech using a phonotactic grammar to generate a phone lattice;

removing silence and filler words from the phone lattice, to yield a revised phone lattice;

normalizing costs in the revised phone lattice such that a cost of a best path is set to zero;

generating a cost-normalized query using factors of interest, wherein an index of words is indexed by the factors of interest;

generating, by performing a first pass of entries in a database, a shortlist of recognized speech possibilities using the revised phone lattice, the index of words, and indices contained in the cost-normalized query;

performing a second pass on the shortlist of recognized speech possibilities using a grammar generated from the entries in the database to obtain a final result; and

providing a response to the speech based on the final result.

10. The computer-readable storage device of claim 9, wherein the index of words is generated based on a plurality of training phoneme lattices and factors of interest from valid entries in a database, wherein the factors of interest comprise trigrams.

11. The computer-readable storage device of claim 10, wherein the factors of interest further comprise N-grams based on the valid entries in the database.