System and method of lattice-based search for spoken utterance retrieval

• US 9,286,890 B2
• Filed: 03/07/2014
• Issued: 03/15/2016
• Est. Priority Date: 08/23/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: AT&T Intellectual Property II LP (AT&T, Inc.)
• Inventors: Saraclar, Murat, Sproat, Richard William
• Primary Examiner(s): ALBERTALLI, BRIAN LOUIS
• Application Number:
  US14/200,700
• Publication Number:
  US 20140188474A1
• Time in Patent Office: 739 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/632 Query formulation
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/142 Hidden Markov Models [HMMs]
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams
• CPC Class Codes: G10L 2015/085 Methods for reducing search complexity, pruning

Abstract

A system and method are disclosed for retrieving audio segments from a spoken document. The spoken document preferably is one having moderate word error rates such as telephone calls or teleconferences. The method comprises converting speech associated with a spoken document into a lattice representation and indexing the lattice representation of speech. These steps are performed typically off-line. Upon receiving a query from a user, the method further comprises searching the indexed lattice representation of speech and returning retrieved audio segments from the spoken document that match the user query.

Claims

1. A method comprising:

converting, via a processor, speech from a spoken document into a lattice representation comprising phones from the speech, wherein the lattice representation belongs to a recognition network represented as weighted finite state machines, and wherein the phones have a pronunciation length longer than a minimum pronunciation length; and

indexing the lattice representation for word and topic searching.

2. The method of claim 1, wherein the method further comprises converting of the speech and indexing of the lattice representation occur off-line.

3. The method of claim 1, wherein the converting of speech further comprises causing the processor to provide timing information to the lattice representation.

4. The method of claim 1, wherein the converting of the speech occurs via automatic speech recognition.

5. The method of claim 4, wherein the automatic speech recognition is Hidden Markov Model based.

6. The method of claim 5, wherein the automatic speech recognition is large vocabulary continuous speech recognition.

7. The method of claim 5, wherein acoustic models used in the automatic speech recognition are decision tree state clustered triphones.

8. The method of claim 7, wherein language models used in the automatic speech recognition are pruned backoff trigram models.

9. The method of claim 5, wherein output distributions are mixtures of Gaussians.

10. A system comprising:

a processor; and

a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising;

converting speech from a spoken document into a lattice representation comprising phones from the speech, wherein the lattice representation belongs to a recognition network represented as weighted finite state machines, and wherein the phones have a pronunciation length longer than a minimum pronunciation length; and

indexing the lattice representation for word and topic searching.

11. The system of claim 10, the computer-readable storage medium having additional instructions stored which result in operations comprising converting of the speech and indexing of the lattice representation occur off-line.

12. The system of claim 10, wherein the converting of speech further comprises causing the processor to provide timing information to the lattice representation.

13. The system of claim 10, wherein the converting of the speech occurs via automatic speech recognition.

14. The system of claim 13, wherein the automatic speech recognition is Hidden Markov Model based.

15. The system of claim 14, wherein the automatic speech recognition is large vocabulary continuous speech recognition.

16. The system of claim 14, wherein acoustic models used in the automatic speech recognition are decision tree state clustered triphones.

17. The system of claim 16, wherein language models used in the automatic speech recognition are pruned backoff trigram models.

18. The system of claim 14, wherein output distributions are mixtures of Gaussians.

19. A computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:

converting speech from a spoken document into a lattice representation comprising phones from the speech, wherein the lattice representation belongs to a recognition network represented as weighted finite state machines, and wherein the phones have a pronunciation length longer than a minimum pronunciation length; and

indexing the lattice representation for word and topic searching.

20. The computer-readable storage device of claim 19, having additional instructions stored which result in operations comprising converting of the speech and indexing of the lattice representation occur off-line.