Fast update implementation for efficient latent semantic language modeling

• US 6,374,217 B1
• Filed: 03/12/1999
• Issued: 04/16/2002
• Est. Priority Date: 03/12/1999
• Status: Expired due to Term

• Current Assignee: Apple Inc.
• Original Assignee: Apple Computer Incorporated (Apple Inc.)
• Inventors: Bellegarda, Jerome R.
• Primary Examiner(s): Knepper, David D.
• Application Number:
  US09/267,334
• Time in Patent Office: 1,131 Days
• Field of Search: 704/236, 704/240, 704/241, 704/242, 704/255-257, 704/275
• US Class Current: 704/240
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams

Abstract

Speech or acoustic signals are processed directly using a hybrid stochastic language model produced by integrating a latent semantic analysis language model into an n-gram probability language model. The latent semantic analysis language model probability is computed using a first pseudo-document vector that is derived from a second pseudo-document vector with the pseudo-document vectors representing pseudo-documents created from the signals received at different times. The first pseudo-document vector is derived from the second pseudo-document vector by updating the second pseudo-document vector directly in latent semantic analysis space in response to at least one addition of a candidate word of the received speech signals to the pseudo-document represented by the second pseudo-document vector. Updating precludes mapping a sparse representation for a pseudo-document into the latent semantic space to produce the first pseudo-document vector. A linguistic message representative of the received speech signals is generated.

Claims

1. A method for performing speech recognition comprising:

receiving speech signals;

processing the received speech signals directly using a language model produced by integrating a latent semantic analysis language model into an n-gram probability language model, wherein the latent semantic analysis language model probability is computed using a first pseudo-document vector derived from a second pseudo-document vector, the first and second pseudo-document vectors representing pseudo-documents created from the received speech signals at different points in time; and

generating a linguistic message representative of the received speech signals.

2. The method of claim 1, wherein deriving the first pseudo-document vector from the second pseudo-document vector comprises updating the second pseudo-document vector directly in latent semantic analysis space in response to at least one addition of a word of the received speech signals to the pseudo-document represented by the second pseudo-document vector.

3. The method of claim 1, further comprising producing an acoustic vector sequence from the received speech signals by a mapping from words of the received speech signals.

4. The method of claim 1, wherein processing further comprises computing the latent semantic analysis language model probability by:

constructing a sparse representation for a current word;

mapping the sparse representation for the current word to a vector for the current word; and

evaluating the closeness between the vector for the current word and the first pseudo-document vector.

5. The method of claim 4, wherein the mapping follows from a singular value decomposition of a matrix of co-occurrences between at least one word and at least one document.

6. The method of claim 2, wherein updating precludes mapping a sparse representation for a pseudo-document into the latent semantic analysis space to produce the first pseudo-document vector, wherein a number of computations of the processing are reduced by a value approximately equal to a vocabulary size.

7. A method for generating a language model for use in a speech recognition system, the method comprising integrating a latent semantic analysis language model into an n-gram probability language model, wherein the latent semantic analysis language model probability is computed using a first pseudo-document vector derived from a second pseudo-document vector, the first and second pseudo-document vectors representing pseudo-documents created from the received speech signals at different points in time.

8. The method of claim 7, wherein deriving the first pseudo-document vector from the second pseudo-document vector comprises updating the second pseudo-document vector directly in latent semantic analysis space in response to at least one addition of a candidate word of the received speech signals to the pseudo-document represented by the second pseudo-document vector.

9. The method of claim 8, wherein updating precludes mapping a sparse representation for a pseudo-document into the latent semantic analysis space to produce the first pseudo-document vector.

10. A speech recognition process comprising a statistical learning technique that uses a language model, the language model produced by integrating a latent semantic analysis language model into an n-gram probability language model, wherein the latent semantic analysis language model probability is computed using a first pseudo-document vector derived from a second pseudo-document vector, the first and second pseudo-document vectors representing pseudo-documents created from the received speech signals at different points in time.

11. The speech recognition process of claim 10, wherein deriving the first pseudo-document vector from the second pseudo-document vector comprises updating the second pseudo-document vector directly in latent semantic analysis space in response to at least one addition of a candidate word of the received speech signals to the pseudo-document represented by the second pseudo-document vector.

12. The speech recognition process of claim 11, wherein updating precludes mapping a sparse representations for a pseudo-document into the latent semantic analysis space to produce the first pseudo-document vector.

13. An apparatus for speech recognition comprising:

at least one processor;

an input coupled to the at least one processor, the input capable of receiving speech signals, the at least one processor configured to recognize the received speech signals using a language model produced by integrating a latent semantic analysis language model into an n-gram probability language model, wherein the latent semantic analysis language model probability is computed using a first pseudo-document vector derived from a second pseudo-document vector, the first and second pseudo-document vectors representing pseudo-documents created from the received speech signals at different points in time; and

an output coupled to the at least one processor, the output capable of providing a linguistic message representative of the received speech signals.

14. The apparatus of claim 13, wherein deriving the first pseudo-document vector from the second pseudo-document vector comprises updating the second pseudo-document vector directly in latent semantic analysis space in response to at least one addition of a candidate word of the received speech signals to the pseudo-document represented by the second pseudo-document vector.

15. The apparatus of claim 13, wherein the at least one processor is further configured to produce an acoustic vector sequence from the received speech signals by a mapping from words of the received speech signals.

16. The apparatus of claim 13, wherein the processor is further configured to compute the latent semantic analysis language model probability by:

constructing a sparse representation for a current word;

mapping the sparse representation for the current word to a vector for the current word; and

evaluating the closeness between the vector for the current word and the first pseudo-document vector.

17. The apparatus of claim 16, wherein the mapping follows from a singular value decomposition of a matrix of co-occurrences between at least one word and at least one document.

18. The apparatus of claim 14, wherein updating precludes mapping a sparse representation for a pseudo-document into the latent semantic analysis space to produce the first pseudo-document vector, wherein a number of computations of the processing are reduced by a value approximately equal to a vocabulary size.

19. A computer readable medium containing executable instructions which, when executed in a processing system, causes the system to perform a method for recognizing speech, the method comprising:

receiving speech signals;

processing the received speech signals directly using a language model produced by integrating a latent semantic analysis language model into an n-gram probability language model, wherein the latent semantic analysis language model probability is computed using a first pseudo-document vector derived from a second pseudo-document vector, the first and second pseudo-document vectors representing pseudo-documents created from the received speech signals at different points in time; and

generating a linguistic message representative of the received speech signals.

20. The computer readable medium of claim 19, wherein deriving the first pseudo-document vector from the second pseudo-document vector comprises updating the second pseudo-document vector directly in latent semantic analysis space in order in response to at least one addition of a candidate word of the received speech signals to the pseudo-document represented by the second pseudo-document vector.

21. The computer readable medium of claim 19, wherein the method further comprises producing an acoustic vector sequence from the received speech signals by a mapping from words of the received speech signals.

22. The computer readable medium of claim 19, wherein processing further comprises computing the latent semantic analysis language model probability by:

constructing a sparse representation for a current word;

mapping the sparse representation for the current word to a vector for the current word; and

evaluating the closeness between the vector for the current word and the first pseudo-document vector.

23. The computer readable medium of claim 22, wherein the mapping follows from a singular value decomposition of a matrix of co-occurrences between at least one word and at least one document.

24. The computer readable medium of claim 20, wherein updating precludes mapping a sparse representation for a pseudo-document into the latent semantic analysis space to produce the first pseudo-document vector, wherein a number of computations of the processing are reduced by a value approximately equal to a vocabulary size.