Method and system for retrieving documents with spoken queries

• US 6,877,001 B2
• Filed: 04/25/2002
• Issued: 04/05/2005
• Est. Priority Date: 04/25/2002
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Ramakrishnan, Bhiksha, Wolf, Peter P.
• Primary Examiner(s): Mizrahi, Diane D.
• Application Number:
  US10/132,753
• Publication Number:
  US 20030204492A1
• Time in Patent Office: 1,076 Days
• Field of Search: 707/10, 707 1- 9, 707/104.1, 707/200, 704/251, 704/270.1, 704/257, 304/32
• US Class Current: 707/711
• CPC Class Codes: G06F 16/3343 using phonetics
• CPC Class Codes: Y10S 707/916 Audio
• CPC Class Codes: Y10S 707/959 Network
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching

Abstract

A system and method indexes and retrieves documents stored in a database. A document feature vector is extracted for each document to be indexed. The feature vector is projected to a low dimension document feature vector, and the documents are indexed according to the low dimension document feature vectors. A spoken query is represented as a lattice indicating possible sequential combinations of words in the spoken query. The lattice is converted to a query certainty vector, which is also projected to a low dimension query certainty vector. The low dimension query vector is compared to each of the low dimension document feature vectors to retrieve a matching result set of documents.

Claims

1. A computer-implemented method for indexing and retrieving documents stored in a database, comprising:

extracting a document feature vector for each of a plurality of documents;

projecting each document feature vector to a low dimension document feature vector;

indexing each of the plurality of documents according the associated low dimension document feature vector in the database;

representing a spoken query as a lattice, the lattice representing likely possible sequential combinations of words in the spoken query;

converting the lattice to a query certainty vector;

projecting the query certainty vector to an associated low dimension query certainty vector;

comparing the low dimension query vector to each of the low dimension document feature vectors; and

retrieving a result set of documents from the database that have low dimension document feature vectors that match the low dimension query certainty vector.

2. The computer-implemented method of claim 1 wherein the document feature vector is a weighted list of all words in each document.

3. The computer-implemented method of claim 2 wherein the weight of each word is equal to a frequency of appearance of the word in the document.

4. The computer-implemented method of claim 1 wherein the projecting uses a singular value decomposition.

5. The computer-implemented method of claim 1 wherein the projecting uses a latent semantic analysis.

6. The computer-implemented method of claim 1 further comprising:

filtering common words from the document feature vector.

7. The computer-implemented method of claim 1 wherein the lattice includes nodes and directed edges, the nodes representing spoken words, and the directed edges connecting the nodes representing possible orders in which the words could have been spoken.

8. The computer-implemented method of claim 7 further comprising:

associating confidence scores with the nodes and the directed edges.

9. The computer-implemented method of claim 8 wherein a Gaussian classifier is used to determine the confidence scores.

10. The computer-implemented method of claim 8 wherein a boosting based classifier is used to determine the confidence scores.

11. The computer-implemented method of claim 8 wherein the confidence scores are derived from rule based heuristics.

12. The computer-implemented method of claim 8 wherein the confidence scores are based on features of an audio signal of the spoken query.

13. The computer-implemented method of claim 12 wherein the audio features include volume and pitch.

14. The computer-implemented method of claim 1 wherein the documents of the result set are in an order of pertinence scores of the documents.

15. The computer-implemented method of claim 1 wherein the documents include multi-media content.

16. The computer-implemented method of claim 1 wherein the lattice includes nodes and directed edges, the nodes representing phonemes, and the directed edges connecting the nodes representing possible orders in which the phonemes could have been spoken.

17. The computer-implemented method of claim 7 further comprising:

stemming the words.

18. The computer-implemented method of claim 1 further comprising:

dynamically constructing a vocabulary for a speech recognizer used to recognize the spoken query from key words extracted from each document.

19. A computer-implemented system for indexing and retrieving documents in a database, comprising:

means for extracting a document feature vector for each of a plurality of documents;

means for projecting each document feature vector to a low dimension document feature vector;

a database indexing each of the plurality of documents according the associated low dimension document feature vector;

means for representing a spoken query as a lattice, the lattice representing likely possible sequential combinations of words in the spoken query;

means for converting the lattice to a query certainty vector;

means for projecting each query certainty vector to an associated low dimension query certainty vector;

means for comparing the low dimension query vector to each of the low dimension document feature vectors; and

a search engine configured to retrieve a result set of documents from the database that have low dimension document feature vectors that match the low dimension query certainty vector.