METHOD AND SYSTEM FOR GENERATING GRAMMAR RULES

• US 20150019205A1
• Filed: 06/23/2014
• Published: 01/15/2015
• Est. Priority Date: 10/17/2000
• Status: Active Grant

• Current Assignee: Telstra Corporation Limited (Telstra Group Ltd.)
• Original Assignee: Telstra Corporation Limited (Telstra Group Ltd.)
• Inventors: Jiang, Jason, Starkie, Bradford Craig
• Granted Patent: US 9,183,204 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing
• CPC Class Codes: G06F 16/3334 Selection or weighting of terms from queries, including natural language queries
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence

Abstract

An information retrieval system, including a natural language parser (3) for parsing documents of a document space (1) to identify key terms of each document based on linguistic structure, and for parsing a search query to determine the search term, a feature extractor (4) for determining an importance score for terms of the document space (1) based on distribution of the terms in the document space (1), an index term generator (5) for generating index terms using the key terms identified by the parser (3) and the extractor (4) and having an importance score above a threshold level, and a query clarifier (16) for selecting from the index terms, on the basis of the search term, index terms for selecting at least one document from the document space (1). A speech recognition engine (12) is used to generate the query, and a bi-gram language module (6) generates grammar rules for the speech recognition engine (12) using the index terms.

Claims

1. A method of generating domain-specific grammar rules using a computer system having data processing logic, the method comprising:

parsing a plurality of documents stored in a digital document database on computer-accessible storage media to identify key terms of each document based on sentence structure;

extracting a plurality of n-grams from each document, wherein one or more of the n-grams include spaces and partial words;

extracting a frequency of each n-gram in each document;

extracting a frequency of each n-gram in the plurality of documents;

assigning a novelty score to each of the n-grams in each corresponding document, said novelty score representing and being based on the extracted frequency of the n-gram in the document and the extracted frequency of the n-gram in the plurality of documents;

determining which of the extracted n-grams are in each identified key term;

assigning a weight to each key term based the novelty scores assigned to at the extracted n-grams in the key term; and

generating the domain-specific grammar rules for a speech recognition engine, said grammar rules including said key terms in association with respective probabilities based on the weights of the key terms, wherein the key terms define phrases that are likely to be spoken from the plurality of documents, and the grammar rules define which of the phrases are likely to follow others of the phrases with the likelihoods defined by the probabilities.

2. A method as claimed in claim 1, wherein a natural language parser executes said parsing, and said key terms are linguistically important terms of each document.

3. A method as claimed in claim 2, wherein said parser generates key-centered phrase structure frames for sentences of each document, and generates at least one frame relation graph that is parsed to determine the frames representative of the sentences of each document, said frames including said key terms.

6. A method as claimed in claim 1, wherein the novelty score is determined on the basis of

10. A method as claimed in claim 1, wherein generating the grammar rules comprises generating a list of phrases including said key terms and said respective weights, and inputting said list as a bi-gram array with said weights representing said probabilities, to generate said grammar rules for said speech recognition engine.

4-5. -5. (canceled)

7-9. -9. (canceled)

11-20. -20. (canceled)

21. An extraction system for generating domain-specific grammar rules, the extraction system including a computer system having data processing logic configured to provide:

a parser for parsing a plurality of documents stored in a digital document database on computer-accessible storage media to identify key terms of each document based on sentence structure;

a feature extractor for;

extracting a plurality of n-grams from each document, wherein one or more of the n-grams include spaces and partial words;

extracting a frequency of each n-gram in each document;

extracting a frequency of each n-gram in the plurality of documents;

assigning a novelty score to each of the n-grams in corresponding documents, said novelty score representing and being based on the extracted frequency of the n-gram in the document and the extracted frequency of the n-gram in the plurality of documents,

determining which of the extracted n-grams are in each identified key term, and assigning a weight to each key term based on the novelty scores assigned to the extracted n-grams in the key term; and

a grammar generator for generating the domain-specific grammar rules for a speech recognition engine, said grammar rules including said key terms in association with respective probabilities based on the weights of the key terms.

31. A system as claimed in claim 21, wherein a natural language parser executes said parsing, and said key terms are linguistically important terms of each document.

32. A system as claimed in claim 31, wherein said parser generates key-centered phrase structure frames for sentences of each document, and generates at least one frame relation graph that is parsed to determine the frames representative of the sentences of each document, said frames including said key terms.

33. A system as claimed in claim 21, wherein the novelty score is determined on the basis of

34. A machine readable medium as claimed in claim 21 wherein generating the grammar rules comprises generating a list of phrases including said key terms and said, and inputting said list as a bi-gram array with said weights representing said probabilities, to generate said grammar rules for said speech recognition engine.

22-29. -29. (canceled)

30. A machine-readable non-transitory medium having stored thereon instructions for generating domain-specific grammar rules comprising machine executable code which when executed by at least one machine, causes the machine to:

parse a plurality of documents stored in a digital document database on a computer-accessible storage media to identify key terms of each document based on sentence structure;

extract a plurality of n-grams from each document, wherein one or more of the n-grams include spaces and partial words;

extract a frequency of each n-gram in each document;

extract a frequency of each n-gram in the plurality of documents;

assign a novelty score to each of the n-grams in each corresponding document, said novelty score representing and being based on the extracted frequency of the n-gram in the document and the extracted frequency of the n-gram in the plurality of documents;

determine which of the extracted n-grams are in each identified key term;

assign a weight to each key term based the novelty scores assigned to at the extracted n-grams in the key term; and

generate the domain-specific grammar rules for a speech recognition engine, said grammar rules including said key terms in association with respective probabilities based on the weights of the key terms, wherein the key terms define phrases that are likely to be spoken from the plurality of documents, and the grammar rules define which of the phrases are likely to follow others of the phrases with the likelihoods defined by the probabilities.

35. A machine readable medium as claimed in claim 30, wherein a natural language parser executes said parsing, and said key terms are linguistically important terms of each document.

36. A machine readable medium as claimed in claim 35, wherein said parser generates key-centered phrase structure frames for sentences of each document, and generates at least one frame relation graph that is parsed to determine the frames representative of the sentences of each document, said frames including said key terms.

37. A machine readable medium as claimed in claim 30, wherein the novelty score is determined on the basis of

38. A machine readable medium as claimed in claim 30 wherein generating the grammar rules comprises generating a list of phrases including said key terms and said, and inputting said list as a bi-gram array with said weights representing said probabilities, to generate said grammar rules for said speech recognition engine.