EXTRACTING STRUCTURED DATA FROM WEB QUERIES

• US 20110270815A1
• Filed: 04/30/2010
• Published: 11/03/2011
• Est. Priority Date: 04/30/2010
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Li, Xiao
• Application Number:
  US12/770,778
• Publication Number:
  US 20110270815A1
• Time in Patent Office: Days
• US Class Current: 707/706
• CPC Class Codes: G06F 16/24522 Translation of natural language queries to structured queries
• CPC Class Codes: G06F 40/30 Semantic analysis

Abstract

Described is processing a web query into structured data, such as for use in formulating a database query. A classifier and segmental conditional random field (CRF) model classify and segment a query into labeled segments used to obtain the structured data. The structured data includes a class, an intent set corresponding to data sought by the query, and an attribute set corresponding to an attribute name and attribute value representing each modifier in the query. The structured data may be converted into a query language (e.g., SQL) query to query the structured data source; e.g., the class may be used to select a database, the attribute set used to find one or more columns and a row in the database, and the intent set matched to a column, whose row data is the result being sought. A web search engine may return the database query as part of its results.

Claims

1. In a computing environment, a method performed on at least one processor comprising, processing a web query into structured data, including classifying the query and segmenting the query into labeled segments, the structured data including a class, an intent set comprising one or more labeled segments corresponding to data sought by the query, and an attribute set comprising one or more labeled segments that each correspond to a modifier of the query.

2. The method of claim 1 wherein classifying the query comprises determining the class based upon classification results of a classifier.

3. The method of claim 1 wherein classifying the query comprises selecting a class based on probability data provided by a classifier, and probability data associated with one or more of the labeled segments.

4. The method of claim 1 further comprising, converting the structured data into a structured query of a query language, querying a database with the structured query, and returning results.

5. The method of claim 4 further comprising, using the class to determine the database from among a plurality of databases, using the intent set to determine one or more columns in the database, and using the attribute set to determine one or more other columns in the database and data within those one or more other columns.

6. The method of claim 1 wherein segmenting the query comprises labeling each segment with an intent, attribute or other name label.

7. The method of claim 1 wherein processing the web query comprises obtaining the intent set and attribute set from a segmental conditional random field model.

8. The method of claim 1 further comprising, training the segmental conditional random field model with labeled training data including segment information, and attribute name and attribute value lexicons.

9. In a computing environment, a system comprising, a semantic structure extractor, including a classifier that classifies a query into classification-related data, a segmental conditional random field model that segments the query into segment-related data including a label for each segment, a mechanism for selecting a class based upon the classification-related data and the segment-related data, the semantic structure extractor providing structured data comprising the class, an intent set comprising one or more labeled segments corresponding to data sought by the query, and an attribute set comprising one or more labeled segments that each correspond to a modifier of the query, in which each label of a segment the attribute set correspond to the class.

10. The system of claim 9 wherein the classifier comprises a maximum entropy classifier, a boosted decision tree classifier, or a neural network classifier.

11. The system of claim 9 wherein the mechanism for selecting a class based upon the classification-related data and the segment-related data uses probability data provided by the classifier and other probability data provided by the segmental conditional random field model.

12. The system of claim 9 wherein the segmental conditional random field model uses one or more features, including an indicator function that indicates whether a segment contains a n-gram that occurred in training, or an indicator function that indicates whether a segment exactly matches a segment phrase that occurred in the training, or both an indicator function that indicates whether a segment contains a n-gram that occurred in training and an indicator function that indicates whether the segment exactly matches a segment phrase that occurred in the training.

13. The system of claim 9 wherein the segmental conditional random field model uses one or more features, including an indicator function that indicates whether a segment exactly matches an item in a particular lexicon.

14. The system of claim 9 wherein the segmental conditional random field model uses one or more features, including normalized edit distance, a Tf-Idf score or a fuzzy match score of at least one segment with respect to a matching lexicon item, or any combination of normalized edit distance, a Tf-Idf score or a fuzzy match score of at least one segment with respect to a matching lexicon item.

15. The system of claim 9 further comprising, a mechanism that converts the structured data into a structured query for querying a database.

16. The system of claim 9 wherein the semantic structure extractor is incorporated into a web search engine or accessed by a web search engine.

17. In a computing environment, a system comprising:

a web search engine that receives a web query;

a semantic structure extractor that receives at least part of the web query from the web search engine and converts the web query into structured data including a class, an intent set comprising one or more labeled segments corresponding to data sought by the query, and an attribute set comprising one or more labeled segments that each correspond to a modifier of the query, the semantic structure extractor determining the class based upon terms in the query, and determining labels for the segments based upon the class and the terms in the query; and

a mechanism that uses the structured data to query a source of structured data to provide results from the source of structured data to the web search engine.

18. The system of claim 17 wherein the semantic structure extractor includes a classifier for use in determining the class.

19. The system of claim 17 wherein the semantic structure extractor includes a segmental conditional random field model for use in obtaining the labeled segments.

20. The system of claim 17 wherein the web search engine returns a results page including results from the source of structured data in response to the web query.