ACTIVE FEATURING IN COMPUTER-HUMAN INTERACTIVE LEARNING
First Claim
1. One or more computer-readable media having embodied thereon computer-usable instructions that, when executed, facilitate a method of interactive feature selection for machine learning, the method comprising:
- providing a first set of data items, wherein one or more of the data items have been previously labeled as examples of a particular class of data item;
utilizing a classifier to determine predicted labels for one or more of the data items;
identifying one or more data items having a discrepancy between a previous label and a predicted label;
presenting via a user interface an indication of the one or more data items having the discrepancy between the previous label and the predicted label, wherein the user interface includes a feature-selection interface configured to receive a user selection of one or more features that are usable as input features to train the classifier;
receiving, via the user interface, the user selection of one or more features; and
training the classifier with the one or more user-selected features as input features.
3 Assignments
0 Petitions
Accused Products
Abstract
A collection of data that is extremely large can be difficult to search and/or analyze. Relevance may be dramatically improved by automatically classifying queries and web pages in useful categories, and using these classification scores as relevance features. A thorough approach may require building a large number of classifiers, corresponding to the various types of information, activities, and products. Creation of classifiers and schematizers is provided on large data sets. Exercising the classifiers and schematizers on hundreds of millions of items may expose value that is inherent to the data by adding usable meta-data. Some aspects include active labeling exploration, automatic regularization and cold start, scaling with the number of items and the number of classifiers, active featuring, and segmentation and schematization.
56 Citations
20 Claims
-
1. One or more computer-readable media having embodied thereon computer-usable instructions that, when executed, facilitate a method of interactive feature selection for machine learning, the method comprising:
-
providing a first set of data items, wherein one or more of the data items have been previously labeled as examples of a particular class of data item; utilizing a classifier to determine predicted labels for one or more of the data items; identifying one or more data items having a discrepancy between a previous label and a predicted label; presenting via a user interface an indication of the one or more data items having the discrepancy between the previous label and the predicted label, wherein the user interface includes a feature-selection interface configured to receive a user selection of one or more features that are usable as input features to train the classifier; receiving, via the user interface, the user selection of one or more features; and training the classifier with the one or more user-selected features as input features. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. One or more computer-readable media having embodied thereon computer-usable instructions that, when executed, facilitate a method of interactive feature selection for machine learning, the method comprising:
-
providing a first set of data items, wherein one or more of the data items include tokens that have been previously labeled as portions of a particular schema; for the one or more data items, utilizing a schematizer to determine predicted labels for one or more of the tokens that have been previously labeled; identifying one or more tokens having a discrepancy between a previous label and a predicted label; presenting via a user interface an indication of the one or more tokens having the discrepancy between the previous label and the predicted label, wherein the user interface includes a feature-selection interface configured to receive a user selection of one or more features that are usable as input features to train the schematizer; receiving, via the user interface, the user selection of one or more features; and training the schematizer with the one or more user-selected features as input features. - View Dependent Claims (10, 11, 12, 13, 14, 15)
-
-
16. One or more computer-readable media having embodied thereon computer-usable instructions that, when executed, facilitate a method of interactive feature selection for machine learning, the method comprising:
-
providing a first set of data items, wherein one or more of the data items have been previously labeled as examples of a particular class of data item; utilizing a classifier to determine predicted labels for one or more of the data items; identifying one or more data items having a discrepancy between a previous label and a predicted label; presenting via a user interface an indication of the one or more data items having the discrepancy between the previous label and the predicted label, wherein the user interface includes a feature-selection interface configured to receive a user selection of one or more features that are usable as input features to train the classifier; receiving a search query via the search interface; executing the search query on the first set of data items, wherein search results are generated; presenting the search results to the user; receiving a user input that selects the search query as a first feature for training the classifier; receiving, via the user interface, a user input that selects a dictionary as a second feature for training the classifier, wherein the dictionary includes words that define a concept that corresponds to the second feature; training the classifier with the search query and the dictionary as input features; utilizing the classifier to determine new predicted labels for one or more of the data items; identifying one or more data items having a discrepancy between a previous label and a new predicted label; presenting via the user interface an indication of the one or more data items having the discrepancy between the previous label and the new predicted label; receiving, via the user interface, a user selection of one or more features; and training the classifier with the one or more user-selected features as input features. - View Dependent Claims (17, 18, 19, 20)
-
Specification