COMPUTER-IMPLEMENTED SYSTEMS AND METHODS FOR TAXONOMY DEVELOPMENT

• US 20150317390A1
• Filed: 07/13/2015
• Published: 11/05/2015
• Est. Priority Date: 12/16/2011
• Status: Active Grant

• Current Assignee: SAS Institute Incorporated
• Original Assignee: SAS Institute Incorporated
• Inventors: Mills, Bruce Monroe, Haws, John Courtney, Brocklebank, John Clare, Lehman, Thomas Robert
• Granted Patent: US 10,366,117 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/3334 Selection or weighting of terms from queries, including natural language queries
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 16/36 Creation of semantic tools, e.g. ontology or thesauri

Abstract

Systems and methods are provided for generating a set of classifiers. A term is identified within a document and a pre-defined threshold distance is determined. A plurality of additional terms in the document are identified, the additional terms being located within the pre-defined threshold distance of the time. A distance between the term and an additional term of the plurality of additional terms is calculated. A corresponding weight for the calculated distance is determined using a proximity weighting scheme. A score for the additional term is calculated using the calculated distance and the corresponding weight. A colocation matrix is generated and a classifier determined using the colocation matrix.

Claims

1. A system, comprising:

one or more processors;

one or more non-transitory computer readable storage mediums containing instructions to cause the one or more processors to perform operations including;

identifying a term within a document;

determining a pre-defined threshold distance;

identifying a plurality of additional terms in the document, wherein the plurality of additional terms are located within the pre-defined threshold distance of the term;

calculating a distance between the term and an additional term of the plurality of additional terms;

determining a corresponding weight for the calculated distance, wherein determining the corresponding weight uses a proximity weighting scheme;

calculating a score for the additional term using the calculated distance and the corresponding weight;

generating a colocation matrix including a plurality of rows, wherein the colocation matrix is generated using the term, the plurality of additional terms, and the score; and

determining a classifier for the document using the colocation matrix.

2. The system of claim 1, wherein the calculated distance corresponds to a number of sentences from the term.

3. The system of claim 1, wherein the proximity weighting scheme includes a decaying weight value, and wherein the decaying weight value decreases as the distance between the term and the additional term increases.

4. The system of claim 1, containing further instructions to cause the one or more processors to perform further operations including:

sorting the plurality of rows of the colocation matrix to generate an ordered colocation matrix, wherein the sorting is according to a corresponding distance between the additional term and the term; and

determining a set of candidate classifiers using the colocation matrix, wherein the classifier is determined from the set of candidate classifiers.

5. The system of claim 1, containing further instructions to cause the one or more processors to perform further operations including:

standardizing each of the plurality of rows by dividing values in each row by a sum of the values in the row.

6. The system of claim 1, containing further instructions to cause the one or more processors to perform further operations including:

identifying a seed row of the plurality of rows, wherein the seed row is associated with a particular term within the document;

determining a distance metric by comparing rows of the colocation matrix to the seed row using a clustering operation; and

generating a set of classifiers for the particular term, wherein the set of classifiers are determined using the distance metrics, and, wherein the classifier is determined from the set of candidate classifiers.

7. The system of claim 6, wherein the distance metric for a particular candidate classifier of the set of classifiers is determined based on a Euclidean distance calculation using values from a row in the colocation matrix and corresponding values in the seed row in the colocation matrix.

8. The system of claim 6, wherein candidate classifiers having corresponding distance metrics within a distance threshold are selected for inclusion in the set of classifiers for the particular term.

9. The system of claim 1, wherein the set of classifiers is provided to a downstream computer for processing.

10. The system of claim 1, wherein the set of classifiers is limited to nouns, verbs, or adjectives.

11. The system of claim 1, containing further instructions to cause the one or more processors to perform further operations including:

identifying a plurality of other terms within the document;

for an individual term of the plurality of other terms;

identify a plurality of other terms that occur within the pre-defined threshold distance of the individual term;

calculate a distance measurement between the individual term and a term of the plurality of other terms;

determine an associated weight for the distance measurement, wherein determining the associated weight uses the proximity weighting scheme; and

calculate a corresponding score for the other term using the distance measurement and the associated weight.

12. A computer program product, tangibly embodied in a non-transitory machine readable storage medium, including instructions operable to cause a data processing apparatus to:

identify a term within a document;

determine a pre-defined threshold distance;

identify a plurality of additional terms in the document, wherein the plurality of additional terms are located within the pre-defined threshold distance of the term;

calculate a distance between the term and an additional term of the plurality of additional terms;

determine a corresponding weight for the calculated distance, wherein determining the corresponding weight uses a proximity weighting scheme;

calculate a score for the additional term using the calculated distance and the corresponding weight;

generate a colocation matrix including a plurality of rows, wherein the colocation matrix is generated using the term, the plurality of additional terms, and the score; and

determine a classifier for the document using the colocation matrix.

13. The computer program product of claim 12, wherein the calculated distance corresponds to a number of sentences from the term.

14. The computer program product of claim 12, wherein the proximity weighting scheme includes a decaying weight value, and wherein the decaying weight value decreases as the distance between the term and the additional term increases.

15. The computer program product of claim 12, including further instructions operable to cause a data processing apparatus to:

sort the plurality of rows of the colocation matrix to generate an ordered colocation matrix, wherein the sorting is according to a corresponding distance between the additional term and the term; and

determine a set of candidate classifiers using the colocation matrix, wherein the classifier is determined from the set of candidate classifiers.

16. The computer program product of claim 12, including further instructions operable to cause a data processing apparatus to:

standardize each of the plurality of rows by dividing values in each row by a sum of the values in the row.

17. The computer program product of claim 12, including further instructions operable to cause a data processing apparatus to:

identify a seed row of the plurality of rows, wherein the seed row is associated with a particular term within the document;

determine a distance metric by comparing rows of the colocation matrix to the seed row using a clustering operation; and

generate a set of classifiers for the particular term, wherein the set of classifiers are determined using the distance metrics, and, wherein the classifier is determined from the set of candidate classifiers.

18. The computer program product of claim 17, wherein the distance metric for a particular candidate classifier of the set of classifiers is determined based on a Euclidean distance calculation using values from a row in the colocation matrix and corresponding values in the seed row in the colocation matrix.

19. The computer program product of claim 17, wherein candidate classifiers having corresponding distance metrics within a distance threshold are selected for inclusion in the set of classifiers for the particular term.

20. A computer-implemented method comprising:

identifying a term within a document;

determining a pre-defined threshold distance;

identifying a plurality of additional terms in the document, wherein the plurality of additional terms are located within the pre-defined threshold distance of the term;

calculating a distance between the term and an additional term of the plurality of additional terms;

determining a corresponding weight for the calculated distance, wherein determining the corresponding weight uses a proximity weighting scheme;

calculating a score for the additional term using the calculated distance and the corresponding weight;

generating a colocation matrix including a plurality of rows, wherein the colocation matrix is generated using the term, the plurality of additional terms, and the score; and

determining a classifier for the document using the colocation matrix.