AUTOMATED COLLECTIVE TERM AND PHRASE INDEX

• US 20160085742A1
• Filed: 09/23/2015
• Published: 03/24/2016
• Est. Priority Date: 09/23/2014
• Status: Active Grant

• Current Assignee: Social Communications Company
• Original Assignee: Kaybus, Inc. (Prysm Systems, Inc.)
• Inventors: Mahmud, Gazi, Banda, Seenu, Liang, Deanna
• Granted Patent: US 9,864,741 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking

Abstract

Knowledge automation techniques may include selecting a knowledge element from a knowledge corpus of an enterprise for extraction of n-grams, and deriving a term vector comprising terms in the knowledge element. Based at least on a frequency of occurrence of each term in the knowledge element, key terms are identified in the term vector. Thereafter, the identified key terms are used to extract one or more n-grams from the knowledge element. Each of the extracted n-grams is scored as a function of at least a frequency of occurrence of each of the n-grams across the knowledge corpus of the enterprise, and based on the scoring, one or more of the n-grams is added to a collective term and phrase index.

Claims

1. A method comprising:

selecting, by a computing system, a knowledge element from a knowledge corpus of an enterprise for extraction of n-grams;

deriving, by a computing system, a term vector comprising terms in the knowledge element;

identifying, by the computing system, key terms in the term vector based at least on a frequency of occurrence of each term in the knowledge element;

extracting, by the computing system, n-grams using the identified key terms;

scoring, by the computing system, each of the extracted n-grams as a function of at least a frequency of occurrence of each of the n-grams across the knowledge corpus of the enterprise; and

adding, by the computing system, one or more of the extracted n-grams to an index based on the scoring.

2. The method of claim 1, wherein the deriving the term vector includes:

identifying each term in the knowledge element;

determining a frequency of occurrence of each term in the knowledge element;

determining a position of each occurrence of each term in the knowledge element; and

modeling the terms identified in the knowledge element as the term vector,

wherein a value for each term in the term vector is calculated as a function of at least the frequency of occurrence of the term and the position of each occurrence of the term in the knowledge element.

3. The method of claim 2, wherein the deriving the term vector further includes performing natural language processing on the terms in the knowledge element, and filtering the terms in the knowledge element based on the natural language processing.

4. The method of claim 1, wherein extracting the one or more n-grams using the identified key terms includes:

identifying one or more terms adjacent to each key term in the knowledge element;

performing natural language processing on the one or more terms adjacent to each key term and the key terms;

calculating a probability of the one or more terms adjacent to each key term in the knowledge element as preceding or following the key term based on a function of the natural language processing;

when the probability of the one or more terms being adjacent to the key term is greater than minimum threshold probability, extracting an n-gram comprising the one or more terms and the key term; and

when the probability of the one or more terms being adjacent to the key term is less than the minimum threshold probability, extracting an n-gram comprising only the key term.

5. The method of claim 4, wherein the calculating the probability of the one or more terms adjacent to each key term as preceding or following the key term in the knowledge element as preceding or following the key term is based on the function of the natural language processing and a frequency of occurrence of the one or more terms adjacent to each key term.

6. The method of claim 1, wherein the scoring each of the extracted n-grams is a function of the frequency of occurrence of the n-gram, a recency of the n-gram, and a commonality of the n-gram across the knowledge corpus of the enterprise.

7. The method of claim 1, wherein the adding the one or more of the extracted n-grams to the index includes:

determining a total number of n-grams extracted for the knowledge element;

determining a top percentage of the n-grams; and

adding the top percentage of the n-grams to the index.

8. The method of claim 1, wherein the adding the one or more of the extracted n-grams to the index includes:

setting a minimum threshold score;

determining whether the score for each of the extracted n-grams is above the minimum threshold score; and

when the score for an n-gram is above the minimum threshold score, adding the n-gram to the index.

9. The method of claim 1, wherein the index is a corporate dictionary comprising a set of n-grams that identify each knowledge element within the knowledge corpus of the enterprise, and the set of n-grams comprises the one or more of the extracted n-grams added to the index.

10. A non-transitory computer-readable storage memory storing a plurality of instructions executable by one or more processors, the plurality of instructions comprising:

instructions that cause the one or more processors to select a knowledge element from a knowledge corpus of an enterprise for extraction of n-grams;

instructions that cause the one or more processors to identify key terms in a term vector associated with the knowledge element based at least on a frequency of occurrence of each term in the knowledge element;

instructions that cause the one or more processors to calculate a probability of one or more terms adjacent to each key term in the knowledge element as preceding or following the key term based on a function of natural language processing;

instructions that cause the one or more processors to extract an n-gram comprising the one or more terms and the key term when the probability of the one or more terms being adjacent to the key term is greater than a minimum threshold probability;

instructions that cause the one or more processors to extract an n-gram comprising only the key term when the probability of the one or more terms being adjacent to the key term is less than the minimum threshold probability;

instructions that cause the one or more processors to score each of the extracted n-grams as a function of at least a frequency of occurrence of each of the n-grams across the knowledge corpus of the enterprise; and

instructions that cause the one or more processors to add one or more of the extracted n-grams to an index based on the scoring.

11. The non-transitory computer-readable storage memory of claim 10, wherein the plurality of instructions further comprise:

instructions that cause the one or more processors to identify each term in the knowledge element;

instructions that cause the one or more processors to determine a frequency of occurrence of each term in the knowledge element;

instructions that cause the one or more processors to determine a position of each occurrence of each term in the knowledge element; and

instructions that cause the one or more processors to model the terms identified in the knowledge element as the term vector,

wherein a value for each term in the term vector is calculated as a function of at least the frequency of occurrence of the term and the position of each occurrence of the term in the knowledge element.

12. The non-transitory computer-readable storage memory of claim 11, wherein the plurality of instructions further comprise:

instructions that cause the one or more processors to perform natural language processing on the terms in the knowledge element; and

instructions that cause the one or more processors to filter the terms in the knowledge element based on the natural language processing.

13. The non-transitory computer-readable storage memory of claim 10, wherein the calculating the probability of the one or more terms adjacent to each key term as preceding or following the key term in the knowledge element as preceding or following the key term is based on the function of the natural language processing and a frequency of occurrence of the one or more terms adjacent to each key term.

14. The non-transitory computer-readable storage memory of claim 10, wherein the scoring each of the extracted n-grams is a function of the frequency of occurrence of the n-gram, a recency of the n-gram, and a commonality of the n-gram across the knowledge corpus of the enterprise.

15. The non-transitory computer-readable storage memory of claim 10, wherein the adding the one or more of the extracted n-grams to the index includes:

determining a total number of n-grams extracted for the knowledge element;

determining a top percentage of the n-grams; and

adding the top percentage of the n-grams to the index.

16. The non-transitory computer-readable storage memory of claim 10, wherein the adding the one or more of the extracted n-grams to the index includes:

setting a minimum threshold score;

determining whether the score for each of the extracted n-grams is above the minimum threshold score; and

when the score for an n-gram is above the minimum threshold score, adding the n-gram to the index.

17. The non-transitory computer-readable storage memory of claim 10, wherein the index is a corporate dictionary comprising a set of n-grams that identify each knowledge element within the knowledge corpus of the enterprise, and the set of n-grams comprises the one or more of the extracted n-grams added to the index.

18. A system comprising:

one or more processors; and

a memory coupled with and readable by the one or more processors, the memory configured to store a set of instructions which, when executed by the one or more processors, causes the one or more processors to;

select a knowledge element from a knowledge corpus of an enterprise for extraction of n-grams;

derive a term vector comprising terms in the knowledge element;

identify key terms in the term vector based at least on a frequency of occurrence of each term in the knowledge element;

extract n-grams using the identified key terms;

score each of the extracted n-grams as a function of at least a frequency of occurrence of each of the n-grams across the knowledge corpus of the enterprise; and

add one or more of the extracted n-grams to an index based on the scoring.

19. The system claim 18, wherein the extracting the one or more n-grams using the identified key terms includes:

identifying one or more terms adjacent to each key term in the knowledge element;

performing natural language processing on the one or more terms adjacent to each key term and the key terms;

calculating a probability of the one or more terms adjacent to each key term in the knowledge element as preceding or following the key term based on a function of the natural language processing;

when the probability of the one or more terms being adjacent to the key term is greater than minimum threshold probability, extracting an n-gram comprising the one or more terms and the key term; and

when the probability of the one or more terms being adjacent to the key term is less than the minimum threshold probability, extracting an n-gram comprising only the key term.

20. The system claim 18, wherein the one or more processors are further caused to provide an interface through which a user can interact with the index, the interface including links to the knowledge element that includes the key terms for the one or more n-grams added to the index.