System and method for identifying phrases in text

• US 8,949,111 B2
• Filed: 12/14/2011
• Issued: 02/03/2015
• Est. Priority Date: 12/14/2011
• Status: Active Grant

• Current Assignee: Brainspace Corporation (Reveal Data Corp.)
• Original Assignee: Brainspace Corporation (Reveal Data Corp.)
• Inventors: Jakubik, Paul A.
• Primary Examiner(s): Guerra-Erazo, Edgar
• Application Number:
  US13/326,120
• Publication Number:
  US 20130158979A1
• Time in Patent Office: 1,147 Days
• Field of Search: 704 1- 10
• US Class Current: 704/9
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06F 40/295 Named entity recognition
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30

Abstract

A method includes accessing text that includes a plurality of words, tagging each of the plurality of words with one of a plurality of parts of speech (POS) tags, and creating a plurality of tokens, each token comprising one of the plurality of words and its associated POS tag. The method further includes clustering one or more of the created tokens into a chunk of tokens, the one or more tokens clustered into the chunk of tokens based on the POS tags of the one or more tokens, and forming a phrase based on the chunk of tokens, the phrase comprising the words of the one or more tokens clustered into the chunk of tokens.

Claims

1. A system for discovering latent relationships in data, the system comprising:

one or more memory units configured to store a plurality of noun tags; and

one or more processing units operable to;

access a phrase comprising a plurality of nouns;

create a plurality of tokens, each token comprising one of the plurality of nouns and an associated noun tag of the plurality of noun tags;

cluster one or more of the created tokens into a chunk of tokens, the one or more tokens clustered into the chunk of tokens based on the noun tags of the one or more tokens;

form one or more sub-phrases of the accessed phrase based on the chunk of tokens, the one or more sub-phrases comprising the nouns of the one or more tokens clustered into the chunk of tokens; and

perform Latent Semantic Analysis (LSA) using the one or more sub-phrases.

2. The system of claim 1, wherein the plurality of noun tags comprises:

a noun tag for a proper noun that is a geographic name;

a noun tag for a proper noun that is a name of a person; and

a noun tag for a proper noun that can be both a geographic name and a name of a person.

3. The system of claim 1, wherein clustering one or more of the created tokens into the chunk of tokens comprises clustering tokens having the same noun tags into the chunk of tokens.

10. The system of claim 1, wherein the phrase is an undifferentiated block of nouns.

11. The system of claim 2, wherein the plurality of noun tags further comprises a noun initial tag for an initial that is part of a person'"'"'s name.

12. The system of claim 2, wherein clustering one or more of the created tokens into the chunk of tokens comprises utilizing a set of rules stored in the one or more memory units.

13. The system of claim 11, wherein the set of rules comprises a rule instructing that if the plurality of tokens includes two adjacent tokens with identified noun tags in a specified order, replace the noun tags with another noun tag that contains the noun tags of the two adjacent tokens.

14. The system of claim 11, wherein the set of rules comprises a rule instructing that if the plurality of tokens includes three adjacent tokens with identified noun tags in a specified order, replace the noun tags with another noun tag that contains the noun tags of the three adjacent tokens.

4. A computer-implemented method system for discovering latent relationships in data, the method comprising:

accessing a phrase by a processing system, the text comprising a plurality of nouns;

creating, by the processing system, a plurality of tokens, each token comprising one of the plurality of nouns and an associated noun tag of a plurality of noun tags;

clustering, by the processing system, one or more of the created tokens into a chunk of tokens, the one or more tokens clustered into the chunk of tokens based on the noun tags of the one or more tokens;

forming, by the processing system, one or more sub-phrases of the accessed phrase based on the chunk of tokens, the one or more sub-phrases comprising the nouns of the one or more tokens clustered into the chunk of tokens; and

performing, by the processing system, Latent Semantic Analysis (LSA) using the one or more sub-phrases.

5. The computer-implemented method of claim 4, wherein the plurality of noun tags comprises:

a noun tag for a proper noun that is a geographic name;

a noun tag for a proper noun that is a name of a person; and

a noun tag for a proper noun that is can be both a geographic name and a name of a person.

6. The computer-implemented method of claim 4, wherein clustering one or more of the created tokens into the chunk of tokens comprises clustering tokens having the same noun tags into the chunk of tokens.

15. The computer-implemented method of claim 4, wherein the phrase is an undifferentiated block of nouns.

16. The computer-implemented method of claim 5, wherein the plurality of noun tags further comprises a noun initial tag for an initial that is part of a person'"'"'s name.

17. The computer-implemented method of claim 16, wherein clustering one or more of the created tokens into the chunk of tokens comprises utilizing a set of rules stored in the one or more memory units.

7. A non-transitory computer-readable medium comprising software, the software when executed by one or more processing units operable to perform operations comprising:

accessing a phrase comprising a plurality of nouns;

creating a plurality of tokens, each token comprising one of the plurality of nouns and an associated noun tag of a plurality of noun tags;

clustering one or more of the created tokens into a chunk of tokens, the one or more tokens clustered into the chunk of tokens based on the noun tags of the one or more tokens;

forming one or more sub-phrases of the accessed phrase based on the chunk of tokens, the one or more sub-phrases comprising the nouns of the one or more tokens clustered into the chunk of tokens; and

performing Latent Semantic Analysis (LSA) using the one or more sub-phrases.

8. The non-transitory computer-readable medium of claim 7, wherein the plurality of noun tags comprises:

a noun tag for a proper noun that is a geographic name;

a noun tag for a proper noun that is a name of a person; and

a noun tag for a proper noun that is can be both a geographic name and a name of a person.

9. The non-transitory computer-readable medium of claim 7, wherein clustering one or more of the created tokens into the chunk of tokens comprises clustering tokens having the same noun tags into the chunk of tokens.

18. The non-transitory computer-readable medium of claim 7, wherein the phrase is an undifferentiated block of nouns.

19. The non-transitory computer-readable medium of claim 8, wherein the plurality of noun tags further comprises a noun initial tag for an initial that is part of a person'"'"'s name.

20. The non-transitory computer-readable medium of claim 19, wherein clustering one or more of the created tokens into the chunk of tokens comprises utilizing a set of rules stored in the one or more memory units.