DETERMINING CONCEPT BLOCKS BASED ON CONTEXT

• US 20150046152A1
• Filed: 08/06/2014
• Published: 02/12/2015
• Est. Priority Date: 08/08/2013
• Status: Abandoned Application

• Current Assignee: Quryon., Inc.
• Original Assignee: Quryon., Inc.
• Inventors: LEE, Woo Joo
• Application Number:
  US14/452,936
• Publication Number:
  US 20150046152A1
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing
• CPC Class Codes: G06F 16/35 Clustering; Classification

Abstract

A method for generating a set of concept blocks is presented, wherein the concept blocks are words in a corpus of documents that can be processed to extract trends, build an efficient inverted search index, or generate a summary report of the content. The method entails generating a plurality of target words from the corpus, determining context strings for the target words, obtaining pattern types that are based on number of words and position of words relative to the target words, and assigning weights to each of the context strings having a particular pattern type. The target words are then expressed as vectors that reflect the weights of the context strings. The vectors are compared and grouped into clusters based on similarity. Target words in the resulting clusters are concept blocks. A subgroup of clusters may be selected for another iteration of the process to catch new concept blocks.

Claims

1. A method of generating a set of concept blocks, the method comprising:

accessing a corpus of documents;

generating a plurality of target words from the corpus;

determining context strings for the target words, wherein the context strings include words that are adjacent to the target words;

obtaining pattern types, wherein the pattern types are based on number of words and position of words relative to the target words;

assigning weights to each of the context strings, such that a weight of a context string having a particular

2. The method of claim 1, wherein the selecting of the subgroup of clusters comprises filtering the clusters based on at least one of frequency threshold and size threshold.

3. The method of claim 1 further comprising expanding words in the subgroup of the clusters by aggregating context strings for the subgroup of clusters.

4. The method of claim 3, wherein the selected subgroup of clusters are original subgroup of clusters, and expanding the selected subgroup of clusters comprises:

setting the concept blocks as a new set of target words;

assigning weights to context strings for the new set of target words;

generating a clustered vector by expressing the new set of target words as a single vector; and

comparing the clustered vector against another clustered vector to produce a set of new clusters based on similarity.

5. The method of claim 4 further comprising extracting sparse words from the new clusters, wherein the sparse words were not present in the original subgroup of clusters.

6. The method of claim 1 further comprising generating target words using n-grams in the content, n being an integer.

7. The method of claim 6, wherein n-grams include at least one of unigram, 2-gram, 3-gram, and 4-gram sequences of words.

8. The method of claim 1, wherein context words include semantically significant words and semantically insignificant words.

9. The method of claim 1 wherein the pattern types comprise overlaps such that two pattern types may include the same string.

10. The method of claim 1, wherein expressing the target words as vectors comprises mapping the context strings on a multi-axis space, wherein one axis corresponds to one context string.

11. The method of claim 1, wherein comparing the vectors to obtain a similarity measure comprises comparing cosine similarity measures.

12. The method of claim 1 further comprising syntactically parsing the words in the plurality of clusters to identify a relation between clusters.

13. The method of claim 12 further comprising:

associating each of the clusters in the subgroup with a subject; and

generating a summary including subjects and members associated with the subjects based on the relation between clusters.

14. The method of claim 1 wherein the documents include SNS posts, further comprising generating keywords based on the concept blocks.

15. The method of claim 14 further comprising extracting keywords from documents that are associated with an individual SNS user account to generate a summary report including subjects of discussion and members of the subjects.

16. The method of claim 15 further comprising indicating in the summary report whether each subject and each member is found in a post that has already been viewed or an unviewed post.

17. The method of claim 14 further comprising building a search index using the keywords for the individual SNS user account, wherein the search index includes an inverted index that is configured to identify one or more posts containing discussions related to a query word.

18. The method of claim 14 further comprising:

extracting keywords from the corpus of documents; and

counting occurrences of some or all of the keywords to determine a trend based on the occurrences.

19. The method of claim 18 further comprising providing information about the trend to a third party.

20. A method of generating a summary of discussions in SNS posts, comprising:

obtaining a categorically arranged set of keywords from a data storage;

identifying occurrences of the keywords in posts that are associated with an SNS user account; and

arranging the identified keywords according to their categories to generate a summary report including subjects of discussion and members of the subjects.

21. The method of claim 20 further comprising indicating in the summary report whether each subject and each member is found in a post that has already been viewed or an unviewed post.

22. The method of claim 20 further comprising building a search index for the individual SNS user account, wherein the search index includes an inverted index that is configured to identify one or more posts containing discussions related to a keyword.

23. The method of claim 20 further comprising counting occurrences of some or all of the keywords to determine a trend based on the occurrences.

24. The method of claim 23 further comprising providing information about the trend to a third party.

25. A method of generating a set of keywords, the method comprising:

accessing a corpus of documents;

generating target words from the corpus of documents;

determining context strings for the target words, wherein the context strings include words that are adjacent to the target words;

obtaining pattern types, wherein the pattern types are based on number of words and position of words relative to the target words;

assigning weights to each of the context strings, such that a weight of a context string having a particular