MANAGING A SET OF DATA

• US 20160124946A1
• Filed: 10/31/2014
• Published: 05/05/2016
• Est. Priority Date: 10/31/2014
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Aaron, Andrew S., Branch, Joel W., Guo, Shang Q., Lenchner, Jonathan, Mazzella, Daniel A., Mukherjee, Maharaj, Nelson, John C.
• Granted Patent: US 9,705,972 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46

Abstract

Aspects of the disclosure include managing a set of data associated with a corpus. By analyzing the corpus, a domain is established to characterize the subject matter of the set of data. A user identifier is generated for a portion of the set of data. Based upon a credibility computation, a quality factor for a portion of the set of data is determined. The credibility computation includes using both the domain and the user identifier to determine the quality factor for the portion of the set of data. The quality factor for the portion of the set of data is compared with a threshold. In response to a quality factor for a portion of the set of data exceeding the threshold, the portion of the set of data is selected.

Claims

1. A computer implemented method for managing a set of data associated with a corpus, comprising:

establishing a domain to characterize the subject matter of the set of data by analyzing the corpus;

generating, from the set of data, a user identifier for a portion of the set of data;

determining, based on a credibility computation using both the domain and the user identifier, a quality factor of the portion of the set of data; and

selecting, in response to the quality factor exceeding a quality factor threshold, a portion of the set of data.

2. The method of claim 1, wherein the set of data includes user-provided content within a corpus.

3. The method of claim 2, wherein the user-provided content includes at least one of commentary or commentator names.

4. The method of claim 1, wherein the corpus includes a user-generated file.

5. The method of claim 4, wherein the user generated file is selected from the group consisting of a news document, published document, patent document and social media document.

6. The method of claim 1, wherein analyzing the corpus includes using at least one of a web crawler technique, a pattern recognition technique, and a natural language processing technique.

7. The method of claim 1, wherein the domain to characterize the subject matter includes a topic to classify a specific data element.

8. The method of claim 1, wherein the generating includes:

extracting, from the set of data, a portion of the set of data having a common feature;

assigning the user identifier to the portion of the set of data having the common feature; and

storing the user identifier in a data repository.

9. The method of claim 8, wherein the portion of the set of data includes a social network identifier associated with the common feature.

10. The method of claim 8, wherein assigning the user identifier includes computing, using a natural language processing technique, a probability that a specific user created the portion of the set of data.

11. The method of claim 8, wherein storing the user identifier in a data repository includes mapping the user identifier to the portion of the set of data.

12. The method of claim 1, wherein the credibility computation includes:

a score value for the user identifier in the domain;

a comparison of the score value for the user identifier with a score value threshold; and

a determination of credibility for a user identifier based on the comparison.

13. The method of claim 1, wherein the quality factor is based on a quality score which indicates the extent of credibility for a user identifier within a particular subject matter.

14. The method of claim 13, wherein the quality score is computed by awarding points to a user identifier based upon historical data.

15. The method of claim 14, wherein the historical data is selected from a group consisting of at least one of:

commentator activity, social network activity, publication activity, or group association activity.

16. The method of claim 1, wherein selecting the portion of the set of data exceeding the quality factor threshold is selected from a group consisting of at least one of:

marking or displaying.

17. The method of claim 8, further comprising:

storing, in the data repository, the portion of the set of data assigned to the user identifier;

evaluating the portion of the set of data to be below a relevancy score value threshold; and

removing the portion of the set of data from the data repository.

18. The method of claim 17, wherein evaluating the portion of the set of data to be below a relevancy score value threshold includes using a technique selected from a group consisting of at least one of:

machine learning techniques, keyword techniques, or embedded link analysis techniques.

19. A computer program for managing a set of data associated with a corpus, the computer program product comprising a computer readable storage medium having instructions embodied therewith, the program instructions executable by a processor to perform a method comprising:

establishing a domain to characterize the subject matter of the set of data by analyzing the corpus;

generating, from the set of data, a user identifier for a portion of the set of data;

determining, based on a credibility computation using both the domain and the user identifier, a quality factor of the portion of the set of data; and

selecting, in response to the quality factor exceeding a quality factor threshold, a portion of the set of data.

20. A computer system for managing a set of data associated with a corpus, the computer system comprising:

a memory; and

a processor in communication with the memory, wherein the computer system is configured to perform a method, the method comprising;

establishing a domain to characterize the subject matter of the set of data by analyzing the corpus;

generating, from the set of data, a user identifier for a portion of the set of data;

determining, based on a credibility computation using both the domain and the user identifier, a quality factor of the portion of the set of data; and

selecting, in response to the quality factor exceeding a quality factor threshold, a portion of the set of data.