Incrementally retrieving data for objects to provide a desired level of detail

• US 10,521,416 B2
• Filed: 02/25/2015
• Issued: 12/31/2019
• Est. Priority Date: 08/15/2013
• Status: Active Grant

• Current Assignee: Breakwater Solutions LLC
• Original Assignee: International Business Machines Corporation
• Inventors: Bishop, Thomas P., Chee, Kevin, McCoy, Jordan R., Szalay, Jozsef, Tran, Michael T.
• Primary Examiner(s): Bromell, Alexandria Y
• Application Number:
  US14/631,244
• Publication Number:
  US 20150278340A1
• Time in Patent Office: 1,770 Days
• Field of Search: 707776, 707709, 707711, 707741, 707754, 707756, 706 25
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 16/21 Design, administration or maintenance of databases
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/24573 using data annotations, e.g. user-defined metadata
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/335 Filtering based on additional data, e.g. user or group profiles filtering in web context G06F16/9535, G06F16/9536
• CPC Class Codes: G06F 16/90332 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation

Abstract

A computer-implemented method is provided, for retrieving data and metadata according to a data harvesting criteria into an indexed repository, comprising providing a user interface allowing a user to define data harvesting criteria for adjusting a subset of data and metadata on an indexed repository. Responsive to a user utilizing the user interface by defining the data harvesting criteria, the subset of data and metadata on the indexed repository is adjusted according to the data harvesting criteria.

Claims

1. A computer-implemented method comprising:

at a processor of a computing device, generating an information set that includes a limited amount of information based on a repository index, wherein the repository index includes information about an indexed repository that includes one or more data server systems;

receiving data harvesting criteria for a data expansion operation to be performed for the information set on the indexed repository;

determining data server systems of the indexed repository to participate in the data expansion operation, wherein each of the data server systems has its own access interface;

causing each participating data server system to execute the data expansion operation against data and metadata on that particular data server system according to the data harvesting criteria while allowing the data and metadata to be left intact on that particular data server system, wherein the data harvesting criteria are refined by utilizing natural language processing and artificial intelligence to classify the data and metadata on each particular data server system;

converting a characterization included in the data harvesting criteria to adaptors with filters to retrieve the data and metadata on each particular data server system according to a user request;

transforming the repository index for the indexed repository to include additional information from the data expansion operation, wherein the transforming adds the additional information to the information set to incrementally update the limited amount of information included in the information set until a desired level of detail is attained,

wherein the updating ensures that subsequently generated information sets created based on the data and metadata subject to the data expansion operation include the additional information wherein the additional information comprises a plurality of subsets, and wherein transforming the repository index comprises;

training a classifier to learn one or more classifications based on machine learning techniques,

classifying each of the subsets into one or more learned classifications during the data expansion operation via the trained classifier, and

determining inclusion of each subset in the desired level of detail according to the classification of the subset by the trained classifier.

2. The computer-implemented method of claim 1, wherein the data harvesting criteria supports a specification selected from a group consisting of per object, per information set, metadata, container, full text, member sets, caching, attribute, classification, files, email, and servers.

3. The computer-implemented method of claim 1, further comprising:

providing support to manage subsets of the data and metadata on each particular data server system when subsets are formed based on new data harvesting criteria.

4. The computer-implemented method of claim 1, wherein the data harvesting criteria is applied to filters to access the data and metadata on each particular data server system.

5. The computer-implemented method of claim 1, wherein incrementally updating the limited amount of information included in the information set until a desired level of detail is attained includes:

excluding information beyond the desired level of detail to reduce a processing time of the processor.

6. The computer-implemented method of claim 1, wherein including the additional information in subsequently generated information sets includes:

avoiding redundant data expansion operations to reduce a processing time of the processor.