Auto indexing with customizable metadata

• US 10,348,829 B2
• Filed: 08/15/2016
• Issued: 07/09/2019
• Est. Priority Date: 08/15/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Iljazi, Ilir
• Primary Examiner(s): Bullock, Joshua
• Application Number:
  US15/237,564
• Publication Number:
  US 20180046728A1
• Time in Patent Office: 1,058 Days
• Field of Search: 707741
• CPC Class Codes: G06F 16/2272 Management thereof
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

An auto indexing method includes identifying a vault of a dispersed storage network for an auto indexing function. The method continues by obtaining search criteria for the auto indexing function. The method further includes scanning the plurality of metadata files to identify metadata files of the plurality of metadata files having at least one of the objective metadata and the subjective metadata corresponding to the search criterion. The method further includes updating an index file that includes the identity of the one or more metadata files of the plurality of metadata files having the at least one of the objective metadata and the subjective metadata corresponding to the search criterion and updating an index structure based on the index file. The method further includes utilizing the index structure to search for one or more data objects having one or more of the search criterion of the search criteria.

Claims

1. An auto indexing method for execution by a computing device of a dispersed storage network (DSN), the method comprises:

identifying a vault of the DSN for an auto indexing function, wherein the vault stores a plurality of data objects and stores a plurality of metadata files corresponding to the plurality of data objects, wherein a metadata file of the plurality of metadata files includes objective metadata and subjective metadata;

obtaining search criteria for the auto indexing function;

for each of at least some search criterion of the search criteria;

scanning the plurality of metadata files to identify one or more metadata files of the plurality of metadata files having at least one of the objective metadata and the subjective metadata corresponding to the search criterion;

updating an index file of an index structure to include the identity of the one or more metadata files of the plurality of metadata files having the at least one of the objective metadata and the subjective metadata corresponding to the search criterion; and

updating the index structure based on the index file; and

utilizing the updated index structure to search for one or more data objects having one or more of the search criterion of the search criteria; and

when identifying the one or more data objects having the one or more of the search criterion;

modifying a corresponding one or more metadata files of the identified one or more data objects to include the identity of the one or more at least one of the objective metadata and the subjective metadata corresponding to the search criterion; and

updating one or more index files of the updated index structure based on the modified one or more metadata files.

2. The method of claim 1 further comprises:

processing a request to modify the subjective metadata of a metadata file associated with a particular data object of the plurality of data objects.

3. The method of claim 1, wherein the objective metadata comprises one or more of:

a data object name;

a data object author;

a timestamp; and

DSN storage information.

4. The method of claim 1, wherein the subjective metadata comprises:

one or more user defined key-words.

5. The method of claim 1 further comprises:

scanning a data object of the plurality of data objects based on the search criterion, wherein the metadata file of the data object currently does not include objective metadata or subjective metadata that corresponds to the search criterion;

when the scanning of the data object based on the search criterion is favorable, updating the metadata file associated with the data object to include the search criterion; and

adding identity of the metadata file to the index file.

6. The method of claim 1 further comprises:

scanning an updated plurality of metadata files to identify another one or more updated metadata files of the updated plurality of metadata files having at least one of the objective metadata and the subjective metadata corresponding to the search criterion, wherein the updated plurality of metadata files corresponds to an updated plurality of data objects; and

updating the index file to include the identity of the one or more updated metadata files of the updated plurality of metadata files having the at least one of the objective metadata and the subjective metadata corresponding to the search criterion.

7. The method of claim 1 further comprises:

identifying the vault in response to enablement of the auto indexing function.

8. The method of claim 1 further comprises:

dispersed storage error encoding the index file to produce a set of encoded index file slices; and

storing the set of encoded index file slices in storage units of the DSN.

9. The method of claim 1, wherein the updating the index file comprises:

creating the index file when the scanning is based on a first instance of the search criterion.

10. A computing device comprises:

an interface;

memory; and

a processing module operably coupled to the interface and the memory, wherein the processing module is operable to;

identify a vault of a dispersed storage network (DSN) for an auto indexing function, wherein the vault stores a plurality of data objects and stores a plurality of metadata files corresponding to the plurality of data objects, wherein a metadata file of the plurality of metadata files includes objective metadata and subjective metadata;

obtain search criteria for the auto indexing function;

for each of at least some search criterion of the search criteria;

scan the plurality of metadata files to identify one or more metadata files of the plurality of metadata files having at least one of the objective metadata and the subjective metadata corresponding to the search criterion;

update an index file of an index structure to include the identity of the one or more metadata files of the plurality of metadata files having the at least one of the objective metadata and the subjective metadata corresponding to the search criterion; and

update the index structure based on the index file; and

utilize the updated index structure to search for one or more data objects having one or more of the search criterion of the search criteria; and

when identifying the one or more data objects having the one or more of the search criterion;

modify a corresponding one or more metadata files of the identified one or more data objects to include the identity of the one or more at least one of the objective metadata and the subjective metadata corresponding to the search criterion; and

update one or more index files of the updated index structure based on the modified one or more metadata files.

11. The computing device of claim 10, wherein the processing module is further operable to:

process a request to modify the subjective metadata of a metadata file associated with a particular data object of the plurality of data objects.

12. The computing device of claim 10, wherein the objective metadata comprises one or more of:

a data object name;

a data object author;

a timestamp; and

DSN storage information.

13. The computing device of claim 10, wherein the subjective metadata comprises:

one or more user defined key-words.

14. The computing device of claim 10, wherein the processing module is further operable to:

scan a data object of the plurality of data objects based on the search criterion, wherein the metadata file of the data object currently does not include objective metadata or subjective metadata that corresponds to the search criterion;

when the scanning of the data object based on the search criterion is favorable, update the metadata file associated with the data object to include the search criterion; and

add identity of the metadata file to the index file.

15. The computing device of claim 10, wherein the processing module is further operable to:

scan an updated plurality of metadata files to identify another one or more updated metadata files of the updated plurality of metadata files having at least one of the objective metadata and the subjective metadata corresponding to the search criterion, wherein the updated plurality of metadata files corresponds to an updated plurality of data objects; and

update the index file to include the identity of the one or more updated metadata files of the updated plurality of metadata files having the at least one of the objective metadata and the subjective metadata corresponding to the search criterion.

16. The computing device of claim 10, wherein the processing module is further operable to:

identify the vault in response to enablement of the auto indexing function.

17. The computing device of claim 10, wherein the processing module is further operable to:

dispersed storage error encode the index file to produce a set of encoded index file slices; and

store the set of encoded index file slices in storage units of the DSN.

18. The computing device of claim 10, wherein the processing module is further operable to update the index file by:

creating the index file when the scan is based on a first instance of the search criterion.