ASYNCHRONOUS METHODS OF DATA CLASSIFICATION USING CHANGE JOURNALS AND OTHER DATA STRUCTURES
First Claim
1. A system for generating a database of metadata that describes data objects, the system comprising:
- a computing device including—
a data storage device that stores data, wherein the data includes data objects and metadata that describes the data objects; and
a system that indexes data objects and metadata stored on the data storage device; and
a data agent configured to—
scan a portion of the system that indexes data objects and metadata during one or more time periods, by—
requesting that the system that indexes data objects and metadata provide metadata related to a data object;
receiving the requested metadata; and
storing the received metadata in a database of metadata;
populate one or more interim change tables, by—
registering for a change notification service that provides change notifications of interactions with data objects stored on the data storage device;
receiving a change notification of an interaction with a data object stored on the data storage device; and
updating an interim change table to store information provided by the received change notification in the interim change table;
identify interactions with data objects that are not reflected by the information stored in the one or more interim change tables;
retrieve information regarding the identified interactions with data objects;
update the one or more interim change tables to store the retrieved information; and
update the database of metadata by—
analyzing the information stored in the one or more interim change tables;
generating or providing metadata based on the analyzed information; and
,adding the generated or provided metadata to the database of metadata;
wherein the data agent is configured to populate the one or more interim change tables substantially simultaneously with scanning the portion of the system that indexes data objects and metadata.
4 Assignments
0 Petitions
Accused Products
Abstract
Aspects of the present invention are generally concerned with systems and methods for generating a database of metadata that describes system data and storage operations. The database of metadata may be referred to herein as a “metabase.” For example, to generate a metabase, a data agent may traverse a file system to obtain certain characteristics of data managed by the file system while substantially simultaneously detecting and recording change notifications. These actions may be performed even if the actions of the data agent are interrupted one or more times during the traversal of the file system. The data agent may process the characteristics and change notifications to generate and update a metabase. Once formed, the metabase may be consulted to determine changes in system data rather than determining the changes by scanning the data files themselves.
-
Citations
17 Claims
-
1. A system for generating a database of metadata that describes data objects, the system comprising:
-
a computing device including— a data storage device that stores data, wherein the data includes data objects and metadata that describes the data objects; and a system that indexes data objects and metadata stored on the data storage device; and a data agent configured to— scan a portion of the system that indexes data objects and metadata during one or more time periods, by— requesting that the system that indexes data objects and metadata provide metadata related to a data object; receiving the requested metadata; and storing the received metadata in a database of metadata; populate one or more interim change tables, by— registering for a change notification service that provides change notifications of interactions with data objects stored on the data storage device; receiving a change notification of an interaction with a data object stored on the data storage device; and updating an interim change table to store information provided by the received change notification in the interim change table; identify interactions with data objects that are not reflected by the information stored in the one or more interim change tables; retrieve information regarding the identified interactions with data objects; update the one or more interim change tables to store the retrieved information; and update the database of metadata by— analyzing the information stored in the one or more interim change tables; generating or providing metadata based on the analyzed information; and
,adding the generated or provided metadata to the database of metadata; wherein the data agent is configured to populate the one or more interim change tables substantially simultaneously with scanning the portion of the system that indexes data objects and metadata. - View Dependent Claims (2, 3, 17)
-
-
4. A method for generating a database of metadata that describes data objects, the method comprising:
-
interfacing with a file system associated with data stored on a storage device, wherein the file system provides access to data objects and metadata included within the data stored on the storage device, and wherein the interfacing includes performing, for each of multiple data objects stored on the storage device— requesting that the file system provide metadata related to the data object; receiving the requested metadata; and storing the received metadata within the database of metadata; populating one or more data structures, wherein populating one or more data structures includes— registering for a change notification service; receiving a change notification, wherein the change notification includes information regarding an interaction with a data object; and updating the one or more data structures to include information provided by the received change notification; and determining if a time required for interfacing with the file system exceeds a first time period; if the required time exceeds the first time period, determining if any relevant change records were created outside of the required time; if relevant change records were created outside of the required time, retrieving information regarding the relevant change records and storing the retrieved information in the one or more data structures; and updating the metadata in the database of metadata to include information stored in the one or more one or more data structures. - View Dependent Claims (5, 6, 7, 8)
-
-
9. A system for generating a database of metadata, the system comprising:
-
a computing device including— a data storage device storing data; and a file system that facilitates access to the data stored on the data storage device; and an agent configured to— scan the file system; store information in a data structure, wherein the information relates to interactions with data stored on the data storage device; identify interactions with data stored on the data storage device that are not reflected by the information stored in the data structure; update the data structure with information regarding interactions with data stored on the data storage device that are not reflected by the information stored in the data structure; access the information stored in the data structure; and based on the accessed information, update metadata in a database of metadata. - View Dependent Claims (10, 11, 12)
-
-
13. One or more computer-readable data storage media storing instructions that, when executed by a computer, perform a method for generating a database of metadata for data files stored on a data storage device, the method comprising:
-
scanning a file system associated with the data storage device, wherein the file system indexes the data files stored on the data storage device; storing information in a data structure, wherein the information relates to interactions with the data files stored on the data storage device; identifying interactions with the data files stored on the data storage device that are not reflected by the information stored in the data structure; updating the data structure with information regarding interactions with the data files stored on the data storage device that are not reflected by the information stored in the data structure; accessing the information stored in the data structure; and based on the accessed information, updating metadata in a database of metadata.
-
-
14. A system for generating a database of metadata that describes data objects, the system comprising:
-
means for scanning a file system associated with a computing device to obtain information, wherein the file system provides access to data stored on a data storage device; means for populating a database of metadata with the obtained information; means for storing information in one or more data structures, wherein the stored information relates to data interactions with the data stored on the data storage device; means for identifying data interactions that are not reflected by the information stored in the one or more data structures; means for updating the one or more data structures with information regarding interactions with data stored on the data storage device that are not reflected by the information stored in the one or more data structures; means for accessing the information stored in the one or more data structures; and means for updating metadata in the database of metadata based on the accessed information. - View Dependent Claims (15, 16)
-
Specification