Methods and systems relating to network based storage

• US 10,516,669 B2
• Filed: 11/08/2016
• Issued: 12/24/2019
• Est. Priority Date: 11/12/2015
• Status: Active Grant

• Current Assignee: Veritas Technologies, LLC (Whitehouse Group Ltd.)
• Original Assignee: HubStor Inc.
• Inventors: Bourgeois, Geoffrey, Campbell, Greg
• Primary Examiner(s): Potratz, Daniel B
• Assistant Examiner(s): Straub, D'Arcy Winston
• Application Number:
  US15/346,094
• Publication Number:
  US 20170142113A1
• Time in Patent Office: 1,141 Days
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0622 in relation to access
• CPC Class Codes: G06F 3/0637 Permissions
• CPC Class Codes: H04L 63/10 for controlling access to devices or network resources
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Cloud storage provides for accessible interfaces, near-instant elasticity and scalability, multi-tenancy, and metered resources within a framework of distributed resources acing to provide highly fault tolerant solutions with high data durability. However, cloud storage has drawbacks and limitations which are addressed by:

a graphical user interface which portrays content ingested from third party sources in its original folder contexts and determines what folder locations each knowledge worker will see in the interface rather than how they are uploaded onto the remote storage;

a centralized determination of incremental ingestion of changes from a data source to a cloud storage repository that should be written; and

a responsive method of obtaining query results for policies that evaluate one or more clauses against a cloud storage repository containing large data sets; and

a means of maintaining/refreshing of statistics for large data sets within a cloud storage repository.

Claims

1. A method comprising:

providing a system comprising one or more microprocessors, one or more non-volatile non-transitory memories;

providing one or more network interfaces supporting electronic communications by the system to and from a first communications network and a second communications network; and

storing computer executable instructions within the one or more non-volatile non-transitory memories;

wherein

the computer executable instructions when executed by the one or more microprocessors configure the system to;

store access control information from a plurality of data sources, the access control information acquired by one or more data source connectors traversing and capturing at least one of data and objects from the plurality of data sources accessible to the system which is stored within a cloud storage repository;

synchronize the access control information with a plurality of unique security identifiers associated with the plurality of data sources where each unique security identifier relates to either a specific user of a plurality of users associated with the captured data from the plurality of data sources or a specific subset of the plurality of users associated with the captured data from the plurality of data sources;

acquire and store user identity records from a plurality of user directory sources;

synchronize the user identity records acquired from the plurality of user directory sources;

manage the plurality of unique security identifiers;

query at least one of the ownership and access rights of an entity, the entity being one of a predetermined user of the plurality of users, a predetermined subset of the plurality of users, and a unique security identifier of the plurality of security identifiers;

wherein

the querying correlates a mapping of access rights of the entity to at least one of one or more specific files contained within the captured data from the plurality of sources stored within the cloud storage repository and one or more specific objects within the objects within the captured objects from the plurality of sources stored within the cloud storage repository;

the plurality of data sources are connected to the first communications network; and

the cloud storage repository is connected to the second communications network.

2. The method according to claim 1, wherein the computer executable instructions when executed by the one or more microprocessors configure the system to:

link the cloud storage repository and the at least one of data and objects stored upon it with one or more user directory services;

wherein

each user directory service maintains a record of each at least one of a user identity and a user group associated with the at least one of the data and objects for a predetermined data source of the plurality of data sources; and

each at least one of a user identity and a user group has one or more unique security identifiers associated with each record of the at least one of the user identity and the user group.

3. The method according to claim 1, wherein the computer executable instructions when executed by the one or more microprocessors configure the system to:

correlate the one or more security identifiers associated with a first user within the plurality of security identifiers associated with the plurality of data sources;

join the access control information associated with each security identifier of the one or more security identifiers for the first user within the plurality of security identifiers associated with the plurality of data sources; and

provide a second user with a global view of data ownership and data access rights across the plurality of data sources for the first user.

4. The method according to claim 1, wherein the computer executable instructions when executed by the one or more microprocessors configure the system to:

resolve one or more relationships between a user and at least one of the one or more security identifiers of the plurality of security identifiers associated with the user upon the plurality of data sources, one or more group membership of the user upon the plurality of data sources, and the at least one of the data and the objects to which the user has access.

5. The method according to claim 4, wherein the one or more relationships are established independent of the order in which the access control information and user identity information is collected by the system.

6. The method according to claim 1, wherein the computer executable instructions when executed by the one or more microprocessors configure the system to:

establish and maintain a queryable structure of user identities, their security identifier(s), their group memberships, and the access control information; and

provide a global view across the plurality of data sources and a plurality of data types of data stored within the plurality of data sources, the global view relating to at least one of the data and objects that at least one of a predetermined user, a predetermined group, and a predetermined security identifier at least one of owns and has access to within the cloud storage repository.

7. The method according to claim 1, wherein the computer executable instructions when executed by the one or more microprocessors configure the system to:

resolve the access control information for a knowledge worker of a plurality of knowledge workers, the knowledge worker associated with one or more knowledge worker identities of a plurality of knowledge worker identities upon the plurality of data sources;

assembling a data map report for the knowledge worker; and

synchronize changes captured from at least one of the plurality of data sources and one or more third party systems associated with the plurality of data sources; and

update the data map report for the knowledge worker of the plurality of knowledge workers.

8. The method according to claim 7, wherein the computer executable instructions when executed by the one or more microprocessors configure the system to:

recording one or more selections made by the knowledge worker of the plurality of knowledge workers within the data map report, each selection relating to a folder location;

upon receipt of a query associated with the knowledge worker of the plurality of knowledge workers the system updates and renders initially to the user a predetermined portion of the data map report, the predetermined portion of the data map report being those portions of the data map report associated with the one or more selections made by the knowledge worker of the plurality of knowledge workers; and

either subsequently updates and renders the remainder of the data map report or awaits a user request to update and render the remainder of the data map report.

9. The method according to claim 1, wherein the computer executable instructions when executed by the one or more microprocessors configure the system to:

provide a network ingestion service which queries a data source connector of a plurality of data source connectors;

receive a response from the data source connector of the plurality of data source connectors;

evaluate with the network ingestion service the response from the data source connector of the plurality of data source connectors against a plurality of policies and a system of record to determine differences;

transmit to the data source connector of the plurality of data source connectors references to items evaluated to have differences;

capture with the data source connector of the plurality of data source connectors the items associated with the references;

store within the cloud storage repository the captured items; and

update the access control information.

10. The method according to claim 9, wherein the data source connector provides for:

interfacing with the cloud storage repository; and

handling calls from the cloud storage repository for content; and

the cloud storage repository supports at least one of directly and indirectly through other cloud storage repositories all of the policies and other capture logic, the user interface(s), and associated system(s) of record.

11. The method according to claim 1, wherein the computer executable instructions when executed by the one or more microprocessors configure the system to:

detect a change relating to a permission of at least one of a portion of the data and an object within a data source of the plurality of data sources and update associated permissions relating to the at least one of the portion of the data and the object in the cloud storage repository;

detect a change in a folder name within a data source of the plurality of data sources and update a corresponding folder name at least one of within the cloud storage repository and within an index stored within the cloud storage repository associated with the stored at least one of data and objects from the plurality of data sources within the cloud storage repository.

12. The method according to claim 1, wherein the computer executable instructions when executed by the one or more microprocessors configure the system to:

record a point-in-time;

restore a predetermined subset of the plurality of data sources from the cloud storage repository to a user;

wherein

the predetermined subset of the plurality of data sources are restored as of the point-in-time;

elements associated the predetermined subset of the plurality of data sources are replaced within the predetermined subset of the plurality of data sources with the respective elements and states at the selected point-in-time; and

the elements are at least one of a list of objects, data, folder names, and permissions.