PERFORMING DATA STORAGE OPERATIONS IN A CLOUD STORAGE ENVIRONMENT, INCLUDING SEARCHING, ENCRYPTION AND INDEXING

• US 20100332479A1
• Filed: 03/31/2010
• Published: 12/30/2010
• Est. Priority Date: 06/30/2009
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Gokhale, Parag, Prahlad, Anand, Kottomtharayil, Rajiv, Kavuri, Srinivas, Vijayan, Manoj
• Granted Patent: US 8,612,439 B2
• Time in Patent Office: Days
• US Class Current: 707/741
• CPC Class Codes: G06F 11/3485 for I/O devices
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/1827 Management specifically adapted to NAS management of storage area networks [SAN] G06F3/067
• CPC Class Codes: G06F 16/1844 Management specifically adapted to replicated file systems
• CPC Class Codes: G06F 16/41 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 3/06 Digital input from, or digital output to, record carriers , e.g. RAID, emulated record carriers or networked record carriers
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0626 Reducing size or complexity of storage systems
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0649 Lifecycle management
• CPC Class Codes: G06F 3/0667 at data level, e.g. file, record or object virtualisation
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0206 Price or cost determination based on market factors
• CPC Class Codes: G06Q 50/188 Electronic negotiation
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/06 : specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/1095 : Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/535 : Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04L 67/56 : Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54
• CPC Class Codes: H04L 67/5682 : Policies or rules for updating, deleting or replacing the stored data
• CPC Class Codes: H04L 69/08 : Protocols for interworking; Protocol conversion
• CPC Class Codes: Y04S 40/20 : Information technology specific aspects, e.g. CAD, simulation, modelling, system security

Abstract

Systems and methods are disclosed for performing data storage operations, including content-indexing, containerized deduplication, and policy-driven storage, within a cloud environment. The systems support a variety of clients and cloud storage sites that may connect to the system in a cloud environment that requires data transfer over wide area networks, such as the Internet, which may have appreciable latency and/or packet loss, using various network protocols, including HTTP and FTP. Methods are disclosed for content indexing data stored within a cloud environment to facilitate later searching, including collaborative searching. Methods are also disclosed for performing containerized deduplication to reduce the strain on a system namespace, effectuate cost savings, etc. Methods are disclosed for identifying suitable storage locations, including suitable cloud storage sites, for data files subject to a storage policy. Further, systems and methods for providing a cloud gateway and a scalable data object store within a cloud environment are disclosed, along with other features.

Claims

1. A non-transitory computer-readable medium storing instructions that when executed by a processor perform a method for encrypting files stored within a cloud storage site, the method comprising:

receiving a request to encrypt a data file located on a cloud storage site;

receiving an indication of a requested type of encryption method to utilize when encrypting the data file located on the cloud storage site, wherein the requested type of encryption method is one of multiple types of encryption methods;

determining if the requested type of encryption method is supported by an application programming interface (API) provided by the cloud storage site; and

if the requested type of encryption method is supported by the API provided by the cloud storage site, then utilizing the API provided by the cloud storage site to encrypt the data file;

if the type of encryption method requested is not supported by the API provided by the cloud storage site, then performing the following;

downloading a copy of the data file from the cloud storage site;

encrypting the downloaded copy of the data file to create an encrypted version of the data file; and

sending the encrypted version of the data file to the cloud storage site via the API to either—

overwrite the data file stored on the cloud storage site with the encrypted version of the data file, or

write the encrypted version of the data file to the cloud storage site and delete the data file stored on the cloud storage site.

2. The method of claim 1, further comprising:

if the requested type of encryption method is supported by the API provided by the cloud storage site, then utilizing a mapping to generate API calls to encrypt the data file.

3. The method of claim 1, further comprising receiving one or more encryption keys.

4. A computing system for managing data associated with a data storage component, wherein the data storage component is coupled to multiple computers over a network and to one or more cloud storage sites, the computing system comprising:

a processor;

a memory;

a data storage management component for managing primary copies of data stored within the data storage component and managing secondary copies of the primary copies of the data,

wherein the secondary copies are stored within the one or more cloud storage sites, and

wherein the secondary copies include copies having a storage format different than a native format of the primary copies of the data;

a content indexing component for creating or updating at least one index of the stored data managed by the data storage management component,

wherein the at least one index includes a first set of information resulting from indexing the primary copies of the data, and a second set of information resulting from indexing the secondary copies of the data, and

wherein the content indexing component creates or updates the second set of data before the secondary copies of data are stored to the one or more cloud storage sites; and

a web-based search component for searching for stored data, wherein the search component is configured to search the first and second sets of information included in the at least one index for content within the primary copies and the secondary copies based on a single query and without having to access the secondary copies of data stored within the one or more cloud storage sites.

5. The computing system of claim 4 wherein the secondary copies are stored on at least two different cloud storage sites each operated by a different vendor, and wherein the web-based search component is further configured to search with the single query the secondary copies stored on the at least two different cloud storage sites.

6. A computer-implemented method for collaboratively searching for stored documents related to an organization, the method comprising:

creating a collaborative document, wherein multiple users can add search criteria to the collaborative document;

posting the collaborative document to a collaborative shared logical location accessible by multiple users;

receiving at least a first search criterion from a first user computer and adding and saving the first search criterion to the collaborative document;

receiving at least a second search criterion from a second user computer and adding and saving the second search criterion to the same collaborative document, wherein the first and second user computers are geographically separated; and

performing a search of the stored documents based on the collaborative document containing the first and second search criteria to create one or more search results that identify stored documents,

wherein the search of the stored documents includes a search of both online and offline documents,

wherein some or all of the offline documents contain data that is no longer available on a local area network of the organization,

wherein offline documents include secondary copies of data stored on two or more cloud storage sites, and

wherein the two or more cloud storage sites are operated by different vendors.

7. The computer-implemented method of claim 6 wherein the collaborative shared logical location is Microsoft SharePoint Services.

8. The method of claim 6, further comprising imposing access restrictions, by restricting a first subset of the multiple users from reviewing a first search result reflecting a first offline data object based on a first set of access control information while permitting a second subset of the multiple users to review the same search result based on a second set of access control information, wherein the access control information comprises an access control list that further comprises a list of access control entries that reflect users or groups that are allowed to access the first offline data object.

9. The method of claim 6, further comprising imposing access restrictions, by restricting a first subset of the multiple users from reviewing a first search result reflecting a first offline data object based on a first set of access control information while permitting a second subset of the multiple users to review the same search result based on a second set of access control information.

10. The method of claim 6, further comprising:

receiving from a user a result indication related to a first search result, wherein the first search result is related to a first document, wherein the result indication is based on the user'"'"'s evaluation of the first search result or the first document, wherein the result indication is either;

a flag to provide additional information relating to the first search result in the collaborative document or

a tag that modifies the first document with additional information; and

associating a received flag or tag with the first document in the online index to permit later searching of the online index based at least in part on the flag or tag.

11. The computer-readable medium of claim 10 wherein the result indication indicates a relevance of the search result to a purpose for the search.

12. The computer-readable medium of claim 10 wherein the result indication flags a search result for removal from the search results.

13. The computer-readable medium of claim 10 wherein the result indication indicates that the first document comprises legally privileged content.

14. The method of claim 6, wherein the collaborative document permits multiple collaborative participants to implement a workflow subsequent to the search process.

15. The method of claim 6, further comprising utilizing one or more parsers to convert content created by one or more applications having an application-specific document format into a format viewable using a web browser.

16. The method of claim 6, further comprising utilizing one or more parsers to convert content created by one or more applications having an application-specific document format into a format viewable using a web browser, wherein the parsers are configured to convert one or more of the following types of content:

spreadsheet content, database tables, instant messaging conversation logs, and email databases.

17. A system for collaboratively searching for stored documents related to an organization, the system comprising:

means for creating a collaborative document,

wherein multiple users can add search criteria to the collaborative document, and

wherein the collaborative document is accessible over a computer network by multiple users;

means for receiving at least a first search criterion from a first user computer and adding and saving the first search criterion to the collaborative document;

means for receiving at least a second search criterion from a second user computer and adding and saving the second search criterion to the same collaborative document,

wherein the first and second user computers are coupled via the computer network, and

wherein the first and second user computers are geographically separated; and

means for performing a search of the stored documents based on the collaborative document containing the first and second search criteria to create one or more search results that identify stored documents,

wherein stored documents include secondary copies of data stored on two or more cloud storage sites, and

wherein the two or more cloud storage sites are operated by different vendors.

18. A system of claim 17, further comprising:

means for parsing stored documents to migrate data from multiple data times to a common data type.

19. A system of claim 17, further comprising:

means for integrating stored documents from at least two different file systems.