Performing data storage operations in a cloud storage environment, including searching, encryption and indexing

• US 8,612,439 B2
• Filed: 03/31/2010
• Issued: 12/17/2013
• Est. Priority Date: 06/30/2009
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Prahlad, Anand, Kottomtharayil, Rajiv, Kavuri, Srinivas, Gokhale, Parag, Vijayan, Manoj
• Primary Examiner(s): Wilson, Kimberly
• Assistant Examiner(s): Liu, Hexing
• Application Number:
  US12/751,713
• Publication Number:
  US 20100332479A1
• Time in Patent Office: 1,357 Days
• Field of Search: 707/614, 707/626, 707/620, 707/623
• US Class Current: 707/736
• CPC Class Codes: G06F 11/3485 for I/O devices
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/1827 Management specifically adapted to NAS management of storage area networks [SAN] G06F3/067
• CPC Class Codes: G06F 16/1844 Management specifically adapted to replicated file systems
• CPC Class Codes: G06F 16/41 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 3/06 Digital input from, or digital output to, record carriers , e.g. RAID, emulated record carriers or networked record carriers
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0626 Reducing size or complexity of storage systems
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0649 Lifecycle management
• CPC Class Codes: G06F 3/0667 at data level, e.g. file, record or object virtualisation
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0206 Price or cost determination based on market factors
• CPC Class Codes: G06Q 50/188 Electronic negotiation
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/06 : specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/1095 : Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/535 : Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04L 67/56 : Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54
• CPC Class Codes: H04L 67/5682 : Policies or rules for updating, deleting or replacing the stored data
• CPC Class Codes: H04L 69/08 : Protocols for interworking; Protocol conversion
• CPC Class Codes: Y04S 40/20 : Information technology specific aspects, e.g. CAD, simulation, modelling, system security

Abstract

Systems and methods are disclosed for performing data storage operations, including content-indexing, containerized deduplication, and policy-driven storage, within a cloud environment. The systems support a variety of clients and cloud storage sites that may connect to the system in a cloud environment that requires data transfer over wide area networks, such as the Internet, which may have appreciable latency and/or packet loss, using various network protocols, including HTTP and FTP. Methods are disclosed for content indexing data stored within a cloud environment to facilitate later searching, including collaborative searching. Methods are also disclosed for performing containerized deduplication to reduce the strain on a system namespace, effectuate cost savings, etc. Methods are disclosed for identifying suitable storage locations, including suitable cloud storage sites, for data files subject to a storage policy. Further, systems and methods for providing a cloud gateway and a scalable data object store within a cloud environment are disclosed, along with other features.

Claims

1. A computer-implemented method for collaboratively searching for stored documents related to an organization, the method comprising:

creating a collaborative document, wherein multiple users can add search criteria to the collaborative document;

posting the collaborative document to a collaborative shared logical location accessible by multiple users;

receiving at least a first search criterion from a first user computer and adding and saving the first search criterion to the collaborative document;

receiving at least a second search criterion from a second user computer and adding and saving the second search criterion to the same collaborative document, wherein the first and second user computers are geographically separated;

generating a third search criterion based on dynamic changes made to at least one of the first search criterion and the second search criterion or based on heuristics-type information from at least one of the first search criterion and the second search criterion; and

performing a search of the stored documents based on the collaborative document containing the first, second, and third search criteria to create one or more search results that identify stored documents,

wherein the search of the stored documents includes a search of both online and offline documents,

wherein some or all of the offline documents contain data that is no longer available on a local area network of the organization,

wherein online documents include secondary copies of data stored at non-public locations on two or more cloud storage sites,

wherein the two or more cloud storage sites are operated by different vendors, and

wherein the search of the stored documents in the two or more cloud storage sites is performed by receiving and executing generic or vendor-agnostic file system calls.

2. The computer-implemented method of claim 1 wherein the collaborative shared logical location is Microsoft SharePoint Services.

3. The method of claim 1, further comprising imposing access restrictions, by restricting a first subset of the multiple users from reviewing a first search result reflecting a first offline data object based on a first set of access control information while permitting a second subset of the multiple users to review the same search result based on a second set of access control information, wherein the access control information comprises an access control list that further comprises a list of access control entries that reflect users or groups that are allowed to access the first offline data object.

4. The method of claim 1, further comprising imposing access restrictions, by restricting a first subset of the multiple users from reviewing a first search result reflecting a first offline data object based on a first set of access control information while permitting a second subset of the multiple users to review the same search result based on a second set of access control information.

5. The method of claim 1, further comprising:

receiving from a user a result indication related to a first search result, wherein the first search result is related to a first document, wherein the result indication is based on the user'"'"'s evaluation of the first search result or the first document, wherein the result indication is either;

a flag to provide additional information relating to the first search result in the collaborative document or

a tag that modifies the first document with additional information; and

associating a received flag or tag with the first document in an online index of the online documents to permit later searching of the online index based at least in part on the flag or tag.

6. The method of claim 5 wherein the result indication indicates a relevance of the search result to a purpose for the search.

7. The method of claim 5 wherein the result indication flags a search result for removal from the search results.

8. The method of claim 5 wherein the result indication indicates that the first document comprises legally privileged content.

9. The method of claim 1, wherein the collaborative document permits multiple collaborative participants to implement a workflow subsequent to the search process and to contribute to steps of the workflow subsequent to the search process.

10. The method of claim 1, further comprising utilizing one or more parsers to convert content created by one or more applications having an application-specific document format into a format viewable using a web browser.

11. The method of claim 1, further comprising utilizing one or more parsers to convert content created by one or more applications having an application-specific document format into a format viewable using a web browser, wherein the parsers are configured to convert one or more of the following types of content:

spreadsheet content, database tables, instant messaging conversation logs, and email databases.

12. A system for collaboratively searching for stored documents related to an organization, the system comprising:

means for creating a collaborative document,

wherein multiple users can add search criteria to the collaborative document, and

wherein the collaborative document is accessible over a computer network by multiple users;

means for receiving at least a first search criterion from a first user computer and adding and saving the first search criterion to the collaborative document;

means for receiving at least a second search criterion from a second user computer and adding and saving the second search criterion to the same collaborative document,

wherein the first and second user computers are coupled via the computer network, and

wherein the first and second user computers are geographically separated;

means for generating a third search criterion based on dynamic changes made to at least one of the first search criterion and the second search criterion or based on heuristics-type information from at least one of the first search criterion and the second search criterion; and

means for performing a search of the stored documents based on the collaborative document containing the first, second, and third search criteria to create one or more search results that identify stored documents,

wherein stored documents include secondary copies of data stored on two or more cloud storage sites,

wherein the two or more cloud storage sites are operated by different vendors, and

wherein means for performing the search of the stored documents in the two or more cloud storage sites includes means for receiving and executing generic or vendor-agnostic file system calls.

13. A system of claim 12, further comprising:

means for parsing stored documents to migrate data from multiple data types to a common data type.

14. A system of claim 12, further comprising:

means for integrating stored documents from at least two different file systems.

15. A non-transitory computer-readable medium having instructions for collaboratively searching for stored documents related to an organization which, if executed by a processor, cause the processor to:

create a collaborative document, wherein multiple users can add search criteria to the collaborative document;

post the collaborative document to a collaborative shared logical location accessible by multiple users;

receive at least a first search criterion from a first user computer and adding and saving the first search criterion to the collaborative document;

receive at least a second search criterion from a second user computer and adding and saving the second search criterion to the same collaborative document, wherein the first and second user computers are geographically separated;

generate a third search criterion based on dynamic changes made to at least one of the first search criterion and the second search criterion or based on heuristics-type information from at least one of the first search criterion and the second search criterion; and

perform a search of the stored documents based on the collaborative document containing the first, second, and third search criteria to create one or more search results that identify stored documents,

wherein the search of the stored documents includes a search of both online and offline documents,

wherein some or all of the offline documents contain data that is no longer available on a local area network of the organization,

wherein online documents include secondary copies of data stored at non-public locations on two or more cloud storage sites,

wherein the two or more cloud storage sites are operated by different vendors, and

wherein the search of the stored documents in the two or more cloud storage sites is performed by receiving and executing generic or vendor-agnostic file system calls.

16. The computer-readable medium of claim 15, wherein the instructions further cause the processor to impose access restrictions, by restricting a first subset of the multiple users from reviewing a first search result reflecting a first offline data object based on a first set of access control information while permitting a second subset of the multiple users to review the same search result based on a second set of access control information.

17. The computer-readable medium of claim 15, wherein the instruction further cause the processor to:

receive from a user a result indication related to a first search result, wherein the first search result is related to a first document, wherein the result indication is based on the user'"'"'s evaluation of the first search result or the first document, wherein the result indication is either;

a flag to provide additional information relating to the first search result in the collaborative document or

a tag that modifies the first document with additional information; and

associate a received flag or tag with the first document in an online index of the online documents to permit later searching of the online index based at least in part on the flag or tag.

18. The computer-readable medium of claim 17, wherein the result indication indicates that the first document comprises legally privileged content.

19. The computer-readable medium of claim 17, wherein the collaborative document permits multiple collaborative participants to implement a workflow subsequent to the search process and to contribute to steps of the workflow subsequent to the search process.