DISTRIBUTED ARCHITECTURE FOR CONTENT INDEXING EMAILS

• US 20190079928A1
• Filed: 09/13/2018
• Published: 03/14/2019
• Est. Priority Date: 09/14/2017
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: KUMAR, Sanath, BHAGI, Sri Karthik, GOKHALE, Parag, PAWAR, Rahul S., DURAISAMY, Arun Prabu, PRADHAN, Manoj Kumar, ZAKHARKIN, Dmitriy Borisovich, SINHA, Prosenjit, PAWALE, Vipul, INAVOLU, Jaya Rama Raju, MUTHA, Manas Bhikchand, SINGH, Pushpendra
• Granted Patent: US 10,846,266 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/14 Details of searching files based on file metadata
• CPC Class Codes: G06Q 10/107 Computer-aided management of electronic mailing [e-mailing]
• CPC Class Codes: H04L 51/216 Handling conversation history, e.g. grouping of messages in sessions or threads
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

An improved content indexing (CI) system is disclosed herein. For example, the improved CI system may include a distributed architecture of client computing devices, media agents, a single backup and CI database, and a pool of servers. After a file backup occurs, the backup and CI database may include file metadata indices and other information associated with backed up files. Servers in the pool of servers may, in parallel, query the backup and CI database for a list of files assigned to the respective server that have not been content indexed. The servers may then request a media agent to restore the assigned files from secondary storage and provide the restored files to the servers. The servers may then content index the received restored files. Once the content indexing is complete, the servers can send the content index information to the backup and CI database for storage.

Claims

1. A networked information management system for content indexing emails, the networked information management system comprising:

a content indexing proxy having one or more first hardware processors, wherein the content indexing proxy is configured with first computer-executable instructions that, when executed, cause the content indexing proxy to;

receive, by a first thread executing on the content indexing proxy, identification of emails assigned to the content indexing proxy by a master content indexing proxy, wherein the identified emails are each associated with an email page in a plurality of email pages; and

for each email page in the plurality of email pages,

transmit, by the first thread to an indexing storage system, a query for secondary copy location data corresponding to the emails associated with the respective email page,

receive, by the first thread, the secondary copy location data,

transmit, by a second thread executing on the content indexing proxy, an instruction to a first computing device that executes a media agent to restore secondary copies stored at locations indicated by the secondary copy location data,

receive, by a third thread executing on the content indexing proxy, an acknowledgment from the first computing device that a restoration of the secondary copies is complete, and

transmit, by a fourth thread executing on the content indexing proxy, a request to content index the restored secondary copies; and

one or more computing devices in communication with the content indexing proxy, wherein the one or more computing devices each have one or more second hardware processors, wherein the one or more computing devices are configured with second computer-executable instructions that, when executed, cause the one or more computing devices to content index the restored secondary copies.

2. The networked information management system of claim 1, wherein the first computer-executable instructions, when executed, further cause the content indexing proxy to simultaneously transmit an instruction to the first computing device to restore secondary copies of emails associated with a first email page in the plurality of email pages and transmit a query for secondary copy location data corresponding to emails associated with a second email page in the plurality of email pages.

3. The networked information management system of claim 1, wherein the first computer-executable instructions, when executed, further cause the content indexing proxy to:

for an attachment file associated with a first email in a first email page in the plurality of email pages,

transmit, by the first thread to the indexing storage system, a query for secondary copy location data corresponding to the attachment file;

receive, by the first thread, the secondary copy location data corresponding to the attachment file;

transmit, by the second thread, an instruction to the first computing device to restore a secondary copy of the attachment file stored at a location indicated by the secondary copy location data corresponding to the attachment file;

receive, by the third thread, an acknowledgment from the first computing device that a restoration of the secondary copy of the attachment file is complete; and

transmit, by the fourth thread, a request to content index the restored secondary copy of the attachment file.

4. The networked information management system of claim 3, wherein the secondary copy of the attachment file is stored separately from a secondary copy of the first email in a secondary storage device.

5. The networked information management system of claim 1, wherein the secondary copy location data comprises at least one of logical paths to secondary copies stored in a secondary storage device or offsets indicating where the secondary copies are stored in the secondary storage device.

6. The networked information management system of claim 1, wherein the emails assigned to the content indexing proxy are emails that have not yet been content indexed.

7. The networked information management system of claim 1, wherein the second computer-executable instructions, when executed, further cause the one or more computing devices to extract one or more keywords and generate one or more previews using the restored secondary copies.

8. The networked information management system of claim 7, wherein the second computer-executable instructions, when executed, further cause the one or more computing devices to store the one or more keywords and the one or more previews in different databases.

9. The networked information management system of claim 7, wherein the second computer-executable instructions, when executed, further cause the one or more computing devices to store the one or more keywords and a path to a storage location of the one or more previews in a backup and content indexing database.

10. The networked information management system of claim 1, wherein the restored secondary copies are in a markup language format.

11. A computer-implemented method for content indexing emails, the computer-implemented method comprising:

receiving, by a first thread executing on a content indexing proxy, identification of emails assigned to the content indexing proxy by a master content indexing proxy, wherein the identified emails are each associated with an email page in a plurality of email pages; and

for each email page in the plurality of email pages,

transmitting, by the first thread to an indexing storage system, a query for secondary copy location data corresponding to the emails associated with the respective email page,

receiving, by the first thread, the secondary copy location data,

transmitting, by a second thread executing on the content indexing proxy, an instruction to a first computing device that executes a media agent to restore secondary copies stored at locations indicated by the secondary copy location data,

receiving, by a third thread executing on the content indexing proxy, an acknowledgment from the first computing device that a restoration of the secondary copies is complete, and

transmitting, by a fourth thread executing on the content indexing proxy, a request to content index the restored secondary copies.

12. The computer-implemented method of claim 11, further comprising simultaneously transmitting an instruction to the first computing device to restore secondary copies of emails associated with a first email page in the plurality of email pages and transmitting a query for secondary copy location data corresponding to emails associated with a second email page in the plurality of email pages.

13. The computer-implemented method of claim 11, further comprising:

for an attachment file associated with a first email in a first email page in the plurality of email pages,

transmitting, by the first thread to the indexing storage system, a query for secondary copy location data corresponding to the attachment file;

receiving, by the first thread, the secondary copy location data corresponding to the attachment file;

transmitting, by the second thread, an instruction to the first computing device to restore a secondary copy of the attachment file stored at a location indicated by the secondary copy location data corresponding to the attachment file;

receiving, by the third thread, an acknowledgment from the first computing device that a restoration of the secondary copy of the attachment file is complete; and

transmitting, by the fourth thread, a request to content index the restored secondary copy of the attachment file.

14. The computer-implemented method of claim 13, wherein the secondary copy of the attachment file is stored separately from a secondary copy of the first email in a secondary storage device.

15. The computer-implemented method of claim 11, wherein the secondary copy location data comprises at least one of logical paths to secondary copies stored in a secondary storage device or offsets indicating where the secondary copies are stored in the secondary storage device.

16. The computer-implemented method of claim 11, wherein the emails assigned to the content indexing proxy are emails that have not yet been content indexed.

17. The computer-implemented method of claim 11, further comprising extracting one or more keywords and generating one or more previews using the restored secondary copies.

18. The computer-implemented method of claim 17, further comprising storing the one or more keywords and the one or more previews in different databases.

19. The computer-implemented method of claim 11, further comprising:

receiving, by a first thread executing on a second content indexing proxy, identification of second emails assigned to the second content indexing proxy by the master content indexing proxy; and

performing, by the second content indexing proxy, operations to content index the second emails.

20. The computer-implemented method of claim 11, wherein the restored secondary copies are in an independent format.