File system for a capture system

• US 8,707,008 B2
• Filed: 03/16/2011
• Issued: 04/22/2014
• Est. Priority Date: 08/24/2004
• Status: Active Grant

• Current Assignee: McAfee, LLC
• Original Assignee: McAfee, Inc. (McAfee, LLC)
• Inventors: Lowe, Rick, Broeker, Stephen, Liu, Weimin
• Primary Examiner(s): PATEL, KAUSHIKKUMAR M
• Application Number:
  US13/049,533
• Publication Number:
  US 20110167212A1
• Time in Patent Office: 1,133 Days
• Field of Search: None
• US Class Current: 711/173
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: H04L 63/123 received data contents, e.g. message integrity
• CPC Class Codes: H04L 63/1408 by monitoring network traffic monitoring network traffic per se H04L43/00

Abstract

A file system can be provided in a capture system to efficiently read and write captured objects. In one embodiment, such a file system includes a plurality of queues to queue captured objects to be written to a disk, each queue being associated with one of a plurality of object types, and each queue containing captured objects of the type associated with each queue. A scheduler can be provided to select one of the plurality of queues, and a block manager to select a partition of a disk, the partition being associated with the object type of the captured objects in the selected queue. A disk controller configured to write contiguous blocks of data from the selected queue to the selected partition is connected to the block manager to enable writing to a disk.

Claims

1. A capture system, comprising:

a file system configured to organize data on a disk and to regulate read and write accesses;

a plurality of queues associated with captured objects that are received through the capture system via network traffic, wherein the queues are associated with captured objects based on content of the captured objects, and wherein the disk is divided into a plurality of partitions for storing at least some of the captured objects, wherein the captured objects are indexed with a tag database comprising a data structure, in which each record of the data structure is configured to index a corresponding captured object according to information about the captured object, the information of each captured object including a content type associated with the captured object;

a scheduler configured to select one of the plurality of queues for performing a particular write operation, wherein the scheduler implements a write policy that defines a threshold level of write performance to optimize the file system for write operations over read operations, and wherein the threshold level of write performance is adjustable up to a physical limit of the disk; and

a block manager configured to select one of the partitions of the disk, wherein blocks of data are written from the selected queue to the selected partition.

2. The capture system of claim 1, wherein the file system is a block-based file system including contiguous blocks that form the plurality of partitions.

3. The capture system of claim 1, wherein the file system includes an application program interface (API) for interfacing with an operating system of the capture system.

4. The capture system of claim 1, further comprising:

a catchall queue for objects not having their own designated queue based on content.

5. The capture system of claim 1, wherein each one of the plurality of queues is associated with the captured objects based on an object type of the captured objects, and wherein object type determinations are performed by identifying a content field of a record associated with a particular captured object.

6. The capture system of claim 1, wherein at least one of the queues is associated with e-mails or word processing documents.

7. The capture system of claim 1, wherein the captured objects represented by particular blocks of data can be retrieved in response to a query to search for a tag included in one or more of the captured objects.

8. The capture system of claim 1, wherein each one of the plurality of queues is associated with the captured objects based on a content type of the captured objects, and wherein a content field of a record associated with a particular captured object represents the content type of the particular captured object.

9. A method, comprising:

receiving network traffic via a capture system configured for capturing objects, wherein the capture system includes a file system configured to organize data on a disk and to regulate read and write accesses, wherein the capture system further includes a plurality of queues associated with captured objects that are received through the capture system via network traffic, wherein the queues are associated with captured objects based on content of the captured objects, and wherein the disk is divided into a plurality of partitions for storing at least some of the captured objects, wherein the capture system further includes a scheduler configured to implement a write policy that defines a threshold level of write performance to optimize the file system for write operations over read operations, and wherein the threshold level of write performance is adjustable up to a physical limit of the disk, and wherein the captured objects are indexed with a tag database comprising a data structure, in which each record of the data structure is configured to index a corresponding captured object according to information about the captured object, the information of each captured object including a content type associated with the captured object;

selecting one of the plurality of queues for performing a particular write operation; and

selecting one of the partitions of the disk, wherein blocks of data are written from the selected queue to the selected partition.

10. The method of claim 9, wherein the file system is a block-based file system including contiguous blocks that form the plurality of partitions.

11. The method of claim 9, wherein the file system includes an application program interface (API) for interfacing with an operating system of the capture system.

12. The method of claim 9, further comprising:

a catchall queue for objects not having their own designated queue based on content.

13. The method of claim 9, wherein each one of the plurality of queues is associated with the captured objects based on an object type of the captured objects, and wherein object type determinations are performed by identifying a content field of a record associated with a particular captured object.

14. The method of claim 9, wherein at least one of the queues is associated with e-mails or word processing documents.

15. The method of claim 9, wherein the captured objects represented by particular blocks of data can be retrieved in response to a query to search for a tag included in one or more of the captured objects.

16. The method of claim 9, wherein each one of the plurality of queues is associated with the captured objects based on a content type of the captured objects, and wherein a content field of a record associated with a particular captured object represents the content type of the particular captured object.

17. Logic encoded in non-transitory media that includes code for execution and when executed by a processor operable to perform operations comprising:

receiving network traffic via a capture system configured for capturing objects, wherein the capture system includes a file system configured to organize data on a disk and to regulate read and write accesses, wherein the capture system further includes a plurality of queues associated with captured objects that are received through the capture system via network traffic, wherein the queues are associated with captured objects based on content of the captured objects, and wherein the disk is divided into a plurality of partitions for storing at least some of the captured objects, wherein the capture system further includes a scheduler configured to implement a write policy that defines a threshold level of write performance to optimize the file system for write operations over read operations, and wherein the threshold level of write performance is adjustable up to a physical limit, of the disk, and wherein the captured objects are indexed with a tag database comprising a data structure, in which each record of the data structure is configured to index a corresponding captured object according to information about the captured object, the information of each captured object including a content type associated with the captured object;

selecting one of the plurality of queues for performing a particular write operation, and selecting one of the partitions of the disk, wherein blocks of data are written from the selected queue to the selected partition.

18. The logic of claim 17, wherein the file system is a block-based file system including contiguous blocks that form the plurality of partitions, and wherein the file system includes an application program interface (API) for interfacing with an operating system of the capture system.

19. The logic of claim 17, wherein the plurality of queues includes a catchall queue for objects not having their own designated queue, and wherein the

captured objects represented by particular blocks of data can be retrieved in response to a query to search for a tag included in one or more of the captured objects.

20. The logic of claim 17, wherein each one of the plurality of queues is associated with the captured objects based on an object type of the captured objects, and wherein object type determinations are performed by observing a content field of a record associated with a particular captured object.