SEARCHABLE ARCHIVE

• US 20070266044A1
• Filed: 05/09/2007
• Published: 11/15/2007
• Est. Priority Date: 02/20/2004
• Status: Active Grant

• Current Assignee: Informatica LLC (Informatica, Inc. (California))
• Original Assignee: Sand Technology Incorporated (Constellation Software Incorporated)
• Inventors: Fadeitchev, Evgueni, Grondin, Richard, Zarouba, Vassili
• Granted Patent: US 8,386,435 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

A searchable archiving system. A searchable archiving system includes one or more compacted files of archive data loosely coupled to a search process. To create a compacted file, an archiving process tokenizes the archive data, optimizes the tokenized archive data, and extracts archive metadata from the tokenized data. The tokenized data may then be compressed in a variety of ways into compressed segments that may be individually accessed and decompressed by the search agents. Before compression, segment metadata is extracted from the segments. The compressed segments and segment metadata are then combined to create a compacted file. The search process accesses the compacted files by consulting locally stored archive metadata extracted from the files during the compaction process. The search process then invokes one or more search agents that actively search the compacted files. The search agents do so by using the segment metadata to identify segments to decompress and search.

Claims

1. A method of retrieving a datum from a searchable archive by a data processing system, the searchable archive comprising a metadata file and one or more compacted files, comprising:

selecting a selected compacted file from the one or more compacted files that may include the datum using the metadata file;

accessing the selected compacted file;

selecting a selected compressed segment from one or more compressed segments in the selected compacted file using metadata stored in the compacted file;

generating a decompressed segment from the selected compressed segment; and

searching the decompressed segment to determine if the decompressed segment includes the datum.

2. The method of claim 1 wherein:

selecting a selected compacted file is performed by a search process; and

accessing the selected compacted file, selecting a selected compressed segment, generating a decompressed segment, and searching the decompressed segment are performed by one or more search agents invoked by the search process.

3. A method of retrieving a datum from a searchable archive by a data processing system, the searchable archive comprising a metadata file and one or more compacted files, comprising:

selecting a selected compacted file from the one or more compacted files that may include the datum using the metadata;

accessing the selected compacted file;

selecting one or more bit vectors from the selected compacted file; and

performing a Boolean operation on the bit vectors included in the selected compacted file to determine if the datum is stored in the selected compacted file.

4. The method of claim 3, wherein:

selecting a selected compacted file is performed by a search process; and

accessing the selected compacted file and performing a Boolean operation is performed by one or more search agents invoked by the search process.

5. A data processing system for retrieving a datum from a searchable archive, the searchable archive comprising a metadata file and one or more compacted files, comprising:

a processor; and

a memory coupled to the processor, the memory having program instructions executable by the processor stored therein, the program instructions including;

selecting a selected compacted file from the one or more compacted files that may include the datum using the metadata file;

accessing the selected compacted file;

selecting a selected compressed segment from one or more compressed segments in the selected compacted file using metadata stored in the compacted file;

generating a decompressed segment from the selected compressed segment; and

searching the decompressed segment to determine if the decompressed segment includes the datum.

6. The data processing system of claim 5, the program instructions further including:

selecting a selected compacted file is performed by a search process; and

accessing the selected compacted file, selecting a selected compressed segment, generating a decompressed segment, and searching the decompressed segment are performed by one or more search agents invoked by the search process.

7. A data processing system for retrieving a datum from a searchable archive, the searchable archive comprising a metadata file and one or more compacted files, comprising:

a processor; and

a memory coupled to the processor, the memory having program instructions executable by the processor stored therein, the program instructions including;

selecting a selected compacted file from the one or more compacted files that may include the datum using the metadata;

accessing the selected compacted file;

selecting one or more bit vectors from the selected compacted file; and

performing a Boolean operation on the bit vectors included in the selected compacted file to determine if the datum is stored in the selected compacted file.

8. The data processing system of claim 7, wherein:

selecting a selected compacted file is performed by a search process; and

accessing the selected compacted file and performing a Boolean operation is performed by one or more search agents invoked by the search process.