Information Source Agent Systems and Methods For Distributed Data Storage and Management Using Content Signatures

• US 20100180128A1
• Filed: 03/24/2010
• Published: 07/15/2010
• Est. Priority Date: 05/22/2003
• Status: Active Grant

• Current Assignee: Callahan Cellular LLC (Intellectual Ventures LLC)
• Original Assignee: Carmenso Data Limited Liability Company (Intellectual Ventures LLC)
• Inventors: Borden, Bruce, Brand, Russell
• Granted Patent: US 8,392,705 B2
• Time in Patent Office: Days
• US Class Current: 713/189
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 16/137 Hash-based content-based indexing of textual data G06F16/31
• CPC Class Codes: G06F 16/162 Delete operations erasing in storage systems G06F3/0652
• CPC Class Codes: G06F 16/1756 based on delta files
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/1873 Versioning file systems, temporal file systems, e.g. file system supporting different historic versions of files
• CPC Class Codes: G06F 16/20 of structured data, e.g. relational data
• CPC Class Codes: G06F 21/10 Protecting distributed programs or content, e.g. vending or licensing of copyrighted material protection in video systems or pay television H04N7/16 ; Digital rights management [DRM]
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload

Abstract

Information source agent systems and methods for distributed content storage and management using content signatures that use file identicality properties are provided. A data management system is provided that includes a content engine for managing the storage of file content, a content signature generator that generates a unique content signature for a file processed by the content engine, a content signature comparator that compares content signatures and a content signature repository that stores content signatures. Information source agents are provided that include content signature generators and content signature comparators. Methods are provided for the efficient management of files using content signatures that take advantage of file identicality properties. Content signature application modules and registries exist within information source clients and centralized servers to support the content signature methods.

Claims

1. A method for storing file information in an indexed archive system on a computing device, comprising:

receiving a file on the computing device from an information source;

generating a file content index for the received file;

extracting metadata for the received file;

storing content of the received file in a memory of the computing device;

storing the file content index in a memory of the computing device; and

storing the metadata in a memory of the computing device.

2. The method of claim 1, further comprising requesting that a client associated with the information source transmit a file to the indexed archive system.

3. The method of claim 1, further comprising encrypting file content prior to storing the content.

4. The method of claim 1, further comprising analyzing the network to capture network file access.

5. The method of claim 1, further comprising generating metadata using an information source agent.

6. The method of claim 1, further comprising suppressing storage of duplicate file contents received from different information sources.

7. The method of claim 1, further comprising suppressing the storage of duplicate file contents received from the same information sources.

8. The method of claim 1, further comprising receiving the file from a desktop computer.

9. The method of claim 1, further comprising receiving the file from a laptop computer.

10. The method of claim 1, further comprising receiving the file from a server computer.

11. The method of claim 1, further comprising receiving the file from a personal digital assistant.

12. A method for storing file information in an indexed archive system of a computing device, comprising:

intercepting file information from a file being stored by a back up system;

generating a file content index for the file;

extracting metadata for the file;

storing a pointer to a location of the file in a memory of a computing device;

storing the file content index in a memory of a computing device; and

storing the metadata in a memory of a computing device.

13. A method for storing file information in an indexed archive system of a computing device, comprising:

receiving a file on a computing device from an information source client;

generating a file content index for the received file; and

comparing the file content index to at least one stored file content index.

14. The method of claim 13, wherein the similarity between the file content index and the at least one stored file content index exceeds a similarity threshold, and further comprising:

comparing differences between the received file and files that have file content indexes that exceed the similarity threshold; and

determining a closest match between the received file and a closest match file from among the stored files that have file content indexes that exceed the similarity threshold.

15. The method of claim 14, further comprising:

creating a delta file of the differences between the received file and the closest match file;

updating a file identifier for the closest match file to account for the differences;

storing the received file content; and

storing the file content index.

16. The method of claim 14, further comprising:

creating a delta file of the differences between the received file and the closest match file;

updating a file identifier for the closest match file to account for the differences;

determining whether a storage factor has been met; and

in response to determining that the storage factor has been met storing the delta file.

17. The method of claim 14, further comprising:

creating a delta file of the differences between the received file and the closest match file;

updating a file identifier for the closest match file to account for the delta;

determining whether a storage factor has been met; and

in response to determining that the storage factor has been not met storing the delta file, the received file, and the file content index.

18. The method of claim 17, wherein determining whether the storage factor has been met comprises determining whether the indexed archive system has exceeded a storage threshold.

19. The method of claim 17, wherein determining whether the storage factor has been met is based on file type.

20. The method of claim 17, wherein determining whether the storage factor has been met is based on level of interest of a characteristic of the file.

21. The method of claim 14, further comprising:

determining whether previously received versions of the received file have been previously indexed; and

storing a link to map previously received versions of the received file to the received file.

22. The method of claim 14, further comprising:

storing a link to identify that the received file shares content indices exceeding a similarity threshold for at least one stored file, wherein a content index of the at least one stored file has a content index similarity with the received file that exceeds the similarity threshold.

23. The method of claim 13, wherein in response to determining that the similarity between the file content index and at least one stored file content index does not exceed a similarity threshold, then:

storing the received file content; and

storing the file content index for the received file.

24. A tangible computer readable medium having stored thereon in digital form computer-executable instructions that, in response to execution by a computing device, cause the computing device to perform operations for storing file information in an indexed archive system, the operations comprising:

receiving a file from an information source;

generating a file content index for the received file;

extracting metadata for the received file;

storing content of the received file;

storing the file content index; and

storing store the metadata.

25. A tangible computer readable medium having stored thereon in digital form computer-executable instructions that, in response to execution by a computing device, cause the computing device to perform operations for storing file information in an indexed archive system, the operations comprising:

intercepting file information from a file being stored by a back-up system;

generating a file content index for the file;

extracting metadata for the file;

storing a pointer to a location of the file;

storing the file content index; and

storing the metadata.

26. A tangible computer readable medium having stored thereon in digital form computer-executable instructions that, in response to execution by a computing device, cause the computing device to perform operations for storing file information in an indexed archive system, the operations comprising:

receiving a file from an information source client;

generating a file content index for the received file; and

comparing the file content index to at least one stored file content index.

27. A tangible computer readable medium having stored thereon in digital form computer-executable instructions that, in response to execution by a computing device, cause the computing device to perform operations for archiving and indexing files contained within information source clients, comprising:

content engine instructions configured to manage the storage of file content;

indexing engine instructions configured to index file data content;

metadata engine instructions configured to manage metadata characterizing a file;

triage engine instructions configured to monitor files as they are received by the system wherein the triage engine instructions are further configured to manage and control the content engine instructions, the indexing engine instructions and the metadata engine instructions; and

information entryway instructions configured to control the triage engine instructions, based on files received from information source clients.

28. The system of claim 27, further comprising information source modification controller instructions configured to send commands to the information source clients.