Distributed data storage system for fixed content

• US 7,552,356 B1
• Filed: 10/12/2004
• Issued: 06/23/2009
• Est. Priority Date: 06/30/2004
• Status: Active Grant

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Botros, Sherif M., Faybishenko, Yaroslav, Waterhouse, Steven Richard
• Primary Examiner(s): Wilson; Yolanda L
• Application Number:
  US10/964,411
• Time in Patent Office: 1,715 Days
• Field of Search: 714/6, 714/4
• US Class Current: 714/6.1
• CPC Class Codes: G06F 16/1837 Management specially adapted to peer-to-peer storage networks topology management mechanisms of peer-to-peer networks H04L67/1042
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

In accordance with the invention, a data storage system for fixed content includes one or more peer node computer systems interconnected by a storage system network, where each peer node computer system includes a symmetric storage system application and locally attached storage. A data object is distributively stored on one or more of the one or more peer node computer systems and identified using a unique object identifier. A data object is flexibly stored on the data storage system according to external criteria. Stored data objects are accessible from any of the one or more peer node computer systems. Applications can be executed on the data storage system using data objects that are stored on, being written to, or being read from the data storage system to generate results accessible by a user.

Claims

1. A data storage system comprising:

a storage system network; and

one or more peer node computer systems interconnected by said storage system network, each of said one or more peer node computer systems comprising;

a memory, the memory storing instructions associated with a computing system implemented process for providing a symmetric storage system;

a processor, the processor executing at least some of the instructions associated with the computing system implemented process for providing a symmetric storage system;

said computing system implemented process for providing a symmetric storage system comprising;

providing an interface module;

providing a node management module connected with said interface module;

providing a core module connected with said node management module;

providing a disklet module connected with said node management module;

providing a metadata module connected with said node management module; and

providing an object archive module connected with said node management module, and

providing a locally attached storage associated with said computing system implemented process for providing a symmetric storage system;

wherein the processor executes at least some of the instructions associated with the computing system implemented process for providing a symmetric storage system such that at least one data object is distributively stored on one or more of said one or more peer node computer systems, and

further wherein said at least one data object is accessed by any of said one or more peer node computer systems.

2. The data storage system of claim 1, wherein said at least one data object is identified using a unique identifier generated by said computing system implemented process for providing a symmetric storage system and based on an intrinsic property of said data object.

3. The data storage system of claim 2, wherein said intrinsic property is a content of said at least one data object.

4. The data storage system of claim 3, wherein said unique identifier is generated using a hashing algorithm on said content.

5. The data storage system of claim 4, wherein said hashing algorithm is an SHA-1 hashing algorithm.

6. The data storage system of claim 2, wherein said at least one data object is further stored according to at least one external criterion.

7. The data storage system of claim 6, wherein said at least one external criterion is a specified level of reliability of storage of said at least one data object on said data storage system.

8. The data storage system of claim 1, said data storage system further for executing at least one disklet.

9. The data storage system of claim 8, wherein said at least one disklet is uploaded to and stored on said data storage system.

10. The data storage system of claim 9, wherein a handle to said at least one disklet is generated and returned, said handle identifying said at least one disklet on said data storage system.

11. The data storage system of claim 9, wherein said at least one disklet is executed on said data storage system using said at least one data object stored on said data storage system.

12. The data storage system of claim 9, wherein said at least one disklet is executed on said data storage system using at least one data object being written to said data storage system.

13. The data storage system of claim 9, wherein said at least one disklet is executed on said data storage system using at least one data object being read from said data storage system.

14. The data storage system of claim 1, further comprising:

a switch, wherein said switch is configured to support communications between said one or more peer node computer system on said storage system network and between said storage system network and an external network.

15. The data storage system of claim 14, wherein said switch is configured to present a virtual internet protocol (IP) address for external network access to said data storage system and an administrative internet protocol (IP) address to permit access to said data storage system for system administration.

16. The data storage system of claim 14, wherein each of said one or more external computer systems includes a client application program interface (API) comprising a load spreading mechanism, and

further wherein access to said one or more peer node computer systems by each of said one or more external computer systems is determined in accordance with said load spreading mechanism.

17. The data storage system of claim 15, wherein said switch comprises a load spreading mechanism, and

further wherein access to said one or more peer node computer systems by each of said one or more external computer systems is determined in accordance with said load spreading mechanism.

18. The data storage system of claim 1, wherein control of said data storage system is by a master node, wherein said master node is elected from said one or more peer node computer systems.

19. The data storage system of claim 1, wherein said computing system implemented process for providing a symmetric storage system divides at least one data object into one or more data fragments and one or more parity fragments, said one or more data fragments and said one or more parity fragments being distributively stored on one or more of said locally attached storage.

20. The data storage system of claim 19, wherein said computing system implemented process for providing a symmetric storage system divides said at least one data object into one or more data blocks, and

further wherein said computing system implemented process for providing a symmetric storage system divides each of said one or more data blocks into one or more data fragments.

21. The data storage system of claim 19, wherein said computing system implemented process for providing a symmetric storage system applies an erasure coding algorithm to said one or more data fragments to generate said one or more parity fragments.

22. The data storage system of claim 21, wherein said erasure coding algorithm is a Reed-Solomon erasure coding algorithm.

23. The data storage system of claim 22, wherein said erasure coding algorithm is based upon a desired level of storage reliability of said at least one data object on said data storage system.

24. The data storage system of claim 19, wherein said computing system implemented process for providing a symmetric storage system generates a unique identifier of said at least one data object based on an intrinsic property of said data object.

25. The data storage system of claim 19, wherein said computing system implemented process for providing a symmetric storage system generates system metadata associated with said data object.

26. The data storage system of claim 25, where at least a portion of said system metadata is stored with each of said one or more data fragments and said one or more parity fragments.

27. The data storage system of claim 1, wherein metadata associated with said at least one data object is distributively stored on one or more of said one or more peer node computer systems.

28. The data storage system of claim 27, wherein said metadata includes system metadata generated by said computing system implemented process for providing a symmetric storage system.

29. The data storage system of claim 27, wherein said metadata includes extended metadata received with said at least one data object.

30. The data storage system of claim 27, wherein said metadata includes extended metadata generated by a disklet.

31. The data storage system of claim 1, further comprising one or more metadata caches.

32. The data storage system of claim 1, wherein said locally attached storage comprises:

one or more storage structures.