System and method for providing a unified storage system that supports file/object duality

• US 8,620,865 B2
• Filed: 06/24/2011
• Issued: 12/31/2013
• Est. Priority Date: 06/24/2011
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Slik, David
• Primary Examiner(s): ALAM, SHAHID AL
• Application Number:
  US13/168,431
• Publication Number:
  US 20120330894A1
• Time in Patent Office: 921 Days
• Field of Search: 707/626, 707/638, 707/705
• US Class Current: 707/626
• CPC Class Codes: G06F 16/1774 Locking methods, e.g. locking methods for file systems allowing shared and concurrent access to files
• CPC Class Codes: G06F 16/1824 implemented using Network-attached Storage [NAS] architecture distributed or networked storage systems G06F3/067; protocols for distributed storage of data in a network H04L67/1097
• CPC Class Codes: G06F 16/1827 Management specifically adapted to NAS management of storage area networks [SAN] G06F3/067
• CPC Class Codes: G06F 16/196 Specific adaptations of the file system to access devices and non-file objects via standard file system access operations, e.g. pseudo file systems dedicated interfaces to storage systems G06F3/0601
• CPC Class Codes: G06F 16/2308 Concurrency control transaction processing G06F9/466
• CPC Class Codes: G06F 16/2336 Pessimistic concurrency control approaches, e.g. locking or multiple versions without time stamps
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/275 Synchronous replication
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/972 Access to data in other repository systems, e.g. legacy data or dynamic Web page generation
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Methods and systems that permit file protocols and object protocols to co-exist using a file namespace and an object namespace in a unified storage system are disclosed. Data stored in the unified storage system are file-objects that preserve the behaviors expected by both file clients and object clients.

Claims

1. A method comprising:

storing a data entity in a network storage system;

accepting both a file protocol and an object protocol to access the data entity, wherein the data entity includes a file representation and an object representation, and further wherein the file representation of the data entity is accessed via the file protocol, and the file representation has a first set of access characteristics comprising non-atomic file-based update and retrieval, and further wherein the object representation of the data entity is accessed via the object protocol, and the object representation has a second set of access characteristics comprising object-based atomic update and retrieval;

if the file representation has been modified, copying the modified file representation to update the object representation; and

if the object representation has been modified, copying the modified object representation to update the file representation,

wherein modifications made via the file protocol are accessible by the object protocol, and wherein modifications made via the object protocol are accessible by the file protocol.

2. The method of claim 1, wherein copying comprises basic copying, copying on write, cloning, or delayed copying.

3. The method of claim 1, further comprising waiting until a holdoff period expires for the file representation prior to copying the file representation to update the object representation.

4. The method of claim 1, further comprising if the file representation is locked for writing, waiting until a write lock on the file representation is released prior to copying the modified object representation to update the file representation.

5. The method of claim 1, further comprising:

creating a file namespace redirector in a file namespace to point to the file representation;

creating an object namespace redirector in an object namespace to point to the object representation;

linking the file namespace redirector in the file namespace to the object namespace redirector in the object namespace; and

linking the object namespace redirector in the object namespace to the file namespace redirector in the file namespace.

6. The method of claim 5, further comprising:

creating an object version redirector in the object namespace to point to a historical version of the object representation, wherein when the object representation is modified, an unmodified version of the object representation is retained as a historical version.

7. The method of claim 6, further comprising:

allocating object identifiers (OID) for the object representation and each of the historical versions of the object representation; and

associating the OIDs with the respective object namespace redirector and object namespace version redirectors.

8. A method comprising:

creating a plurality of representations of a data entity stored in a network storage system;

upon receiving a file protocol request to access the data entity, non-atomically retrieving or non-atomically updating a first representation of the plurality of representations stored in the network storage system;

upon receiving an object protocol request to access the data entity, atomically retrieving or atomically updating a second representation of the plurality of representations, wherein a file namespace is used by the file protocol to locate the first representation, and an object namespace is used by the object protocol to locate the second representation;

if the first representation has been modified, copying the modified first representation to update the second representation; and

if the second representation has been modified, copying the modified second representation to update the first representation,

wherein modifications made via the file protocol are accessible via the object protocol, and modifications made via the object protocol are accessible via the file protocol,

and further wherein copying comprises basic copying, copying on write, cloning or delayed copying.

9. The method of claim 8, further comprising waiting until a holdoff period expires for the first representation prior to copying the first representation to update the second representation.

10. The method of claim 8, further comprising if the first representation is locked for writing, waiting until a write lock on the first representation is released prior to copying the modified second representation to update the first representation.

11. The method of claim 8, further comprising:

creating a file namespace redirector in a file namespace to point to the first representation;

creating an object namespace redirector in an object namespace to point to the second representation;

linking the file namespace redirector in the file namespace to the object namespace redirector in the object namespace; and

linking the object namespace redirector in the object namespace to the file namespace redirector in the file namespace.

12. The method of claim 11, further comprising:

creating an object version redirector in the object namespace to point to a historical version of the second representation, wherein when the second representation is modified, an unmodified version of the second representation is retained as a historical version.

13. The method of claim 12, further comprising:

allocating object identifiers (OID) for the second representation and each of the historical versions of the second representation; and

associating the OIDs with the respective object redirector and object version redirectors.

14. A method of providing a unified storage system that supports both object protocols and file protocols, comprising:

if a file protocol update of a file representation of a data entity is to be performed;

writing the file protocol update to the file representation; and

copying contents of the updated file representation to update an object representation of the data entity; and

if an object protocol update of the object representation of the data entity is to be performed;

copying the object representation to a new object representation;

applying the object protocol update to the new object representation; and

copying contents of the new object representation to update the file representation,

wherein the file representation is linked to the object representation and the object protocol operates upon object representations, and a file protocol operates upon file representations, and further wherein copying comprises basic copying, copying on write, cloning or delayed copying.

15. The method of claim 14, further comprising retaining the object representation as a historical version of the new object representation.

16. The method of claim 14, further comprising waiting until a holdoff period has expired for the updated file representation prior to copying the contents of the updated file representation to update the object representation.

17. The method of claim 14, further comprising if the file representation is locked for writing, waiting until a write lock is released on the file representation prior to copying the new object representation to update the file representation.

18. A network storage server system, comprising:

a receiving module configured to receive a request from a client to access a data entity stored in a network storage system, wherein the data entity comprises a first representation that is accessed using a file protocol, and a second representation that is accessed using an object protocol,

an object identifier module configured to allocate an object identifier to the second representation;

a file namespace module configured to create a namespace redirector in a file namespace that points to the first representation;

an object namespace module configured to create an object location redirector in an object namespace that points to the second representation; and

a copying module configured to copy the first representation to update the second representation when the first representation has been modified and to copy the second representation to update the first representation when the second representation has been modified, wherein the first representation is modified non-atomically, and further wherein the second representation is modified atomically.

19. The system of claim 18, wherein copying comprises basic copying, copying on write, cloning, or delayed copying.

20. The system of claim 18, wherein the second representation further comprises zero or more historical object versions, and the object identifier module is further configured to allocate object identifiers to each of the historical object versions.

21. A non-transitory machine readable medium storing instructions which when executed by a machine, cause the machine to:

store a data entity in a network storage system;

accept both a file protocol and an object protocol to access the data entity, wherein the data entity includes a file representation and an object representation, and further wherein the file representation of the data entity is accessed via the file protocol, and the file representation has a first set of access characteristics comprising non-atomic file-based update and retrieval, and further wherein the object representation of the data entity is accessed via the object protocol, and the object representation has a second set of access characteristics comprising object-based atomic update and retrieval;

if the file representation has been modified, copy the modified file representation to update the object representation; and

if the object representation has been modified, copy the modified object representation to update the file representation,

wherein modifications made via the file protocol are accessible by the object protocol, and wherein modifications made via the object protocol are accessible by the file protocol.

22. The machine readable medium of claim 21, wherein copying comprises basic copying, copying on write, cloning, or delayed copying.

23. The machine readable medium of claim 21, wherein the machine is further caused to wait until a holdoff period expires for the file representation prior to copying the file representation to update the object representation.

24. The machine readable medium of claim 21, wherein the machine is further caused to if the file representation is locked for writing, wait until a write lock on the file representation is released prior to copying the modified object representation to update the file representation.

25. The machine readable medium of claim 21, wherein the machine is further caused to:

create a file namespace redirector in a file namespace to point to the file representation;

create an object namespace redirector in an object namespace to point to the object representation;

link the file namespace redirector in the file namespace to the object namespace redirector in the object namespace; and

link the object namespace redirector in the object namespace to the file namespace redirector in the file namespace.

26. The machine readable medium of claim 21, wherein the machine is further caused to:

create an object version redirector in the object namespace to point to a historical version of the object representation, wherein when the object representation is modified, an unmodified version of the object representation is retained as a historical version.

27. The machine readable medium of claim 21, wherein the machine is further caused to:

allocate object identifiers (OID) for the object representation and each of the historical versions of the object representation; and

associate the OIDs with the respective object namespace redirector and object version redirectors.

28. A method comprising:

storing a data entity in a network storage system;

maintaining two representations of the data entity, wherein a first representation has a first set of access characteristics comprising non-atomic update and retrieval, and a second representation has a second set of access characteristics comprising non-atomic update and retrieval;

accepting both a first protocol and an second protocol to access the data entity, wherein the first representation of the data entity is accessed via the first protocol, and the second representation of the data entity is accessed via the second protocol;

if the first representation has been modified, copying the modified first representation to update the second representation; and

if the second representation has been modified, copying the modified second representation to update the first representation,

wherein modifications made via the first protocol are accessible by the second protocol, and wherein modifications made via the second protocol are accessible by the first protocol.

29. The method of claim 28, further comprising:

creating a first namespace redirector in a first namespace to point to the first representation;

creating a second namespace redirector in a second namespace to point to the second representation;

linking the first namespace redirector in the first namespace to the second namespace redirector in the second namespace; and

linking the second namespace redirector in the second namespace to the first namespace redirector in the first namespace.