Writing data in a distributed data storage system

• US 10,037,146 B2
• Filed: 11/22/2016
• Issued: 07/31/2018
• Est. Priority Date: 03/05/2010
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Wright, David D.
• Primary Examiner(s): Fan, Hua
• Application Number:
  US15/358,601
• Publication Number:
  US 20170075587A1
• Time in Patent Office: 616 Days
• CPC Class Codes: G06F 12/0284 Multiple user address space allocation, e.g. using different base addresses interprocessor communication G06F15/163
• CPC Class Codes: G06F 15/173 using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake
• CPC Class Codes: G06F 15/17331 Distributed shared memory [DSM], e.g. remote direct memory access [RDMA]
• CPC Class Codes: G06F 2212/1032 Reliability improvement, data loss prevention, degraded operation etc
• CPC Class Codes: G06F 2212/1044 Space efficiency improvement
• CPC Class Codes: G06F 2212/154 Networked environment
• CPC Class Codes: G06F 2212/163 Server or database system
• CPC Class Codes: G06F 2212/263 Network storage, e.g. SAN or NAS
• CPC Class Codes: G06F 2212/401 Compressed data
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0673 Single storage device
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Methods, systems, and apparatuses, including computer programs encoded on computer-readable media, for receiving a write request that includes data and a client address at which to store the data. The data is segmented into the one or more storage units. A storage unit identifier for each of the one or more storage units is computed that uniquely identifies content of a storage unit. A mapping between each storage unit identifier to a block server is determined. For each of the one or more storage units, the storage unit and the corresponding storage unit identifier is sent to a block server. The block server stores the storage unit and information on where the storage unit is stored on the block server for the storage unit identifier. Multiple client addresses associated with a storage unit with the same storage unit identifier are mapped to a single storage unit.

Claims

1. A method for writing data, the method comprising:

based on receipt of a write request that includes data and a client address for accessing the data, segmenting the data into storage units;

computing a storage unit identifier for each of the storage units, wherein the storage unit identifier for each storage unit uniquely identifies content of the storage unit;

determining, by a metadata server, mappings between the storage unit identifiers to two or more block servers, wherein different sets of the storage unit identifiers map to different ones of the two or more block servers; and

sending the storage units with their storage unit identifiers to the block servers based upon the mappings, wherein each block server stores the one or more of the storage units sent to the block server in a storage medium of the block server and stores location information on where the storage unit is stored on the block server in association with the corresponding storage unit identifier, and wherein multiple client addresses associated with a same storage unit identifier resolve to a same block server location.

2. The method of claim 1, further comprising receiving a confirmation from each of the two or more block servers that the corresponding ones of the storage units have been stored.

3. The method of claim 2, further comprising indicating, for each storage unit identifier, the location information of the storage unit in the corresponding one of the block servers based on receipt of the location information from the block servers.

4. The method of claim 1, wherein the mapping between each storage unit identifier to one of the two or more block servers comprises a hash of the storage unit identifier.

5. The method of claim 1, wherein each of the two or more block servers updates an in use flag corresponding to the storage unit.

6. The method of claim 1, wherein each of the two or more block servers compresses the storage unit prior to storing the storage unit.

7. The method of claim 1, wherein each block server determines if the storage unit identifier sent to the block server already exists on the block server, wherein the block server does not store the storage unit on the block server if the storage unit identifier already exists on the block server, and wherein the block server stores the storage unit if the storage unit identifier does not exist on the block server.

8. The method of claim 1, further comprising mapping the client address to the storage unit identifiers and redundantly storing the mapping between the client address and the storage unit identifiers in multiple metadata servers, wherein multiple write requests are performed to redundantly store the mapping.

9. A non-transitory computer-readable storage medium containing instructions for a metadata server to service requests for writing data in a distributed storage system, the instructions for controlling a computer system to perform operations comprising:

based on receipt of a write request that includes data and a client address for accessing the data, segmenting the data into storage units;

computing a storage unit identifier for each of the storage units, wherein the storage unit identifier for each storage unit uniquely identifies content of the storage unit;

determining mappings between the storage unit identifiers to two or more block servers, wherein different sets of the storage unit identifiers map to different ones of the two or more block servers; and

sending the storage units and their corresponding storage unit identifiers to the two or more block servers based upon the mappings, wherein each block server stores the one or more of the storage units sent to the block server in a storage medium of the block server and stores location information on where the storage unit is stored on the block server in association with the corresponding storage unit identifier, and wherein multiple client addresses associated with a same storage unit identifier resolve to a same block server location.

10. The non-transitory computer-readable storage medium of claim 9, wherein the operations further comprise receiving a confirmation from each of the two or more block servers that the corresponding ones of the storage units have been stored.

11. The non-transitory computer-readable storage medium of claim 9, wherein the operations further comprise indicating, for each storage unit identifier, the location information of the storage unit in the corresponding one of the block servers based on receipt of the location information from the block servers.

12. The non-transitory computer-readable storage medium of claim 9, wherein the mapping between each storage unit identifier to one of the two or more block servers comprises a hash of the storage unit identifier.

13. The non-transitory computer-readable storage medium of claim 9, wherein each of the two or more block servers updates an in use flag corresponding to the storage unit.

14. The non-transitory computer-readable storage medium of claim 9, wherein each of the two or more block servers compresses the storage unit prior to storing the storage unit.

15. The non-transitory computer-readable storage medium of claim 9, wherein computing a storage unit identifier for each of the one or more storage units comprises computing a hash of content for a respective storage unit.

16. The non-transitory computer-readable storage medium of claim 9, wherein the operations further comprise mapping the client address to the storage unit identifiers and redundantly storing the mapping between the client address and the storage unit identifiers in multiple metadata servers, wherein multiple write requests are performed to redundantly store the mapping.

17. A system comprising:

a metadata server comprising a processor and a non-transitory machine-readable medium having stored therein instructions executable by the processor to cause the metadata server to;

based on receipt of a write request that includes data and a client address for accessing the data, segment the data into storage units;

compute a storage unit identifier for each of the storage units, wherein the storage unit identifier for each storage unit uniquely identifies content of the storage unit;

determine mappings between the storage unit identifiers to two or more block servers, wherein different sets of the storage unit identifiers map to different ones of the two or more block servers; and

send the storage units and their corresponding storage unit identifiers to the two or more block servers based upon the mappings, wherein multiple client addresses associated with a same storage unit identifier resolve to a same block server location; and

two or more block servers each comprising a processor, storage media, and a non-transitory machine-readable medium having stored therein instructions executable by the processor to cause the block server to;

store in the storage media a storage unit from the metadata server and send location information on where the storage unit is stored on the block server in association with the storage unit identifier to the metadata server.

18. The system of claim 17, wherein the machine-readable medium of each of the two or more block servers further has stored therein instructions executable by the processor of the block server to cause the block server to compress the storage unit prior to storing the storage unit.

19. The system of claim 17, wherein the non-transitory machine-readable medium of the metadata server further has stored therein instructions executable by the processor of the metadata server to cause the metadata server to associate each storage unit identifier with corresponding location information from the two or more block servers.