Systems, methods and interfaces for data virtualization

• US 10,102,144 B2
• Filed: 04/15/2014
• Issued: 10/16/2018
• Est. Priority Date: 04/16/2013
• Status: Active Grant

• Current Assignee: SanDisk Technologies LLC (Western Digital Corporation)
• Original Assignee: SanDisk Technologies LLC (Western Digital Corporation)
• Inventors: Sundararaman, Swaminathan, Talagala, Nisha, Subramanian, Sriram
• Primary Examiner(s): Parikh, Kalpit
• Application Number:
  US14/253,645
• Publication Number:
  US 20140310499A1
• Time in Patent Office: 1,645 Days
• CPC Class Codes: G06F 12/0269 Incremental or concurrent garbage collection, e.g. in real-time systems G06F12/0261 takes precedence
• CPC Class Codes: G06F 12/109 for multiple virtual address spaces, e.g. segmentation G06F12/1036 takes precedence
• CPC Class Codes: G06F 16/2308 Concurrency control transaction processing G06F9/466
• CPC Class Codes: G06F 2212/657 Virtual address space management

Abstract

A data services module performs log storage operations in response to requests by storing data on one or more storage devices, and appending information pertaining to the requests to a separate metadata log. A log order of the metadata log may correspond to an order in which the requests were received, regardless of the order in which data of the requests are written to the storage devices. The requests may correspond to identifiers of a logical address space. The data services module implements an any-to-any translation layer configured to map identifiers of the logical address space to the stored data. The virtualization module may include a metadata management module configured to checkpoint the translation layer metadata by, inter alia, appending aggregate, checkpoint entries to the metadata log. The data services module may leverage the translation layer between the logical identifiers and underlying storage locations to efficiently implement logical manipulation operations.

Claims

1. An apparatus, comprising:

an interface module configured to receive an atomic storage request pertaining to a target set of logical addresses of a logical address space;

an atomic storage module configured to implement storage operations corresponding to the atomic storage request using a transactional set of logical addresses separate from the target set of logical addresses, wherein implementing the storage operations comprises binding one or more logical addresses of the transactional set to one or more data sets stored on a storage resource; and

a data services module configured to append logical manipulation entries to a metadata log, the logical manipulation entries corresponding to modifications of associations between logical addresses and data stored on the storage resource, wherein the data services module is configured to close the atomic storage request by appending a logical manipulation entry to bind the target set of logical addresses to the one or more data sets,

wherein the interface module, the atomic storage module, and the data services module comprise one or more of logic hardware and executable code, the executable code stored on a non-transitory computer readable medium.

2. The apparatus of claim 1, further comprising:

a translation module configured to maintain a forward map comprising any-to-any mappings between logical addresses of the logical address space and virtual addresses of the storage resource; and

a logical move module configured to implement a logical move operation of data stored on the storage resource, wherein the data is stored at a physical storage address of the storage resource that corresponds to a first virtual address associated with a first logical address, and wherein the logical move operation comprises a) updating the forward map to associate the first virtual address with a second, different logical address, and b) recording a logical manipulation entry corresponding to the logical move operation.

3. The apparatus of claim 2, wherein the recorded logical manipulation entry is configured to indicate that the first virtual address is bound to the second, different logical address, and to indicate that the first logical address is unbound.

4. The apparatus of claim 1, further comprising:

a translation module configured to maintain a forward map comprising any-to-any mappings between logical addresses of the logical address space and virtual addresses of the storage resource; and

a logical replication module configured to implement a logical replication operation of data stored on the storage resource and mapped to a first logical address through a first virtual address, wherein the logical replication operation comprises a) updating the forward map to associate the first virtual address with a second, different logical address, and b) storing a logical manipulation entry to indicate that the first virtual address is associated with both the first logical address and the second, different logical address.

5. The apparatus of claim 1, further comprising a snapshot module configured to create a snapshot of a first set of logical addresses within the logical address space by writing a logical manipulation entry to a metadata log, wherein the appended logical manipulation entry is configured to tie a second set of logical addresses to data tied to the first set of logical addresses.

6. The apparatus of claim 1, wherein the data services module is further configured to append one or more mapping entries to the metadata log corresponding to the atomic storage request, wherein the one or more mapping entries indicate one or more associations between logical addresses and data stored on the storage resource.

7. The apparatus of claim 1, further comprising a translation module configured to maintain a forward map comprising any-to-any mappings between logical addresses of the logical address space and virtual addresses of the storage resource, wherein the data services module is further configured to modify the forward map in response to the atomic storage request.

8. The apparatus of claim 7, wherein the data services module is further configured to modify the forward map by changing a mapping between logical addresses and virtual addresses and appending a logical manipulation entry that records the changed mapping to the metadata log.

9. The apparatus of claim 1, further comprising a data log module configured to append data of the atomic storage request to a data log comprising a plurality of storage resources, and wherein the metadata log is maintained on a storage resource that is separate from the storage resources comprising the data log.

10. The apparatus of claim 1, wherein:

an order of the logical manipulation entries within the metadata log corresponds to an order in which requests are received from one or more clients;

the requests are serviced in an order that differs from the order in which the requests are received from the one or more clients; and

the data services module is configured to acknowledge completion of a request in response to appending an entry corresponding to the request to the metadata log.

11. A method, comprising:

maintaining metadata pertaining to data stored on a storage device, the metadata comprising a forward map comprising any-to-any mappings between logical addresses of a logical address space and data stored on the storage device; and

servicing an atomic storage request pertaining to a target set of logical block addresses by;

implementing storage operations corresponding to the atomic storage request using a transactional set of logical addresses separate from the target set of logical addresses, wherein implementing the storage operations comprises binding one or more logical addresses of the transactional set to one or more data sets stored on the storage device

modifying a mapping in the forward map to bind the target set of logical addresses to the one or more data sets, and

appending a record to a persistent metadata log corresponding to the modified mapping.

12. The method of claim 11, wherein the forward map is configured to bind logical addresses to virtual addresses of the storage device, the virtual addresses corresponding to physical storage units of the storage device, the method further comprising storing the data sets on the storage device by appending the data sets to segments of a virtual data log, wherein the segments comprise sets of virtual addresses.

13. The method of claim 11, wherein servicing the atomic storage request comprises:

modifying an entry of the forward map configured to bind a first logical address to the data through a particular virtual address, such that the entry binds the particular virtual address to a second logical address, wherein the record appended to the persistent metadata log corresponds to the modified entry.

14. The method of claim 13, further comprising removing an association between the first logical address and the particular virtual address from the forward map.

15. The method of claim 13, wherein the persistent metadata log indicates that the particular virtual address is bound to both the first logical address and the second logical address.

16. A computer-program product comprising computer-readable program code stored on a non-transitory computer-readable storage medium and configured to cause a computing system to perform operations, comprising:

servicing requests pertaining to logical block addresses of a logical address space by appending data of the requests to a data log maintained on a plurality of storage devices, wherein an upper-level namespace corresponding to the plurality of storage devices comprises the logical address space;

translating logical block addresses of the logical address space to addresses of an intermediate translation layer, wherein the addresses of the intermediate translation layer are within a lower-level namespace corresponding to one of the storage devices, the addresses of the intermediate translation layer corresponding to physical storage units of the storage device such that a forward map comprising any-to-any mappings between the logical block addresses and addresses of the intermediate translation layer is independent of relationships between the addresses of the intermediate translation layer and the physical storage units;

altering a mapping between a logical block address and an address of the intermediate translation layer in response to a request, wherein altering the mapping comprises appending a persistent note to a metadata log corresponding to the altered mapping, wherein the metadata log is maintained on a storage device that is separate from the plurality of storage devices comprising the data log; and

servicing an atomic storage request pertaining to a target set of logical block addresses by;

implementing storage operations corresponding to the atomic storage request in a transactional set of logical block addresses separate from the target set of logical block addresses, wherein implementing the storage operations comprises binding logical block addresses of the transactional set to particular addresses within the intermediate translation layer, and

altering mappings between logical block addresses and addresses of the intermediate translation layer to bind the target set of logical block addresses to the particular addresses.

17. The computer-program product of claim 16, wherein the persistent note appended to the metadata log is configured to associate two or more logical block addresses of the logical address space to a single address of the intermediate translation layer in response to a virtual copy operation.

18. The computer-program product of claim 16, wherein the persistent note is configured to replace an association between a first logical block address of the logical address space and a particular address of the intermediate translation layer with an association between a second, different logical block address of the logical address space and the particular address in response to a virtual move operation.