Abstracting programmatic representation of data storage systems

• US 9,104,534 B2
• Filed: 02/11/2013
• Issued: 08/11/2015
• Est. Priority Date: 02/02/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Terek, Soner, Sadovsky, Vladimir, Verma, Surendra, Geiger, Avi R.
• Primary Examiner(s): Tseng, Cheng-Yuan
• Assistant Examiner(s): Dewan, Kamal K
• Application Number:
  US13/764,566
• Publication Number:
  US 20130173872A1
• Time in Patent Office: 911 Days
• Field of Search: 711/103, 711/112, 711/114, 711/164, 711/173, 711/E12.008, 711/E12.095, 365/218, 375/240.03, 707/758, 707/999.104
• US Class Current: 1/1
• CPC Class Codes: G06F 12/00 Accessing, addressing or allocating within memory systems or architectures digital input from, or digital output to record carriers, e.g. to disk storage units, G06F3/06
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0626 Reducing size or complexity of storage systems
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0671 In-line storage system

Abstract

Providing for a paradigm shift in block-level abstraction for storage devices is described herein. At a block-level, storage is characterized as a variable size data record, rather than a fixed size sector. In some aspects, the variable size data record can comprise a variable binary key-data pair, for addressing and identifying a variable size block of data, and for dynamically specifying the size of such block in terms of data storage. By changing the key or data values, the location, identity or size of block-level storage can be modified. Data records can be passed to and from the storage device to facilitate operational commands over ranges of such records. Block-level data compression, space management and transactional operations are provided, mitigating a need of higher level systems to characterize underlying data storage for implementation of such operations.

Claims

1. A data storage apparatus comprising:

a plurality of physical data blocks, wherein ones of the plurality of physical data blocks have one or more fixed lengths;

a processing circuit on the data storage apparatus and coupled to the plurality of physical data blocks, the processing circuit configured to map subsets of the plurality of physical data blocks to corresponding variable-length data records, the variable-length data records including corresponding key-value pairs, the variable-length data records representing information stored on the subsets; and

an aggregation component configured to cluster the information stored on the subsets of the plurality of data blocks based on the corresponding key-value pairs.

2. The data storage apparatus of claim 1, wherein the aggregation component is further configured to re-allocate at least some of the stored information to different ones of the plurality of physical data blocks based at least on changes to ones of the corresponding key-value pairs of the variable-length data records.

3. The data storage apparatus of claim 1, further comprising a transport component configured to report clustering patterns of the variable-length data records.

4. The data storage apparatus of claim 1, further comprising an operations component configured to defragment the information represented by the variable-length data records.

5. The data storage apparatus of claim 1, further comprising an operations component configured to perform error correction on ones of the variable-length data records.

6. The data storage apparatus of claim 1, further comprising an operations component configured to perform error detection on ones of the variable-length data records.

7. The data storage apparatus of claim 1, further comprising a protection component configured to encrypt one or more of the variable data length records marked by an external host for protection.

8. The data storage apparatus of claim 1, further comprising:

a transport component configured to interface with an external processing device and to convey operational commands from the external processing device to the aggregation component; and

a protection component configured to establish a secure tunnel connection to the external processing device.

9. A data storage device comprising:

a processing circuit on the data storage device coupled to a plurality of physical data blocks, ones of the plurality of physical data blocks having one or more fixed sizes, the processing circuit configured to determine a mapping of subsets of the plurality of physical data blocks corresponding to variable-sized data records, the variable-sized data records including corresponding key-value pairs, the variable-sized data records representing information stored on the subsets; and

an atomic record component configured to;

dynamically update the mapping based at least on changes to the corresponding key-value pairs of the variable-sized data records; and

re-allocate, based on changes to at least some of the corresponding key-value pairs, at least some of the information to different ones of the subsets of the plurality of fixed-sized physical data blocks.

10. The data storage device of claim 9, wherein the atomic record component is further configured to re-allocate at least some of the information to different ones of the subsets of the plurality of fixed-size physical data blocks based at least on a change in a key portion of one of the key-value pairs.

11. The data storage device of claim 9, wherein the atomic record component is further configured to re-allocate at least some of the information to different ones of the subsets of the plurality of fixed-size physical data blocks based at least on a change in a value portion of one of the key-value pairs.

12. The data storage device of claim 9, wherein the determining the mapping includes clustering ones of the variable-sized data records based on value portions of the key-value pairs.

13. The data storage device of claim 12, wherein the atomic record component is further configured to re-allocate the at least some of the information to different ones of the subsets of the plurality of fixed-size physical data blocks based at least on a change in a value portion of one of the key-value pairs.

14. A method comprising:

at a block layer of abstraction, mapping subsets of a plurality of physical data blocks of a storage device to corresponding variable-length data records, the variable-length data records including corresponding key-value pairs, the variable-length data records representing information stored on the subsets, the physical data blocks having one or more fixed lengths;

clustering the information stored on the subsets of the plurality of physical data blocks based on corresponding key-value pairs at the block layer of abstraction; and

at the block layer of abstraction, re-allocating at least some of the variable-length data records to different ones of the subsets of the plurality of physical data blocks based on changes in the corresponding key-value pairs of the variable-length data records.

15. The method of claim 14, further comprising reporting, to an external host, clustering patterns of the variable-length data records.

16. The method of claim 14, further comprising at the block layer of abstraction, defragmenting the information stored on the subsets of the plurality of physical data blocks.

17. The method of claim 14, wherein the re-allocating at least some of the variable-length data records to different ones of the subsets of the plurality of physical data blocks is based at least on changes in a key portion of at least one of the key-value pairs.

18. The method of claim 14, wherein the mapping and clustering are based at least on key portions of the key-value pairs.

19. The method of claim 14, wherein the re-allocating at least some of the variable-length data records to different ones of the plurality of physical data blocks is based at least on changes in a value portion of at least one of the key-value pairs.

20. The method of claim 14, wherein the mapping and clustering are based at least on value portions of the key-value pairs.