SYSTEMS AND METHODS FOR TRACKING A SEQUENTIAL DATA STREAM STORED IN NON-SEQUENTIAL STORAGE BLOCKS

• US 20140122796A1
• Filed: 10/31/2012
• Published: 05/01/2014
• Est. Priority Date: 10/31/2012
• Status: Abandoned Application

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: DeKoning, Rodney A.
• Application Number:
  US13/664,558
• Publication Number:
  US 20140122796A1
• Time in Patent Office: Days
• US Class Current: 711/114
• CPC Class Codes: G06F 12/0862 with prefetch
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: G06F 2212/464 Multimedia object, e.g. image, video
• CPC Class Codes: G06F 2212/6022 Using a prefetch buffer or dedicated prefetch cache
• CPC Class Codes: G06F 2212/6028 Prefetching based on hints or prefetch instructions
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0656 Data buffering arrangements
• CPC Class Codes: G06F 3/0683 Plurality of storage devices

Abstract

A process for block-level tracking of a sequential data stream that is sub-divided into multiple parts, and stored, by a file system, within non-sequential storage blocks. The process creates block-level metadata as the sequential data stream is written to the storage blocks, wherein the metadata stores pointers to the non-sequential storage blocks used to store the multiple parts of the sequential data stream. This metadata can subsequently be used by a block-level controller to more efficiently read the sequential data stream back to the file system using read-ahead processes.

Claims

1. A method for reading a data stream stored in non-sequential storage blocks, comprising:

storing a first part of a data stream in a first storage block and a second part of a data stream in a second storage block being sequentially offset from the first storage block;

generating, using a stream metadata processor, a first metadata block associated with the first storage block;

storing, in the first metadata block using the stream metadata processor, a pointer to the second storage block; and

buffering the second part of the data stream into a stream buffer as the first part of the data stream is being read to a requesting computer system, wherein a read-ahead processor reads the first metadata block, and uses the pointer to the second storage block to buffer the second part of the data stream prior to a request for the second storage block being made from the requesting computer system.

2. The method according to claim 1, further comprising;

generating, using the stream metadata processor, the first and a second metadata block as the data stream is being written to the first and second storage blocks, respectively.

3. The method according to claim 1, wherein the first and second storage blocks are physical blocks on one or more storage devices.

4. The method according to claim 1, wherein the first and second storage blocks are abstractions of physical disk blocks, and subdivisions of a virtual storage volume.

5. The method according to claim 2, wherein the first and second metadata blocks are stored separately from the first and second storage blocks.

6. The method according to claim 5, wherein the first and second metadata blocks are stored in a metadata block array for the data stream.

7. The method according to claim 1, wherein generating the pointer includes determining the logical block address where the second metadata block is stored.

8. The method according to claim 1, wherein generating the pointer includes assigning a null value if the end of the data stream is stored in the first storage block.

9. The method according to claim 1, further comprising;

storing, using the stream metadata processor, an offset value in the first metadata block representing the point in the second storage block at which the second part of the data stream starts.

10. The method according to claim 9, wherein the offset value is a number of bytes.

11. The method according to claim 1, further comprising;

storing, using the stream metadata processor, a first and a second logical unit number in the first and a second metadata block, respectively, wherein the first and the second logical unit numbers correspond to physical storage devices used to store the first and second parts of the data stream, respectively.

12. The method according to claim 1, further comprising;

storing, using the stream metadata processor, a first size value and a second size value in the first and a second metadata block, respectively, wherein the first size value and the second size value correspond to the end points of the first and second parts of the data stream, respectively.

13. The method according to claim 1, further comprising;

updating, using a metadata update processor, the first metadata block if the requesting computer system reads a third part of the data stream directly from a third storage block instead of the second part of the data stream from the stream buffer.

14. The method according to claim 1, wherein the requesting computer system uses file metadata to request the first and second parts of the data stream in sequence, and the file metadata is not available to the read-ahead processor.

15. A system for improving read performance of a data stream stored in two non-sequential storage blocks, comprising:

a stream metadata processor, configured to;

store a first part of a data stream in a first storage block and a second part of a data stream in a second storage block being sequentially offset from the first storage block,

generate a first metadata block associated with a first storage block,

store, in the first metadata block, a pointer to the second storage block; and

a read-ahead processor, for buffering the second part of the data stream into a stream buffer, wherein the read-ahead processor reads the first metadata block, and uses the pointer to the second storage block to buffer the second part of the data stream prior to a request for the second storage block being made from a requesting computer system.

16. The system according to claim 15, further comprising;

a stream metadata processor, for generating the first and a second metadata block as the data stream is being written to the first and second storage blocks, respectively.

17. The system according to claim 15, wherein the first and second storage blocks are physical blocks on one or more storage devices.

18. The system according to claim 15, wherein the first and second storage blocks are abstractions of physical disk blocks, and subdivisions of a virtual storage volume.

19. The system according to claim 16, wherein the first and second metadata blocks are stored separately from the first and second storage blocks.

20. The system according to claim 19, wherein the first and second metadata blocks are stored in a metadata block array for the data stream.

21. The system according to claim 15, wherein generating the pointer includes determining, by the stream metadata processor, the logical block address where the second metadata block is stored.

22. The system according to claim 15, wherein generating pointer includes assigning, by the stream metadata processor, a null value if the end of the data stream is stored in the first storage block.

23. The system according to claim 15, further comprising;

the stream metadata processor for storing an offset value in the first metadata block representing the point in the second storage block at which the second part of the data stream starts.

24. The system according to claim 23, wherein the offset value is a number of bytes.

25. The system according to claim 15, further comprising;

the stream metadata processor, for storing a first and a second logical unit number in the first and a second metadata block, respectively, wherein the first and the second logical unit numbers correspond to the physical storage devices used to store the first and second parts of the data stream, respectively.

26. The system according to claim 15, further comprising;

the stream metadata processor, for storing a first size value and a second size value in the first and a second metadata block, respectively, wherein the first size value and the second size value correspond to the end points of the first and second parts of the data stream, respectively.

27. The system according to claim 15, further comprising;

a metadata update processor, for updating the first metadata block if the requesting computer system reads a third part of the data stream directly from a third storage block instead of the second part of the data stream from the stream buffer.

28. The system according to claim 15, wherein the requesting computer system uses file metadata to request the first and second parts of the data stream in sequence, and the file metadata is not available to the read-ahead processor.

29. A method for storage management of a data stream, comprising;

dividing the data stream into a plurality of segments;

storing a first data stream segment, from the plurality of segments, in a block of a block-level storage system;

storing metadata in the block-level storage system and associated with the first data stream segment containing the storage location of a second data stream segment, from the plurality of segments, that follows sequentially from the first data stream segment; and

buffering the second data stream segment using a block-level storage system read-ahead process, wherein the read-ahead process uses the stored metadata associated with the first data stream segment to anticipated a request from a file system for the second data stream segment.