Managing concurrent I/O operations

• US 10,514,865 B2
• Filed: 04/24/2018
• Issued: 12/24/2019
• Est. Priority Date: 04/24/2018
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Bassov, Ivan, Fang, Hao
• Primary Examiner(s): Cardwell, Eric
• Application Number:
  US15/960,806
• Publication Number:
  US 20190324687A1
• Time in Patent Office: 609 Days
• Field of Search: None
• CPC Class Codes: G06F 12/0804 with main memory updating G06F12/0806 takes precedence
• CPC Class Codes: G06F 12/0868 Data transfer between cache memory and other subsystems, e.g. storage devices or host systems
• CPC Class Codes: G06F 2212/1016 Performance improvement
• CPC Class Codes: G06F 2212/1044 Space efficiency improvement
• CPC Class Codes: G06F 2212/222 Non-volatile memory
• CPC Class Codes: G06F 2212/608 Details relating to cache mapping
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Techniques for a managing concurrent I/Os in a file system may include receiving a sequence of conflicting I/O lists of write data stored in a cache, the sequence specifying a sequential order in which the I/O lists are to be flushed to a file stored on non-volatile storage; determining a first I/O list of the sequence having a conflict with a second I/O list of the sequence, wherein the conflict between the first I/O list and the second I/O list is a first common block written to by both the first and second I/O lists; and performing first processing that modifies the first I/O list and the second I/O list to remove the conflict.

Claims

1. A method of managing concurrent I/Os in a file system comprising:

receiving a sequence including a plurality of I/O lists of write data stored in a cache, wherein each I/O list of the sequence conflicts with at least one other I/O list of the sequence, wherein the sequence specifies a sequential order of the plurality of I/O lists corresponding to a dependency order in which a plurality of write I/O operations are received, wherein the plurality of write operations write the write data of the plurality of I/O lists to a file stored on non-volatile storage;

determining a first I/O list of the sequence having a first conflict with a second I/O list of the sequence, wherein the first I/O list includes write data of a first write of the plurality of write operations and the second I/O list includes write data of a second write of the plurality of write I/O operations, and wherein the first conflict between the first I/O list and the second I/O list includes one or more common blocks included in both the first I/O list and the second list, and wherein data is written by both the first write and the second write to the one or more common blocks, wherein the first I/O list occurs in the sequence prior to the second I/O list, and wherein the second write is received after the first write; and

performing first processing that modifies the first I/O list and the second I/O list to remove the first conflict, wherein the second I/O list includes one or more data values written to the one or more common blocks by the second write and the first processing comprises;

eliminating the one or more common blocks from the second I/O list; and

merging the one or more data values from the second I/O list with the first I/O list.

2. The method of claim 1, wherein the first processing is performed prior to commencing flushing of the first I/O list and the second I/O list and while each of the first and the second I/O lists are waiting for at least one other I/O list to be flushed to the file stored on the non-volatile storage.

3. The method of claim 2, wherein a third I/O list of the sequence conflicts with the first I/O list, the third I/O list includes write data of a third write, and the first I/O list waits for the third I/O list to complete flushing to the file stored on the non-volatile storage prior to commencing flushing of the first I/O list.

4. The method of claim 3, wherein the first write is received prior to the second write, and the third write is received prior to the first write, and wherein the sequential order of the sequence indicates a dependency ordering of the third I/O list, the first I/O list and then the second I/O list corresponding, respectively, to a receiving order of the third write, the first write and the second write.

5. The method of claim 4, wherein the first I/O list includes a second one or more data values written to the one or more common blocks, and wherein said merging includes overwriting the second one or more data values of the first I/O list with the one or more data values of the second I/O list.

6. The method of claim 5, wherein prior to performing said first processing, the method includes performing processing comprising:

determining that no client is waiting for a commit that the second I/O list, or any portion thereof, has been flushed to the file stored on the non-volatile storage; and

responsive to determining no client is waiting for a commit that the second I/O list or any portion thereof, has been flushed to the file stored on the non-volatile storage, determining to perform said first processing.

7. The method of claim 5, wherein after performing said first processing whereby the first conflict between the first I/O list and the second I/O list is removed, and wherein the second I/O list does not conflict with the third I/O list, the method includes performing processing comprising:

concurrently flushing the second I/O list and the third I/O list to the file stored on the non-volatile storage.

8. The method of claim 7, wherein the second I/O list is flushed to the file stored on the non-volatile storage in parallel with flushing the third I/O list to the file stored on the non-volatile storage.

9. The method of claim 1, wherein each of the first write and the second write is either an asynchronous or a synchronous write request issued by a client.

10. A system comprising:

at least one processor; and

a memory comprising code stored thereon that, when executed, performs a method of managing concurrent I/Os in a file system comprising;

receiving a sequence including a plurality of I/O lists of write data stored in a cache, wherein each I/O list of the sequence conflicts with at least one other I/O list of the sequence, wherein the sequence specifies a sequential order of the plurality of I/O lists corresponding to a dependency order in which a plurality of write I/O operations are received, wherein the plurality of write operations write the write data of the plurality of I/O lists to a file stored on non-volatile storage;

determining a first I/O list of the sequence having a first conflict with a second I/O list of the sequence, wherein the first I/O list includes write data of a first write of the plurality of write operations and the second I/O list includes write data of a second write of the plurality of write I/O operations, and wherein the first conflict between the first I/O list and the second I/O list includes one or more common blocks included in both the first I/O list and the second list, and wherein data is written by both the first write and the second write to the one or more common blocks, wherein the first I/O list occurs in the sequence prior to the second I/O list, and wherein the second write is received after the first write; and

performing first processing that modifies the first I/O list and the second I/O list to remove the first conflict, wherein the second I/O list includes one or more data values written to the one or more common blocks by the second write and the first processing comprises;

eliminating the one or more common blocks from the second I/O list; and

merging the one or more data values from the second I/O list with the first I/O list.

11. A non-transitory computer readable medium comprising code stored thereon that, when executed, performs a method of managing concurrent I/Os in a file system comprising:

receiving a sequence including a plurality of I/O lists of write data stored in a cache, wherein each I/O list of the sequence conflicts with at least one other I/O list of the sequence, wherein the sequence specifies a sequential order of the plurality of I/O lists corresponding to a dependency order in which a plurality of write I/O operations are received, wherein the plurality of write operations write the write data of the plurality of I/O lists to a file stored on non-volatile storage;

determining a first I/O list of the sequence having a first conflict with a second I/O list of the sequence, wherein the first I/O list includes write data of a first write of the plurality of write operations and the second I/O list includes write data of a second write of the plurality of write I/O operations, and wherein the first conflict between the first I/O list and the second I/O list includes one or more common blocks included in both the first I/O list and the second list, and wherein data is written by both the first write and the second write to the one or more common blocks, wherein the first I/O list occurs in the sequence prior to the second I/O list, and wherein the second write is received after the first write; and

performing first processing that modifies the first I/O list and the second I/O list to remove the first conflict, wherein the second I/O list includes one or more data values written to the one or more common blocks by the second write and the first processing comprises;

eliminating the one or more common blocks from the second I/O list; and

merging the one or more data values from the second I/O list with the first I/O list.

12. The non-transitory computer readable medium of claim 11, wherein the first processing is performed prior to commencing flushing of the first I/O list and the second I/O list and while each of the first and the second I/O lists are waiting for at least one other I/O list to be flushed to the file stored on the non-volatile storage.

13. The non-transitory computer readable medium of claim 12, wherein a third I/O list of the sequence conflicts with the first I/O list, the third I/O list includes write data of a third write, and the first I/O list waits for the third I/O list to complete flushing to the file stored on the non-volatile storage prior to commencing flushing of the first I/O list.

14. The non-transitory computer readable medium of claim 13, wherein the first write is received prior to the second write, and the third write is received prior to the first write, and wherein the sequential order of the sequence indicates a dependency ordering of the third I/O list, the first I/O list and then the second I/O list corresponding, respectively, to a receiving order of the third write, the first write and the second write.

15. The non-transitory computer readable medium of claim 14, wherein the first I/O list includes a second one or more data values written to the one or more common blocks, and wherein said merging includes overwriting the second one or more data values of the first I/O list with the one or more data values of the second I/O list.

16. The non-transitory computer readable medium of claim 15, wherein after performing said first processing whereby the first conflict between the first I/O list and the second I/O list is removed, and wherein the second I/O list does not conflict with the third I/O list, the method further comprising:

concurrently flushing the second I/O list and the third I/O list to the file stored on the non-volatile storage.