METHOD, DEVICE AND COMPUTER PROGRAM PRODUCT FOR FLUSHING METADATA IN MULTI-CORE SYSTEM

• US 20190332543A1
• Filed: 10/29/2018
• Published: 10/31/2019
• Est. Priority Date: 01/18/2018
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Dell Products LP (Dell Technologies Inc.)
• Inventors: Zhang, Lester, Gong, Chen, Liu, Brown Zan
• Granted Patent: US 10,606,757 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 12/0238 Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory
• CPC Class Codes: G06F 12/0804 with main memory updating G06F12/0806 takes precedence
• CPC Class Codes: G06F 12/0868 Data transfer between cache memory and other subsystems, e.g. storage devices or host systems
• CPC Class Codes: G06F 12/0891 using clearing, invalidating or resetting means
• CPC Class Codes: G06F 16/901 Indexing; Data structures therefor; Storage structures for retrieval from the web G06F16/951
• CPC Class Codes: G06F 16/907 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 2212/1044 Space efficiency improvement
• CPC Class Codes: G06F 2212/466 Metadata, control data

Abstract

Embodiments of the present disclosure provide a method, device and computer program product for flushing metadata in a multi-core system. The method comprises: moving a metadata identifier included in a sub-list of a first list to a corresponding sub-list of a second list, the sub-list of the first list and the corresponding sub-list of the second list being associated with the same processing unit; moving the metadata identifier from the corresponding sub-list of the second list to a third list based on a storage position of the metadata identifier; and determining metadata to be flushed from the third list to a metadata repository. By means of the method and device for flushing metadata as proposed in the present disclosure, metadata synchronization contention can be reduced, IO efficiency can be improved, response time can be decreased, and the cache hit can be increased.

Claims

1. A method of flushing metadata, comprising:

removing a metadata identifier included in a sub-list of a first list to a corresponding sub-list of a second list, the sub-list of the first list and the corresponding sub-list of the second list being associated with the same processing unit;

removing the metadata identifiers from the corresponding sub-list of the second list to a third list based on a storage position of the metadata identifier; and

determining metadata to be flushed from the third list to a metadata repository.

2. The method according to claim 1, wherein removing a metadata identifier to a corresponding sub-list of a second list comprises:

in response to at least one of trigger conditions being satisfied, removing the metadata identifier included in a sub-list of a first list to a corresponding sub-list of a second list, the trigger conditions comprising;

a utilization efficiency of a log system for recording metadata flushes exceeding a first threshold;

a utilization efficiency of a cache associated with metadata exceeding a second threshold; and

a predefined time interval being reached.

3. The method according to claim 1, wherein removing the metadata identifiers from the corresponding sub-list of the second list to a third list comprises:

determining a number of metadata identifiers to be removed to the third list according to a utilization efficiency of a cache associated with metadata;

obtaining a predefined flush sequence of the metadata from a log system for recording metadata flushes; and

determining the metadata identifiers to be moved to the third list based on the predefined flush sequence of the metadata and the number.

4. The method according to claim 3, wherein determining the metadata identifiers to be removed to the third list comprises:

determining the number of metadata to be flushed first based on the predefined flush sequence of the metadata; and

determining the metadata identifiers corresponding to the determined metadata as the metadata identifiers to be removed to the third list.

5. The method according to claim 1, wherein determining from the third list metadata to be flushed to a metadata repository comprises:

obtaining a predefined flush sequence of the metadata from a log system for recording metadata flushes; and

grouping the metadata identifiers in the third list based on the predefined flush sequence of the metadata, each group of metadata identifiers comprising one or more metadata identifiers, metadata corresponding to the one or more metadata identifiers having a predefined flush sequence within a threshold range; and

flushing metadata corresponding to each group of metadata identifiers to the metadata repository.

6. A device of flushing metadata, comprising:

at least one processor; and

a memory coupled to the at least one processor and comprising instructions stored thereon, the instructions, when executed by the at least one processor, causing the device to perform acts comprising;

removing a metadata identifier included in a sub-list of a first list to a corresponding sub-list of a second list, the sub-list of the first list and the corresponding sub-list of the second list being associated with the same processing unit;

removing the metadata identifiers from the corresponding sub-list of the second list to a third list based on a storage position of the metadata identifier; and

determining metadata to be flushed from the third list to a metadata repository.

7. The device according to claim 6, wherein removing a metadata identifier to a corresponding sub-list of a second list comprises:

in response to at least one of trigger conditions being satisfied, removing the metadata identifier included in a sub-list of a first list to a corresponding sub-list of a second list, the trigger conditions comprising;

a utilization efficiency of a log system for recording metadata flushes exceeding a first threshold;

a utilization efficiency of a cache associated with metadata exceeding a second threshold; and

a predefined time interval being reached.

8. The device according to claim 6, wherein removing the metadata identifier from the corresponding sub-list of the second list to a third list comprises:

determining the a number of metadata identifiers to be removed to the third list according to a utilization efficiency of a cache associated with metadata;

obtaining a predefined flush sequence of the metadata from a log system for recording metadata flushes; and

determining the metadata identifier to be moved to the third list based on the predefined flush sequence of the metadata and the number.

9. The device according to claim 8, wherein determining the metadata identifiers to be removed to the third list comprises:

determining the number of metadata to be flushed first based on the predefined flush sequence of the metadata; and

determining the metadata identifiers corresponding to the determined metadata as the metadata identifiers to be removed to the third list.

10. The device according to claim 6, wherein determining from the third list metadata to be flushed to a metadata repository comprises:

obtaining a predefined flush sequence of the metadata from a log system for recording metadata flushes; and

grouping the metadata identifiers in the third list based on the predefined flush sequence of the metadata, each group of metadata identifiers comprising one or more metadata identifiers, metadata corresponding to the one or more metadata identifiers having a predefined flush sequence within a threshold range; and

flushing metadata corresponding to each group of metadata identifiers to the metadata repository.

11. A computer program product being tangibly stored on a non-transient computer readable medium and comprising machine executable instructions which, when executed, cause a machine to perform steps of:

removing a metadata identifier included in a sub-list of a first list to a corresponding sub-list of a second list, the sub-list of the first list and the corresponding sub-list of the second list being associated with the same processing unit;

removing the metadata identifiers from the corresponding sub-list of the second list to a third list based on a storage position of the metadata identifier; and

determining metadata to be flushed from the third list to a metadata repository.

12. The computer program product according to claim 11, wherein removing a metadata identifier to a corresponding sub-list of a second list comprises:

in response to at least one of trigger conditions being satisfied, removing the metadata identifier included in a sub-list of a first list to a corresponding sub-list of a second list, the trigger conditions comprising;

a utilization efficiency of a log system for recording metadata flushes exceeding a first threshold;

a utilization efficiency of a cache associated with metadata exceeding a second threshold; and

a predefined time interval being reached.

13. The computer program product according to claim 11, wherein removing the metadata identifiers from the corresponding sub-list of the second list to a third list comprises:

determining a number of metadata identifiers to be removed to the third list according to a utilization efficiency of a cache associated with metadata;

obtaining a predefined flush sequence of the metadata from a log system for recording metadata flushes; and

determining the metadata identifiers to be moved to the third list based on the predefined flush sequence of the metadata and the number.

14. The computer program product according to claim 13, wherein determining the metadata identifiers to be removed to the third list comprises:

determining the number of metadata to be flushed first based on the predefined flush sequence of the metadata; and

determining the metadata identifiers corresponding to the determined metadata as the metadata identifiers to be removed to the third list.

15. The computer program product according to claim 11, wherein determining from the third list metadata to be flushed to a metadata repository comprises:

obtaining a predefined flush sequence of the metadata from a log system for recording metadata flushes; and

grouping the metadata identifiers in the third list based on the predefined flush sequence of the metadata, each group of metadata identifiers comprising one or more metadata identifiers, metadata corresponding to the one or more metadata identifiers having a predefined flush sequence within a threshold range; and

flushing metadata corresponding to each group of metadata identifiers to the metadata repository.