METHOD, DEVICE AND COMPUTER PROGRAM PRODUCT FOR DATA PROCESSING

• US 20200133780A1
• Filed: 03/06/2019
• Published: 04/30/2020
• Est. Priority Date: 10/26/2018
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Zhang, Yong, Wu, Pengfei, Zhang, Ming, Wang, Kun
• Granted Patent: US 11,093,334 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 11/1425 by reconfiguration of node membership
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 16/156 Query results presentation
• CPC Class Codes: G06F 16/16 File or folder operations, e.g. details of user interfaces specifically adapted to file systems
• CPC Class Codes: G06F 2201/88 Monitoring involving counting
• CPC Class Codes: H04L 67/1044 Group management mechanisms management of multicast group membership H04L12/185; reconfiguring of node membership in a computing system to eliminate errors G06F11/1425
• CPC Class Codes: H04L 67/1051 Group master selection mechanisms

Abstract

Embodiments of the present disclosure provide a method, a device and a computer program product for data processing. The method comprises determining, at a first node of a plurality of nodes of a metadata management system, a set of nodes from the plurality of nodes that store access information associated with a shard of metadata; determining, from the set of nodes, the number of available nodes capable of providing the shard; and adjusting the number of available nodes based on a threshold number. In this way, a distributed management of the shard of metadata in each node could be achieved without using the management node in a conventional way.

Claims

1. A method of data storage, comprising:

determining, at a first node of a plurality of nodes of a metadata management system, a set of nodes from the plurality of nodes that store access information associated with a shard of metadata;

determining, from the set of nodes, the number of available nodes capable of providing the shard; and

adjusting the number of available nodes based on a threshold number.

2. The method of claim 1, wherein determining the set of nodes comprises:

in response to receiving an access request for the shard, determining a set of nodes storing the access information associated with the shard.

3. The method of claim 1, wherein determining the set of nodes comprises:

obtaining data to be queried periodically as the shard; and

determining a set of nodes storing the access information associated with the shard.

4. The method of claim 1, wherein determining the set of nodes comprises:

transmitting, to other nodes of the plurality of nodes than the first node, a first query request for the access information;

determining, based on a response to the first query request received from the other nodes, a node from the other nodes that comprises the access information; and

determining the set of nodes based on the determined node and the first node.

5. The method of claim 4, wherein the first query request comprises an identifier of the shard, to enable the other nodes to search for the access information associated with the shard from stored access information based on the identifier, to provide the response.

6. The method of claim 1, wherein determining the number of available nodes comprises:

transmitting, to a node in the set of nodes, a second query request for availability of the shard; and

determining, based on a response to the second query request received from the node in the set of nodes, determining the number of available nodes in the set of nodes.

7. The method of claim 6, wherein the second query request comprises an identifier of the shard, to enable the node in the set of nodes to search for path information associated with the shard from stored path information based on the identifier, to provide the response.

8. The method of claim 1, wherein adjusting the number comprises:

in response to the number of available nodes being different from the threshold number, equaling the adjusted number of available nodes to the threshold number by adding or removing one or more nodes.

9. A device for data processing, comprising:

at least one processor; and

a memory coupled to the at least one processor and comprising instructions stored therein, the instructions, when executed by the at least one processor, causing the device to perform operations, the operations comprising;

determining, at a first node of a plurality of nodes of a metadata management system, a set of nodes from the plurality of nodes that store access information associated with a shard of metadata;

determining, from the set of nodes, the number of available nodes capable of providing the shard; and

adjusting the number of available nodes based on a threshold number.

10. The device of claim 9, wherein determining the set of nodes comprises:

in response to receiving an access request for the shard, determining a set of nodes storing the access information associated with the shard.

11. The device of claim 9, wherein determining the set of nodes comprises:

obtaining data to be queried periodically as the shard; and

determining a set of nodes storing the access information associated with the shard.

12. The device of claim 9, wherein determining the set of nodes comprises:

transmitting, to other nodes of the plurality of nodes than the first node, a first query request for the access information;

determining, based on a response to the first query request received from the other nodes, a node from the other nodes that comprises the access information; and

determining the set of nodes based on the determined node and the first node.

13. The device of claim 12, wherein the first query request comprises an identifier of the shard, to enable the other nodes to search for the access information associated with the shard from stored access information based on the identifier, to provide the response.

14. The device of claim 9, wherein determining the number of available nodes comprises:

transmitting, to a node in the set of nodes, a second query request for availability of the shard; and

determining, based on a response to the second query request received from the node in the set of nodes, determining the number of available nodes in the set of nodes.

15. The device of claim 14, wherein the second query request comprises an identifier of the shard, to enable the node in the set of nodes to search for path information associated with the shard from stored path information based on the identifier, to perform the response.

16. The device of claim 9, wherein adjusting the number comprises:

in response to the number of available nodes being different from the threshold number, equaling the adjusted number of available nodes to the threshold number by adding or removing one or more nodes.

17. A computer program product being tangibly stored on non-transient computer-readable medium and comprising machine-executable instructions, the machine-executable instructions, when executed, causing a machine to perform operations, the operations comprising:

determining, at a first node of a plurality of nodes of a metadata management system a set of nodes from the plurality of nodes that store access information associated with a shard of metadata;

determining, from the set of nodes, the number of available nodes capable of providing the shard; and

adjusting;

the number of available nodes based on a threshold number.

18. The computer program product of claim 17, wherein determining the set of nodes comprises:

in response to receiving an access request for the shard, determining a set of nodes storing the access information associated with the shard.

19. The computer program product of claim 17, wherein determining the set of nodes comprises:

obtaining data to be queried periodically as the shard; and

determining a set of nodes storing the access information associated with the shard.

20. The computer program product of claim 17, wherein determining the set of nodes comprises:

transmitting, to other nodes of the plurality of nodes than the first node, a first query request for the access information;

determining, based on a response to the first query request received from the other nodes, a node from the other nodes that comprises the access information; and

determining the set of nodes based on the determined node and the first node.