Transaction model for data stores using distributed file systems

• US 9,582,520 B1
• Filed: 03/29/2013
• Issued: 02/28/2017
• Est. Priority Date: 02/25/2013
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Wang, Zhanwei, Chang, Lei, Ma, Tao, Lonergan, Luke, Jian, Lirong, Ma, Lili
• Primary Examiner(s): Mofiz, Apu
• Assistant Examiner(s): BIBBEE, JARED M
• Application Number:
  US13/853,803
• Time in Patent Office: 1,432 Days
• Field of Search: 707/718
• US Class Current: 1/1
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/148 File search processing
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/1858 Parallel file systems, i.e. file systems supporting multiple processors
• CPC Class Codes: G06F 16/217 Database tuning G06F16/2282 takes precedence; database performance monitoring G06F11/3409
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/24524 Access plan code generation and invalidation; Reuse of access plans
• CPC Class Codes: G06F 16/2453 Query optimisation
• CPC Class Codes: G06F 16/24532 of parallel queries
• CPC Class Codes: G06F 16/24542 Plan optimisation
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/43 Querying
• CPC Class Codes: G06F 16/907 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: H04L 65/60 Network streaming of media packets
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A transaction model for data stores using a distributed file system is disclosed. In various embodiments, a transaction-related metadata is sent to each of a plurality of segments to which a corresponding assigned portion of a query plan is assigned. A metadata modification record is received from zero or more of the segments, reflecting changes, if any, to system metadata required to be made to reflect changes to data made by that segment in processing the portion of the query plan assigned to that segment. It is determined whether a metadata modification record has been received from each of the segments in the plurality of segments. In the event a metadata modification record is determined to have been received from each of the segments in the plurality of segments, a system metadata store is updated to reflect the respective metadata modification records received from the plurality of segments.

Claims

1. A method, comprising:

sending to each of a plurality of segments to which a corresponding assigned portion of a query plan is assigned an associated transaction-related metadata, wherein the transaction-related metadata sent to each of the plurality of segments is associated with the corresponding assigned portion of the query plan assigned to the corresponding one of the plurality of segments and is determined to be used by a corresponding one of the plurality of segments to process the corresponding assigned portion of the query plan;

receiving from zero or more of the plurality of segments a corresponding metadata modification record reflecting changes, if any, to system metadata required to be made to reflect changes to data made by that segment in processing the portion of the query plan assigned to that segment;

determining whether the corresponding metadata modification record has been received from each of the of segments in the plurality of segments;

in the event the corresponding metadata modification record is determined to have been received from each of the segments in the plurality of segments, updating a system metadata store to reflect the corresponding metadata modification records received from the plurality of segments, wherein the updating of the system metadata store comprises;

replaying, by a master node, the corresponding metadata modification records received from each of the plurality of segments; and

committing, by the master node, a transaction with which the query plan is associated, wherein the transaction is committed in response to determining that all of the plurality of segments to which a corresponding portion of the query plan is assigned have successfully completed the corresponding assigned portion for the query plan; and

in the event that one or more of the plurality of segments fail to successfully complete the corresponding assigned portion of the query plan, aborting the transaction with which the query plan is associated.

2. The method of claim 1, further comprising committing an associated system-wide transaction based at least in part on the determination that the corresponding metadata modification record has been received from each of the segments in the plurality of segments.

3. The method of claim 1, wherein the query plan is associated with a query.

4. The method of claim 3, further comprising receiving the query.

5. The method of claim 3, further comprising generating the query plan.

6. The method of claim 1, wherein the transaction-related metadata includes for a file to be modified by the receiving segment in accordance with the portion of the query plan assigned to that segment one or more of a file size and a file extent of the file.

7. The method of claim 6, wherein the segment is configured to compare the received one or more of a file size and a file extent of the file to a corresponding data obtained from a file system in which the file is stored persistently.

8. The method of claim 7, wherein the segment is further configured to truncate the file as stored in the file system based at least in part on a determination that the file as stored in the file system include data in addition a valid data indicated by the one or more of a file size and a file extent of the file.

9. The method of claim 1, wherein the transaction with which the query plan is associated is aborted in the event it is determined that the corresponding metadata modification record has not been received from one or more segments included in the plurality of segments.

10. The method of claim 9, wherein aborting the transaction includes not updating the system metadata to reflect one or more corresponding metadata modification records that have been received.

21. The method of claim 1, further comprising:

in response to receiving a query from a user, recoding a current snapshot of a distributed database system.

22. The method of claim 21, wherein the current snapshot includes transaction identifiers for transactions that are running when a transaction with which the query plan is associated starts.

23. The method of claim 1, wherein modifications to metadata on one or more segments are not visible on the master node until the master node receives all corresponding metadata modification from the respective plurality of segments to which the corresponding assigned portion of the query plan is sent, and replays the corresponding metadata modification records.

24. The method of claim 1, wherein in the event that the transaction is aborted, the system metadata is not updated to incorporate any of the corresponding modification records received the corresponding plurality of segments that successfully completed the corresponding assigned portion of the query plan.

25. The method of claim 1, wherein the aborting the transaction with which the query plan is associated in the event that that one or more of the plurality of segments fail to successfully complete the corresponding assigned portion of the query plan comprises:

determining whether a file size as stored on a distributed file system matches a size of a file reflected in system metadata according to a reporting by the master node; and

in the event that the file size stored on the distributed file system is greater than the size of the file reflected in the system metadata according to the reporting by the master node, truncating the file as stored on the distributed file system so that the size of the file as stored on the distributed file system matches the size of the file reflected in the system metadata at time a current transaction started.

11. A system, comprising:

a communication interface; and

a processor coupled to the communication interface and configured to;

send, via the communication interface, to each of a plurality of segments to which a corresponding assigned portion of a query plan is assigned, an associated transaction-related metadata, wherein the transaction-related metadata sent to each of the plurality of segments is associated with the corresponding assigned portion of the query plan assigned to the corresponding one of the plurality of segments and is determined to be used by a corresponding one of the plurality of segments to process the corresponding assigned portion of the query plan;

receive from zero or more of the plurality of segments a corresponding metadata modification record reflecting changes, if any, to system metadata required to be made to reflect changes to data made by that segment in processing the portion of the query plan assigned to that segment;

determine whether the corresponding metadata modification record has been received from each of the of segments in the plurality of segments;

in the event the corresponding metadata modification record is determined to have been received from each of the segments in the plurality of segments, update a system metadata store to reflect the corresponding metadata modification records received from the plurality of segments, wherein to update the system metadata store comprises;

replaying the corresponding metadata modification records received from each of the plurality of segments; and

committing a transaction with which the query plan is associated, wherein the transaction is committed in response to determining that all of the plurality of segments to which a corresponding portion of the query plan is assigned have successfully completed the corresponding assigned portion for the query plan; and

in the event that one or more of the plurality of segments fail to successfully complete the corresponding assigned portion of the query plan, aborting the transaction with which the query plan is associated.

12. The system of claim 11, wherein the processor is further configured to commit an associated system-wide transaction based at least in part on the determination that the corresponding metadata modification record has been received from each of the segments in the plurality of segments.

13. The system of claim 11, wherein the transaction-related metadata includes for a file to be modified by the receiving segment in accordance with the portion of the query plan assigned to that segment one or more of a file size and a file extent of the file.

14. The system of claim 13, wherein the segment is configured to compare the received one or more of a file size and a file extent of the file to a corresponding data obtained from a file system in which the file is stored persistently.

15. The system of claim 14, wherein the segment is further configured to truncate the file as stored in the file system based at least in part on a determination that the file as stored in the file system include data in addition a valid data indicated by the one or more of a file size and a file extent of the file.

16. The system of claim 11, wherein the processor is further configured to abort a transaction with which the query plan is associated in the event it is determined that the corresponding modification record has not been received from one or more segments included in the plurality of segments.

17. The system of claim 11, wherein aborting the transaction includes not updating the system metadata to reflect one or more corresponding metadata modification records that have been received.

18. A computer program product embodied in a tangible, non-transitory computer-readable storage medium, comprising computer instructions for:

sending to each of a plurality of segments to which a corresponding assigned portion of a query plan is assigned an associated transaction-related metadata, wherein the transaction-related metadata sent to each of the plurality of segments is associated with the corresponding assigned portion of the query plan assigned to the corresponding one of the plurality of segments and is determined to be used by a corresponding one of the plurality of segments to process the corresponding assigned portion of the query plan;

receiving from zero or more of the plurality of segments a corresponding metadata modification record reflecting changes, if any, to system metadata required to be made to reflect changes to data made by that segment in processing the portion of the query plan assigned to that segment;

determining whether the corresponding metadata modification record has been received from each of the of segments in the plurality of segments;

in the event the corresponding metadata modification record is determined to have been received from each of the segments in the plurality of segments, updating a system metadata store to reflect the corresponding metadata modification records received from the plurality of segments, wherein updating the system metadata store comprises;

replaying the corresponding metadata modification records received from each of the plurality of segments; and

committing a transaction with which the query plan is associated, wherein the transaction is committed in response to determining that all of the plurality of segments to which a corresponding portion of the query plan is assigned have successfully completed the corresponding assigned portion for the query plan; and

in the event that one or more of the plurality of segments fail to successfully complete the corresponding assigned portion of the query plan, aborting the transaction with which the query plan is associated.

19. The computer program product of claim 18, wherein the transaction-related metadata includes for a file to be modified by the receiving segment in accordance with the portion of the query plan assigned to that segment one or more of a file size and a file extent of the file.

20. The computer program product of claim 18, further comprising computer instructions to abort a transaction with which the query plan is associated in the event it is determined that the corresponding modification record has not been received from one or more segments included in the plurality of segments.