Mechanism for reliable update of virtual disk device mappings without corrupting data

• US 6,789,122 B1
• Filed: 06/05/2000
• Issued: 09/07/2004
• Est. Priority Date: 05/12/1998
• Status: Expired due to Term

• Current Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Block, Robert J., Zheng, Xiaoyan, Slaughter, Gregory L., Traversat, Bernard A.
• Primary Examiner(s): Wiley, David
• Assistant Examiner(s): BOUTAH, ALINA A
• Application Number:
  US09/587,677
• Time in Patent Office: 1,555 Days
• Field of Search: 709/229, 709/212, 709/217, 709/219, 714/4
• US Class Current: 709/229
• CPC Class Codes: G06F 11/1423 by reconfiguration of paths
• CPC Class Codes: G06F 11/201 between storage system components
• CPC Class Codes: G06F 2201/815 Virtual middleware or OS functionality using virtual machines to implement generic software techniques for error detection or fault masking G06F11/1484

Abstract

A cluster implements a virtual disk system that provides each node of the cluster access to each storage device of the cluster. The virtual disk system provides high availability such that a storage device may be accessed and data access requests are reliably completed even in the presence of a failure. To ensure consistent mapping and file permission data among the nodes, data are stored in a highly available cluster database. Because the cluster database provides consistent data to the nodes even in the presence of a failure, each node will have consistent mapping and file permission data. A cluster transport interface is provided that establishes links between the nodes and manages the links. Messages received by the cluster transports interface are conveyed to the destination node via one or more links. The configuration of a cluster may be modified during operation. Prior to modifying the configuration, a reconfiguration procedure suspends data access requests and waits for pending data access requests to complete. The reconfiguration is performed and the mapping is modified to reflect the new configuration. The node then updates the internal representation of the mapping and resumes issuing data access requests.

Claims

1. A distributed computing system comprising:

a first node, a second node, and a third node coupled via a communication link;

a storage device connected to said second and third nodes and configured to store data;

a database for storing data, wherein said data comprises mapping data identifying said second node as a primary node for accessing said storage device and said third node as an alternate node for accessing said storage device;

wherein said first node is configured to store an instance of the database and to access said storage device using the mapping data; and

wherein in response to said mapping data being updated, said first node stops issuing data requests to said storage device prior to said mapping data being updated, and said first node resumes issuing data requests to said storage device after said mapping data is updated.

2. The distributed computing system of claim 1 wherein said database is configured to update said mapping data, and wherein said first node sends an acknowledge signal to said database when said first node has stopped issuing data requests to said storage device.

3. The distributed computing system of claim 2 wherein said first node waits for outstanding data requests issued to said storage device to complete prior to sending said acknowledge signal.

4. The distributed computing system of claim 3 wherein said database is configured to update said mapping data after receiving said acknowledge signal.

5. The distributed computing system of claim 4 wherein said database is configured to produce a first synchronization signal to indicate a pending update of said mapping data.

6. The distributed computing system of claim 5 wherein said database is configured to produce a second synchronization signal to indicate said mapping data is updated.

7. The distributed computing system of claim 6 wherein said database waits for said acknowledge signal prior to updating said mapping data.

8. The distributed computing system of claim 7 whereby said distributed computing system reconfigures physical connections between nodes and said storage device without losing or corrupting data.

9. A method of updating mapping data for accessing a storage device within a distributed computing system comprising a plurality of nodes, the method comprising:

said nodes storing a copy of said mapping data, wherein said mapping data identifies a primary node and an alternate node of the plurality of nodes for accessing said storage device;

providing an indication to said nodes that an update of said mapping data is pending;

said nodes suspending data requests to said storage device in response to said indication that an update of said mapping data is pending;

said nodes waiting for outstanding data requests issued to said storage device to complete;

said nodes invalidating the stored copy of the mapping data;

said nodes producing acknowledge signals when the outstanding data requests issued to said storage device are complete;

updating said mapping data when said acknowledge signals are received from said nodes;

providing an indication to said nodes that said update is complete;

said nodes requesting an updated copy of said mapping data in response to said indication that said update is complete; and

said nodes resuming sending said data requests to said storage device.

10. The method of claim 9 wherein if an acknowledge signal is not received from one of said nodes, a cancel indication is provided to said nodes to reactivate said stored copy of the mapping data.

11. The method of claim 9 wherein said indications to said nodes are synchronization signals.

12. The method of claim 11 wherein said synchronization signals are issued to said nodes concurrently.

13. The method of claim 12 whereby said mapping data is updated without losing or corrupting data.

14. A method of updating mapping data for accessing a storage device within a distributed computing system comprising a plurality of nodes, the method comprising:

said nodes storing a copy of said mapping data, wherein said mapping data identifies a primary node and an alternate node of the plurality of nodes for accessing said storage device;

providing an indication to said nodes that an update of the mapping data is pending;

said nodes suspending data requests to said storage device in response to said indication to said nodes that an update of the mapping data is pending;

updating said mapping data;

providing an indication to said nodes that said update is complete; and

said nodes resuming sending said data requests to said storage device.

15. The method of claim 14 further comprising:

said nodes waiting for outstanding data requests issued to said storage device to complete; and

said nodes producing acknowledge signals when the outstanding data requests issued to said storage device are complete.

16. The method of claim 15 further comprising:

said nodes invalidating the stored copy of the mapping data.

17. The method of claim 16 further comprising said nodes requesting an updated copy of said mapping data in response to said indication that said update is complete.

18. The method of claim 14 wherein if an acknowledge signal is not received from one of said nodes, a cancel indication is provided to said nodes to reactivate said stored copy of the mapping data.

19. The method of claim 18 wherein said indications to said nodes are synchronization signals.

20. The method of claim 19 wherein said synchronization signals are issued to said nodes concurrently.

21. The method of claim 20 whereby said mapping data is updated without losing or corrupting data.

22. A computer-readable storage medium comprising program instructions for updating mapping data for accessing a storage device within a distributed computing system comprising a plurality of nodes, wherein said program instructions execute on the plurality of nodes of the distributed computing system and said program instructions are operable to implement the steps of:

said nodes storing a copy of said mapping data, wherein said mapping data identifies a primary node and an alternate node of the plurality of nodes for accessing said storage device;

providing an indication to said nodes that an update of the mapping data is pending;

said nodes suspending data requests to said storage device in response to indication to said nodes that an update of the mapping data is pending;

said nodes waiting for outstanding data requests issued to said storage device to complete;

said nodes invalidating the stored copy of the mapping data;

said nodes providing acknowledge signals when the outstanding data requests issued to said storage device are complete;

updating said mapping data when said acknowledge signals are received from said nodes;

outputting an indication to said nodes that said update is complete;

said nodes requesting an updated version of said mapping data in response to said indication that said update is complete; and

said nodes resuming sending said data requests to said storage device.