Method and system for mirroring and archiving mass storage

• US 6,397,307 B2
• Filed: 02/23/1999
• Issued: 05/28/2002
• Est. Priority Date: 02/23/1999
• Status: Expired due to Term

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Legato Systems, Inc. (Dell Technologies Inc.)
• Inventors: Ohran, Richard S.
• Primary Examiner(s): BRAGDON, REGINALD GLENWOOD
• Application Number:
  US09/255,486
• Publication Number:
  US 20010052058A1
• Time in Patent Office: 1,190 Days
• Field of Search: 711/161,162 707/204,202,203 380/280,283
• US Class Current: 711/161
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/2064 while ensuring consistency
• CPC Class Codes: G06F 11/2066 Optimisation of the communication load
• CPC Class Codes: G06F 11/2074 Asynchronous techniques
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

A method and system for mirroring and archiving mass storage. A primary mass storage and a secondary mass storage are synchronized to contain the same data. Thereafter, a primary system tracks changes made to the primary mass storage. These changes are consolidated periodically into update files, the consolidations representing changes made to the primary mass storage during a time interval that ends when the primary mass storage is in a logically consistent state. These update files contain only those changes necessary to represent the modified state of the primary mass storage at the time of the update. The primary system then transfers the update files to a secondary system to bring the secondary mass storage current with the primary mass storage. The consolidation minimizes the amount of information that must be transferred and therefore allows for a relatively low band width communication channel. In addition, update files may be cached by the primary system so that requests for mirrored or archived data may be fulfilled by the primary system cache rather than requiring access to the secondary system.

Claims

1. In a primary system that has a primary mass storage and is in communication with a secondary system having a secondary mass storage, a method for mirroring and archiving the primary mass storage by transmitting to the secondary mass storage only those changes to the primary mass storage that are necessary to bring the secondary mass storage current with the primary mass storage while representing a plurality of logically consistent states of the primary mass storage, the method comprising the steps of:

synchronizing the secondary mass storage with the primary mass storage; and

for each of a plurality of distinct time intervals;

changing data stored in storage locations of the primary mass storage;

identifying when the primary mass storage is in a logically consistent state, so as to identify an end of a current time interval of the plurality of distinct time intervals;

as the data is changed, storing information representing all changes that have been made in the storage locations during the each of the plurality of time intervals in updates that correspond to the plurality of time intervals;

consolidating the stored information in each update by retaining, for each storage location that has changed during the corresponding time interval, the information representing the most recent change to the particular storage location for that update, with the information representing any previous changes to the particular storage location being discarded for that update, such that each update includes only the changes that are required for bringing the secondary mass storage device current with the primary mass storage device as the data in the primary mass storage device exists at the end of the corresponding time interval; and

transferring each update to the secondary mass storage at the end of the corresponding time interval, wherein each update is stored at the secondary mass storage such that the plurality of logically consistent states of the primary mass storage are determinable from the plurality of updates stored at the secondary mass storage.

2. A method as recited in claim 1 further comprising the steps of:

storing a copy of each update in a cache holding area associated with the primary system; and

accessing data included in a mirrored and archived version of the primary mass storage by first determining whether the data is included in one of the updates stored in the cache holding area and then accessing the data from the secondary mass storage if the data is not included in one of the updates in the cache holding area.

3. A method as recited in claim 1 wherein the step of storing information representing all changes and the step of consolidating the stored information are conducted independently of any particular file structure associated with the primary mass storage or the secondary mass storage.

4. A method as recited in claim 1 wherein the step of storing information representing all changes is conducted at a disk level of the primary mass storage.

5. A method as recited in claim 1 wherein the primary mass storage is subdivided and comprises a first subdivision that includes the storage locations in which said data is stored and further comprises a second subdivision that includes a cache holding area in which is stored the information representing all changes that have been made in the storage locations during each of the plurality of time intervals.

6. A method as recited in claim 5 wherein the first subdivision is one or more files.

7. A method as recited in claim 5 wherein the first subdivision is a disk.

8. A method as recited in claim 5 wherein the first subdivision is a disk partition.

9. A method as recited in claim 5 wherein the step of storing information representing all changes to the storage locations comprises the step of filtering to store information representing the changes to the first subdivision and to not store information representing the changes to the second subdivision.

10. A method as recited in claim 1, wherein the step of consolidating the stored information in each update is conducted in an ongoing manner during the corresponding time interval.

11. A method as recited in claim 1, wherein the step of consolidating the stored information in each update is conducted at or after the end of the corresponding time interval.

12. A method as recited in claim 1 further comprising the step of:

encrypting data transferred between the primary system and the secondary system by conducting the steps of;

exchanging values between the primary system and the secondary system, the values being subject to change from one moment to the next;

manipulating, by the primary system, a first value exchanged from the secondary system to the primary system to generate a primary system key seed;

manipulating, by the secondary system, a second value exchanged from the primary system to the secondary system to generate a secondary system key seed, the primary system key seed and the secondary system key seed being equal;

using the primary key seed at the primary system to generate a primary encryption key;

using the secondary key seed at the secondary system to generate a secondary encryption key, the primary encryption key and the secondary encryption being equal, as if they had been generated using a single key seed; and

encrypting and decrypting data transmitted between the primary system and the secondary system using the primary encryption key and the secondary encryption key.

13. A method as recited in claim 12 wherein the step of encrypting data further comprises the step of generating multiple keys and then using a different key to encrypt and decrypt data transmitted in each of multiple data transmissions between the primary system and the secondary system.

14. A method as recited in claim 12 wherein the values exchanged between the primary system and the secondary system are a representation of time.

15. A method as recited in claim 1, the method further comprising at least one of the steps of:

consolidating one or more of the updates stored at the secondary mass storage into a single update;

merging one or more of the updates stored at the secondary mass storage with data of the secondary mass storage; and

discarding one or more of the updates stored at the secondary mass storage.

16. A method as recited in claim 1, wherein the step of identifying when the primary mass storage is in a logically consistent state is conducted for each of the plurality of distinct time intervals such that the duration of each of the plurality of distinct time intervals is variable.

17. In a primary system that has a primary mass storage and is in communication with a secondary system having a secondary mass storage, a method for mirroring and archiving the primary mass storage by transmitting to the secondary mass storage only those changes to the primary mass storage that are necessary to bring the secondary mass storage current with the primary mass storage while representing a plurality of logically consistent states of the primary mass storage, the method comprising the steps of:

creating a cache holding area associated with the primary system;

synchronizing, at an initial time, the secondary mass storage with the primary mass storage; and

iteratively conducting the following steps, such that a plurality of logically consistent states of the primary mass storage are determinable from a plurality of updates stored at the secondary mass storage;

tracking all changes in storage locations of the primary mass storage during a current time interval, said all changes being tracked by storing, in the cache holding area, information representing the changes;

identifying when the primary mass storage is in a logically consistent state, so as to identify an end of the current time interval;

consolidating said information representing the changes to create an update stored in the cache holding area by conducting at least the steps of;

eliminating any of the changes that represent an intermediate state of the data stored in the primary mass storage as opposed to representing said logically consistent state; and

retaining any of the changes that represent said logically consistent state; and

transmitting the update to the secondary mass storage, wherein the update is stored at the secondary mass storage.

18. A method as recited in claim 17, further comprising the step of accessing data included in a mirrored and archived version of the primary mass storage by first determining whether the data is included in one of the updates stored in the cache holding area and then accessing the data from the secondary mass storage if the data is not included in one of the updates stored in the cache holding area.

19. A method as recited in claim 17 wherein the step of consolidating said all tracked changes to create an update further comprises the step of deleting another, least recently used update stored in the cache holding area if there is otherwise insufficient space in the cache holding area to store the update.

20. A method as recited in claim 17, further comprising the step of encrypting the update prior to transmitting the update to the secondary mass storage.

21. A method as recited in claim 17 wherein the secondary system serves to backup the primary mass storage system.

22. A method as recited in claim 17 wherein the step of tracking all changes is conducted at a disk level of the primary mass storage.

23. In a computing environment including a primary system having a primary mass storage and a cache holding area and a secondary system having a secondary mass storage, a method for mirroring and archiving the primary mass storage, independently of any file structure associated with the mass storage by transmitting to the secondary mass storage only those changes that are necessary to bring the secondary mass storage current with the primary mass storage while representing a plurality of logically consistent states of the primary mass storage at the secondary mass storage, the method comprising the steps of:

synchronizing, at an initial time, the secondary mass storage with the primary mass storage;

iteratively conducting the steps of;

tracking all changes in storage locations of the primary mass storage during a current time interval, said all changes being tracked by storing, in the cache holding area, information representing the changes;

identifying when the primary mass storage is in a logically consistent state, so as to identify an end of the current time interval;

consolidating said information representing the changes to create an update stored in the cache holding area by conducting at least the steps of;

eliminating any of the changes that represents an intermediate state of the data stored in the primary mass storage as opposed to representing said logically consistent state; and

retaining any of the changes that represent said logically consistent state;

encrypting the update; and

transmitting the encrypted update to the secondary mass storage; and

storing, in the secondary mass storage, each of the updates to provide an archive of changes to the primary mass storage such that the plurality of logically consistent states of the primary mass storage are determinable from the updates stored in the secondary mass storage.

24. A method as recited in claim 23 further comprising the step of accessing data included in a mirrored and archived version of the primary mass storage by first determining whether the data is included in one of the updates stored in the cache holding area and then accessing the data from the secondary mass storage if the data is not included in one of the stored in the cache holding area.

25. A method as recited in claim 23 wherein the secondary system serves to backup the primary mass storage.

26. A system for mirroring and archiving data stored in a primary mass storage independently of any particular file structure associated with the primary mass storage, the system comprising:

a primary mass storage included in a primary system;

a secondary mass storage included in a secondary system, the secondary system being in communication with the primary system;

primary processor means included in the primary system for iteratively performing the steps of;

tracking all changes in storage locations of the primary mass storage during a current time interval, said all changes being tracked by storing, in a cache holding area, information representing the changes;

identifying when the primary mass storage is in a logically consistent state, so as to identify an end of the current time interval;

consolidating said information representing the changes to create an update by conducting at least the steps of;

eliminating any of the changes that represents an intermediate state of the data stored in the primary mass storage as opposed to representing said logically consistent state; and

retaining any of the changes that represent said logically consistent state; and

transmitting the update to the secondary mass storage at the end of the time interval; and

secondary processor means included in the secondary system for receiving the updates and for storing the updates such that a plurality of logically consistent states of the primary mass storage can be determined from the updates.

27. A system as recited in claim 26 wherein the primary processor means comprises:

means for creating the cache holding area associated with the primary mass storage; and

means for storing each update in the cache holding area.

28. A system as recited in claim 26 wherein the primary processor means operates at a disk level of the primary mass storage when conducting the step of tracking all changes.

29. A system as recited in claim 26 wherein:

the primary processor means comprises;

means for decrypting data received from the secondary processor means; and

means for encrypting data sent to the secondary processor means; and

the secondary processor means comprises;

means for decrypting data received from the primary processor means; and

means for encrypting data sent to the primary processor means.