Continuous data backup using distributed journaling

• US 7,318,134 B1
• Filed: 03/16/2004
• Issued: 01/08/2008
• Est. Priority Date: 03/16/2004
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Oliveira, Fernando, Schriesheim, Jeffrey A.
• Primary Examiner(s): NGUYEN, THAN VINH
• Application Number:
  US10/802,489
• Time in Patent Office: 1,393 Days
• Field of Search: None
• US Class Current: 711/162
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

Described are techniques used in connection with providing continuous data backups of primary storage. A first technique uses distributed partial journals and a copy of the primary storage. A distributed partial journal is used for each input port. The partial journals are periodically incorporated into a main journal in accordance with a time stamp for the data modification recorded in each partial journal entry. A second technique, a copy on write, uses the distributed partial journals without requiring a copy of the primary storage. For each write to an address, the current contents of the address is journaled before the address on primary storage is updated.

Claims

1. A method for providing a backup copy of data on a primary storage comprising:

receiving a write request from one of a plurality of sources;

journaling said write request in one of a plurality of partial journals, wherein each of said plurality of partial journals is used to journal a portion of write requests received from a different one of said plurality of sources, wherein each of said plurality of partial journals is emptied to a primary journal including partial journal entries and merged partial journal entries, each of the merged partial journal entries representing application of one or more data modifications for a time period to a storage location; and

updating said primary storage in accordance with said write request.

2. The method of claim 1, further comprising:

receiving each of said plurality of partial journals having a plurality of partial journal entries;

merging at least a portion of said plurality of partial journal entries from one or more of said plurality of partial journals in accordance with a predetermined merging time period forming one of said merged partial journal entries for a particular storage location, said one merged partial journal entry reflecting application of data modifications for said particular storage location for said predetermined merging time period.

3. The method of claim 1, wherein each partial journal entry in said plurality of partial journals includes a time stamp associated with a write request.

4. The method of claim 1, further comprising:

periodically flushing said plurality of partial journals to said primary journal.

5. The method of claim 1, wherein said write request is made with respect to a copy of data included on said primary storage.

6. The method of claim 5, wherein each of said time stamp values is obtained using a phase lock loop processing technique and using actual time values periodically broadcast from a stable time base.

7. The method of claim 1, wherein said primary storage is one of:

a physical device, a logical device, a virtual device, and a portion of one or more different devices.

8. The method of claim 1, wherein each of said partial journal entries includes a time stamp value associated with a corresponding write request, wherein each time stamp value included in a first partial journal is synchronized with other time stamp values included in a second different partial journal.

9. The method of claim 8, wherein each of said time stamp values is obtained using a phase lock loop processing technique and using actual time values periodically broadcast from a stable time base.

10. The method of claim 1, wherein said journaling and said updating are issued simultaneously.

11. The method of claim 10, wherein each partial journal entry includes a status indicator reflecting a status of both said journaling and said updating.

12. A method for providing a backup copy of data on a primary storage comprising:

receiving a write request;

journaling said write request in one of a plurality of partial journals;

updating said primary storage in accordance with said write request;

receiving each of said plurality of partial journals having a plurality of partial journal entries; and

merging said plurality of partial journal entries in accordance with a predetermined merging time period forming a merged partial journal entry for a particular storage location, said merged partial journal entry reflecting application of data modifications for said particular storage location for said predetermined merging time period,

wherein a primary journal includes a first portion of for received partial journal entries and a second portion for merged partial journal entries.

13. A method for providing a backup copy of data on a primary storage comprising:

receiving a write request;

journaling said write request in one of a plurality of partial journals;

updating said primary storage in accordance with said write request; and

periodically flushing said plurality of partial journals to a primary journal,

wherein contents of each partial journal are included in a primary journal when said each partial journal is filled in accordance with a predetermined capacity level.

14. The method of claim 13, wherein partial journal entries within said primary journal are ordered in accordance with a time stamp value associated with each of said partial journal entries, and each of said merged partial journal entries within said primary journal are ordered in accordance with a time stamp value associated with each of said merged journal entries.

15. A method for providing a backup copy of data on a primary storage comprising:

receiving a write request;

journaling said write request in one of a plurality of partial journals; and

updating said primary storage in accordance with said write request, wherein each of said plurality of partial journals is associated with an input port in a switch and includes journal entries for all write requests received on said associated input port.

16. The method of claim 15, wherein journaling of write requests received on an input port is performed by a processor associated with said input port and a partial log associated with said input port is stored on a portion of memory associated with said input port.

17. A method for providing a backup copy of data on a primary storage comprising:

receiving a write request;

journaling said write request in one of a plurality of partial journals;

updating said primary storage in accordance with said write request;

receiving each of said plurality of partial journals having a plurality of partial journal entries; and

merging said plurality of partial journal entries in accordance with a predetermined merging time period forming a merged partial journal entry for a particular storage location, said merged partial journal entry reflecting application of data modifications for said particular storage location for said predetermined merging time period,

wherein each partial journal is associated with an input port and all write requests from that input port are journaled in the corresponding each partial journal and wherein each merged partial journal entry includes an indicator indicating the validity of journal entries received from each input port in accordance with a time stamp value associated with said merged partial journal entry.

18. A system that performs a continuous back up of data included in a primary storage unit, comprising:

a switch including input ports, each of said input ports being associated with a processor and a portion of memory;

a plurality of partial journals, each of said plurality of partial journals being associated with a unique one of said input ports and being stored in said portion of memory associated with said unique input port, said each partial journal including journal entries for write requests received on said unique input port; and

a primary journal incorporating portions of said plurality of partial journals associated with write requests that have occurred earlier in time than those entries currently included in said partial journals.

19. The system of claim 18, further comprising:

a stable time base that broadcasts an actual time periodically to each of said processors included in said switch; and

wherein associated with each of said processors is a phase lock loop component that maintains a time value synchronized with other time values maintained by other phase lock loop components, each of said journal entries in said partial journals including a time stamp value associated with a received write request in accordance with a time of an associated phase lock loop component.

20. A system that performs a continuous back up of data included in a primary storage unit, comprising:

a switch including input ports, each of said input ports being associated with a processor and a portion of memory;

a plurality of partial journals, each of said plurality of partial journals being associated with a unique one of said input ports and being stored in said portion of memory associated with said unique input port, said each partial journal including journal entries for write requests received on said unique input port; and

a primary journal incorporating portions of said plurality of partial journals associated with write requests that have occurred earlier in time than those entries currently included in said partial journals,

wherein each partial journal entry included in a partial journal includes a data value of a location after applying a modification to data at said location associated with a write request corresponding to said each partial journal entry.

21. A system that performs a continuous back up of data included in a primary storage unit, comprising:

a switch including input ports, each of said input ports being associated with a processor and a portion of memory;

a plurality of partial journals, each of said plurality of partial journals being associated with a unique one of said input ports and being stored in said portion of memory associated with said unique input port, said each partial journal including journal entries for write requests received on said unique input port; and

a primary journal incorporating portions of said plurality of partial journals associated with write requests that have occurred earlier in time than those entries currently included in said partial journals,

wherein each partial journal entry included in a partial journal includes a data value of a location prior to applying an update of a write request corresponding to said each partial journal entry.

22. A method for providing a backup copy of data on a primary storage comprising:

receiving, from one of a plurality of sources, a write request for a location in primary storage;

receiving a copy of data at said location on said primary storage;

journaling said write request in one of a plurality of partial journals in a partial journal entry including said copy prior to performing said write request, each of said plurality of partial journals being associated with journaled write requests received from a different one of said plurality of sources, wherein each of said plurality of partial journals is emptied to a primary journal including partial journal entries and merged partial journal entries, each of the merged partial journal entries representing application of one or more data modifications for a time period to a storage location; and

updating said primary storage in accordance with said write request.

23. The method of claim 22, wherein partial journal entries in said partial journals have an associated time stamp value, said partial journal entries being ordered in accordance with associated time stamp values.

24. The method of claim 23, further comprising:

merging a portion of said partial journal entries for a predetermined time period for write requests to a storage location producing a merged partial journal entry for said storage location.

25. The method of claim 24, wherein a backup copy of said primary storage with respect to a point in time equal to or less than a current time value is produced using said plurality of partial journals and said primary storage without maintaining a copy of said primary storage.

26. A method of creating a point-in-time copy of a primary storage comprising:

receiving journal entries corresponding to write requests formed using a plurality of distributed partial journals, each partial journal being associated with a portion of write requests, said journal entries being ordered in accordance with a time value associated with each of said write requests, each of said journal entries including a value for a corresponding portion of primary storage before performing a write request;

creating a map of pointers to said primary storage wherein each pointer is associated with a particular location identifier; and

performing, for each of said journal entries including a location identifier and a data value prior to performing a write request;

determining a corresponding map pointer for each location identifier included in a journal entry; and

if said corresponding map pointer points to a value in the primary storage, then adjusting the map pointer to point to said data value in said each journal entry.

27. The method of claim 26, wherein each of said partial journals includes partial journal entries for write requests received on a particular input port to a switch, said each partial journal being stored in a portion of memory associated with said input port, wherein journaling of write requests to that input port is performed by a processor associated with said input port.

28. A computer readable medium encoded with a computer program that provides a backup copy of data on a primary storage, the computer program comprising code that:

receives a write request from one of a plurality of sources;

journals said write request in one of a plurality of partial journals, wherein each of said plurality of partial journals is used to journal a portion of write requests received from a different one of said plurality of sources, wherein each of said plurality of partial journals is emptied to a primary journal including partial journal entries and merged partial journal entries, each of the merged partial journal entries representing application of one or more data modifications for a time period to a storage location; and

updates said primary storage in accordance with said write request.

29. The computer readable medium of claim 28, further comprising code that:

receives each of said plurality of partial journals having a plurality of partial journal entries; and

merges at least a portion of said plurality of partial journal entries from one or more of said plurality of partial journal entries in accordance with a predetermined merging time period forming one of said merged partial journal entries for a particular storage location, said one merged partial journal entry reflecting application of data modifications for said particular storage location for said predetermined merging time period.

30. The computer readable medium of claim 28, wherein each partial journal entry in said plurality of partial journals includes a time stamp associated with a write request.

31. The computer readable medium of claim 28, further comprising code that:

periodically flushes said plurality of partial journals to said primary journal.

32. The computer readable medium of claim 28, wherein said write request is made with respect to a copy of data included on said primary storage.

33. The computer readable medium of claim 32, wherein each of said time stamp values is obtained using a phase lock loop processing technique and using actual time values periodically broadcast from a stable time base.

34. The computer readable medium of claim 28, wherein said primary storage is one of:

a physical device, a logical device, a virtual device, and a portion of one or more different devices.

35. The computer readable medium of claim 28, wherein each of said partial journal entries includes a time stamp value associated with a corresponding write request, wherein each time stamp value included in a first partial journal is synchronized with other time stamp values included in a second different partial journal.

36. The computer readable medium of claim 35, wherein each of said time stamp values is obtained using a phase lock loop processing technique and using actual time values periodically broadcast from a stable time base.

37. The computer readable medium of claim 28, wherein said journaling and said updating are issued simultaneously.

38. The computer readable medium of claim 37, wherein each partial journal entry includes a status indicator reflecting a status of both said journaling and said updating.

39. A computer readable medium encoded with a computer program that provides a backup copy of data on a primary storage, the computer program comprising code that:

receives a write request;

journals said write request in one of a plurality of partial journals;

updates said primary storage in accordance with said write request;

receives each of said plurality of partial journals having a plurality of partial journal entries; and

merges said plurality of partial journal entries in accordance with a predetermined merging time period forming a merged partial journal entry for a particular storage location, said merged partial journal entry reflecting application of data modifications for said particular storage location for said predetermined merging time period;

wherein a primary journal includes a first portion of for received partial journal entries and a second portion for merged partial journal entries.

40. A computer readable medium encoded with a computer program that provides a backup copy of data on a primary storage, the computer program comprising code that:

receives a write request;

journals said write request in one of a plurality of partial journals;

updates said primary storage in accordance with said write request; and

periodically flushes said plurality of partial journals to a primary journal,

wherein contents of each partial journal are included in a primary journal when said each partial journal is filled in accordance with a predetermined capacity level.

41. The computer readable medium of claim 40, wherein partial journal entries within said primary journal are ordered in accordance with a time stamp value associated with each of said partial journal entries, and each of said merged partial journal entries within said primary journal are ordered in accordance with a time stamp value associated with each of said merged journal entries.

42. A computer readable medium encoded with a computer program that provides a backup copy of data on a primary storage, the computer program comprising code that:

receives a write request;

journals said write request in one of a plurality of partial journals; and

updates said primary storage in accordance with said write request,

wherein each of said plurality of partial journals is associated with an input port in a switch and includes journal entries for all write requests received on said associated input port.

43. The computer readable medium of claim 42, wherein journaling of write requests received on an input port is performed by a processor associated with said input port and a partial log associated with said input port is stored on a portion of memory associated with said input port.

44. A computer readable medium encoded with a computer program that provides a backup copy of data on a primary storage, the computer program comprising code that:

receives a write request;

journals said write request in one of a plurality of partial journals;

updates said primary storage in accordance with said write request;

receives each of said plurality of partial journals having a plurality of partial journal entries; and

merges said plurality of partial journal entries in accordance with a predetermined merging time period forming a merged partial journal entry for a particular storage location, said merged partial journal entry reflecting application of data modifications for said particular storage location for said predetermined merging time period;

wherein each partial journal is associated with an input port and all write requests from that input port are journaled in the corresponding each partial journal and wherein each merged partial journal entry includes an indicator indicating the validity of journal entries received from each input port in accordance with a time stamp value associated with said merged partial journal entry.

45. A computer readable medium encoded with a computer program that provides a backup copy of data on a primary storage, the computer program comprising code that:

receives, from one of a plurality of sources, a write request for a location in primary storage;

receives a copy of data at said location on said primary storage;

journals said write request in one of a plurality of partial journals in a partial journal entry including said copy prior to performing said write request, each of said plurality of partial journals being associated with journaled write requests received from a different one of said plurality of sources, wherein each of said plurality of partial journals is emptied to a primary journal including partial journal entries and merged partial journal entries, each of the merged partial journal entries representing application of one or more data modifications for a time period to a storage location; and

updates said primary storage in accordance with said write request.

46. The computer readable medium of claim 45, wherein partial journal entries in said partial journals have an associated time stamp value, said partial journal entries being ordered in accordance with associated time stamp values.

47. The computer readable medium of claim 46, further comprising code that:

merges a portion of said partial journal entries for a predetermined time period for write requests to a storage location producing a merged partial journal entry for said storage location.

48. The computer readable medium of claim 47, wherein a backup copy of said primary storage with respect to a point in time equal to or less than a current time value is produced using said plurality of partial journals and said primary storage without maintaining a copy of said primary storage.

49. A computer readable medium encoded with a computer program that creates a point-in-time copy of a primary storage, the computer program comprising code that:

receives journal entries corresponding to write requests formed using a plurality of distributed partial journals, each partial journal being associated with a portion of write requests, said journal entries being ordered in accordance with a time value associated with each of said write requests, each of said journal entries including a value for a corresponding portion of primary storage before performing a write request;

creates a map of pointers to said primary storage wherein each pointer is associated with a particular location identifier; and

performs, for each of said journal entries including a location identifier and a data value prior to performing a write request;

determining a corresponding map pointer for each location identifier included in a journal entry; and

if said corresponding map pointer points to a value in the primary storage, then adjusting the map pointer to point to said data value in said each journal entry.

50. The computer readable medium of claim 49, wherein each of said partial journals includes partial journal entries for write requests received on a particular input port to a switch, said each partial journal being stored in a portion of memory associated with said input port, wherein journaling of write requests to that input port is performed by a processor associated with said input port.