Managing log transactions in storage systems

• US 9,697,219 B1
• Filed: 12/29/2014
• Issued: 07/04/2017
• Est. Priority Date: 12/29/2014
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Armangau, Philippe, Bono, Jean-Pierre, Wang, Yubing, Seibel, Christopher
• Primary Examiner(s): Stevens, Robert
• Application Number:
  US14/584,341
• Time in Patent Office: 918 Days
• Field of Search: 707823
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD

Abstract

A method is used in managing log transactions in storage systems. A first portion of a transaction log space is allocated from a persistent journal associated with a file system to a first core of a multi-core storage processor and a second portion of the transaction log space is allocated from the persistent journal to a second core of the multi-core storage processor. A set of per-core bitmaps is used for allocating a set of transaction log entries from the persistent journal. Each per-core bitmap is associated with a respective core of the multi-core storage processor. A per-core bitmap of the set of per-core bitmaps associated with a core of the multi-core storage processor is used for allocating a transaction log entry for a write request performed by the core.

Claims

1. A method for use in managing log transactions in storage systems, the method comprising:

allocating a first portion of a transaction log space from a persistent journal associated with a file system to a first core of a multi-core storage processor and a second portion of the transaction log space from the persistent journal to a second core of the multi-core storage processor; and

using a set of per-core bitmaps for allocating a set of transaction log entries from the persistent journal, wherein each per-core bitmap is associated with a respective core of the multi-core storage processor, wherein a per-core bitmap of the set of per-core bitmaps associated with a core of the multi-core storage processor is used for allocating a transaction log entry for a write request performed by the core.

2. The method of claim 1, wherein a first per-core bitmap of the set of per-core bitmaps is used for allocating a transaction log entry from the first portion of the transaction log space for a write request performed by the first core, wherein a second per-core bitmap of the set of per-core bitmaps is used for allocating a transaction log entry from the second portion of the transaction log space for a write request performed by the second core.

3. The method of claim 1, further comprising:

using a global data log bitmap associated with the file system for tracking portions of transaction log space of the persistent journal allocated to the set of per-core bitmaps.

4. The method of claim 1, wherein the multi-core storage processor resides in a storage system and includes a set of cores.

5. The method of claim 1, wherein a global data cache includes a set of global data block cache descriptors for storing information for a set of buffer cache objects stored in the volatile memory of the data storage system, wherein a global data block cache descriptor of the set of global data block cache descriptors is associated with a buffer cache object of the set of buffer cache objects, wherein a buffer cache object stores a data block of the file of the file system.

6. The method of claim 1, wherein the persistent journal is a data transaction log, wherein the data transaction log is apportioned into a set of sections, wherein a section of the set of sections include a header area and a record area, wherein the header area includes a chunk header and a set of data log headers, wherein the record area includes a set of data log records, wherein a data log record of the set of data log records includes data and a data log header associated with the data.

7. The method of claim 1, wherein the persistent journal is a file system transaction log for a file system, wherein the file system transaction log stores metadata transactions for the file system.

8. The method of claim 1, wherein the set of per-core bitmaps are processed concurrently for performing a set of write I/Os directed to the multi-core storage processor.

9. The method of claim 1, further comprising:

determining whether each entry of a portion of the transaction log space allocated to a per-core bitmap of the set of per-core bitmaps has been consumed;

based on the determination, determining whether to clear bits of the per-core bitmap; and

based on the determination, allocating another portion of the transaction log space to the per-core bitmap and associating the bits to the other portion of the transaction log space.

10. The method for claim 1, further comprising:

receiving a request to replay log transactions of the persistent journal; and

creating a set of log transaction entries in a volatile memory of a storage system in an ordered arrangement, wherein the set of log transaction entries are created from log transaction entries allocated using each per-core bitmap of the set of per-core bitmaps, wherein a timestamp information stored in each log transaction entry is used to arrange the set of log transaction entry in the ordered arrangement.

11. A system for use in managing log transactions in storage systems, the system comprising:

first logic allocating a first portion of a transaction log space from a persistent journal associated with a file system to a first core of a multi-core storage processor and a second portion of the transaction log space from the persistent journal to a second core of the multi-core storage processor; and

second logic using a set of per-core bitmaps for allocating a set of transaction log entries from the persistent journal, wherein each per-core bitmap is associated with a respective core of the multi-core storage processor, wherein a per-core bitmap of the set of per-core bitmaps associated with a core of the multi-core storage processor is used for allocating a transaction log entry for a write request performed by the core.

12. The system of claim 11, wherein a first per-core bitmap of the set of per-core bitmaps is used for allocating a transaction log entry from the first portion of the transaction log space for a write request performed by the first core, wherein a second per-core bitmap of the set of per-core bitmaps is used for allocating a transaction log entry from the second portion of the transaction log space for a write request performed by the second core.

13. The system of claim 11, further comprising:

third logic using a global data log bitmap associated with the file system for tracking portions of transaction log space of the persistent journal allocated to the set of per-core bitmaps.

14. The system of claim 11, wherein the multi-core storage processor resides in a storage system and includes a set of cores.

15. The system of claim 11, wherein a global data cache includes a set of global data block cache descriptors for storing information for a set of buffer cache objects stored in the volatile memory of the data storage system, wherein a global data block cache descriptor of the set of global data block cache descriptors is associated with a buffer cache object of the set of buffer cache objects, wherein a buffer cache object stores a data block of the file of the file system.

16. The system of claim 11, wherein the persistent journal is a data transaction log, wherein the data transaction log is apportioned into a set of sections, wherein a section of the set of sections include a header area and a record area, wherein the header area includes a chunk header and a set of data log headers, wherein the record area includes a set of data log records, wherein a data log record of the set of data log records includes data and a data log header associated with the data.

17. The system of claim 11, wherein the persistent journal is a file system transaction log for a file system, wherein the file system transaction log stores metadata transactions for the file system.

18. The system of claim 11, wherein the set of per-core bitmaps are processed concurrently for performing a set of write I/Os directed to the multi-core storage processor.

19. The system of claim 11, further comprising:

third logic determining whether each entry of a portion of the transaction log space allocated to a per-core bitmap of the set of per-core bitmaps has been consumed;

fourth logic determining, based on the determination, whether to clear bits of the per-core bitmap; and

fifth logic allocating, based on the determination, another portion of the transaction log space to the per-core bitmap and associating the bits to the other portion of the transaction log space.

20. The system for claim 11, further comprising:

third logic receiving a request to replay log transactions of the persistent journal; and

fourth logic creating a set of log transaction entries in a volatile memory of a storage system in an ordered arrangement, wherein the set of log transaction entries are created from log transaction entries allocated using each per-core bitmap of the set of per-core bitmaps, wherein a timestamp information stored in each log transaction entry is used to arrange the set of log transaction entry in the ordered arrangement.