RECOVERY MECHANISM FOR LOW LATENCY METADATA LOG

• US 20180285198A1
• Filed: 03/30/2017
• Published: 10/04/2018
• Est. Priority Date: 03/30/2017
• Status: Active Grant

• Current Assignee: Pavilion Data Systems, Inc.
• Original Assignee: Pavilion Data Systems, Inc.
• Inventors: Dantkale, Suhas, Satish, Venkeepuram R., Govindasamy, Raghuraman
• Granted Patent: US 10,191,812 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 12/0246 in block erasable memory, e.g. flash memory
• CPC Class Codes: G06F 12/0811 with multilevel cache hierarchies
• CPC Class Codes: G06F 12/0868 Data transfer between cache memory and other subsystems, e.g. storage devices or host systems
• CPC Class Codes: G06F 12/1063 the data cache being concurrently virtually addressed
• CPC Class Codes: G06F 12/12 Replacement control
• CPC Class Codes: G06F 12/128 adapted to multidimensional cache systems, e.g. set-associative, multicache, multiset or multilevel
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 2212/1024 Latency reduction
• CPC Class Codes: G06F 2212/1032 Reliability improvement, data loss prevention, degraded operation etc
• CPC Class Codes: G06F 2212/283 Plural cache memories
• CPC Class Codes: G06F 2212/285 Redundant cache memory
• CPC Class Codes: G06F 2212/312 In storage controller
• CPC Class Codes: G06F 2212/621 Coherency control relating to peripheral accessing, e.g. from DMA or I/O device
• CPC Class Codes: G06F 2212/681 Multi-level TLB, e.g. microTLB and main TLB
• CPC Class Codes: G06F 2212/70 Details relating to dynamic memory management
• CPC Class Codes: G06F 2212/7203 Temporary buffering, e.g. using volatile buffer or dedicated buffer blocks

Abstract

A storage server includes an IO controller, a management controller and physical drives. The IO controller generates multiple metadata updates and writes a cache entry that includes the multiple metadata updates to a first cache in memory of the management controller. The IO controller additionally writes a copy of the cache entry to a second cache in a memory of the IO controller and increments a commit pointer in the first and second caches to indicate that the metadata updates are committed.

Claims

1. A storage server comprising:

a plurality of physical drives;

a management controller comprising a first memory; and

an input/output (IO) controller operatively coupled to the plurality of physical drives and to the management controller, the IO controller comprising a second memory, wherein the IO controller is to;

generate a first metadata update for a first page of a first metadata table, wherein the first metadata update is associated with storage of first data in at least one of a first physical block address (PBA) on a first physical drive of the plurality of physical drives or a first virtual block address (VBA) mapped to the first PBA;

generate a second metadata update for a second page of one of the first metadata table or a second metadata table, wherein the second metadata update is associated with at least one of the first VBA or the first PBA;

write a first cache entry to a first cache in a first memory of the management controller, the first cache entry comprising the first metadata update and the second metadata update;

write a copy of the first cache entry to a second cache in a second memory of the IO controller, wherein the second cache is a copy of the first cache; and

increment a commit pointer in the first cache and the second cache to indicate that the data has been stored in the first PBA and that all metadata updates associated with storage of the data at the first PBA have been committed.

2. The storage server of claim 1, wherein the IO controller is further to:

periodically read contents of the first cache to verify that writes to the first cache are successful; and

responsive to receiving an error associated with reading the contents of the first cache, determine that the management controller has experienced a failure.

3. The storage server of claim 2, further comprising:

a second management controller, wherein the second management controller is to;

send a notice to the IO controller indicating that the second management controller is assuming a role of the management controller; and

map a third cache in a third memory of the second management controller to a virtual address space of the IO controller;

wherein the IO controller is to write the first cache entry to the third cache of the second management controller.

4. The storage server of claim 1, wherein the management controller is further to:

detect that the IO controller has experienced a failure; and

write a second copy of the first cache entry to a third cache in at least one of the plurality of physical drives, wherein the third cache is a second copy of the first cache.

5. The storage server of claim 4, wherein to detect that the IO controller has failed the management controller is to:

write a heartbeat message to an area of the first memory that is accessible to the IO controller; and

determine that a response message has not been written to the area of the first memory within a threshold time period.

6. The storage server of claim 4, wherein:

the first cache entry is one of a plurality of cache entries stored in a first page in the first cache; and

the IO controller is to perform the following after recovering from the failure;

read the second copy of the first cache entry from the third cache;

overwrite contents of the first cache with the second copy of the first cache; and

write the second copy of the first cache to the second cache.

7. The storage server of claim 6, wherein the IO controller is further to:

determine, based on a location of the commit pointer, that the first cache entry has been committed;

write the first metadata update to at least one of the first metadata table or a first metadata table cache for the first metadata table;

write the second metadata update to at least one of the second metadata table or a second metadata table cache for the second metadata table; and

free one or more areas of the first cache and the second cache on which the first cache entry was stored.

8. The storage server of claim 7, wherein prior to writing the first metadata update and the second metadata update, the IO controller is further to:

read pages of a third cache located on one or more of the plurality of physical drives, each of the pages comprising a plurality of cache entries, wherein the third cache is larger than the first cache and the second cache; and

write a plurality of additional metadata updates from the plurality of cache entries in the pages of the third cache to at least one of the first metadata table or the second metadata table.

9. The storage server of claim 6, wherein the first cache is a circular log comprising a plurality of pages, and wherein the IO controller is further to:

determine a starting page of the plurality of pages based on a start page identifier and an end page of the plurality of pages based on an end page identifier, wherein metadata updates included in the plurality of pages are performed sequentially starting from metadata updates in the start page.

10. The storage server of claim 4, wherein:

the management controller is to;

assign the virtual drive to a second IO controller; and

notify the second IO controller of a location of the third cache in the at least one of the plurality of physical drives; and

the second IO controller is to;

read the second copy of the first cache entry from the third cache;

perform the first metadata update; and

perform the second metadata update.

11. The storage server of claim 1, wherein the IO controller is further to:

receive a message from the management controller indicating an imminent loss of power; and

write a second copy of the first cache entry to a third cache in at least one of the plurality of physical drives, wherein the third cache is a second copy of the first cache.

12. The storage server of claim 1, wherein the IO controller is further to:

determine that the first VBA is already mapped to a second PBA; and

generate a third metadata update for the second metadata table, wherein the third metadata update indicates that the second PBA is invalid, and wherein the first cache entry further comprises the third metadata update.

13. A method comprising:

sending, by a first controller, a command to store data at a first physical block address (PBA) of a first physical drive that is to be mapped to a first virtual block address (VBA) of a virtual drive;

generating, by the first controller, a first metadata update for a first metadata table associated with the virtual drive, wherein the first metadata update indicates a mapping of the first VBA to the first PBA;

generating, by the first controller, a second metadata update for a second metadata table, wherein the second metadata table identifies statuses of PBAs, and wherein the second metadata update indicates that the first PBA is valid;

writing a first cache entry to a first cache in a first memory of a second controller operatively coupled to the first controller, the first cache entry comprising the first metadata update and the second metadata update;

writing a copy of the first cache entry to a second cache in a second memory of the first controller, wherein the second cache is a copy of the first cache; and

incrementing a commit pointer in the first cache and the second cache to indicate that the data has been stored in the first PBA and that all metadata updates associated with storage of the data at the first PBA have been committed.

14. The method of claim 13, further comprising:

periodically reading contents of the first cache by the first controller to verify that writes to the first cache are successful; and

responsive to receiving an error associated with reading the contents of the first cache, determining by the first controller that the second controller has experienced a failure.

15. The method of claim 14, further comprising:

receiving, by the first controller, a notice from a third controller that the third controller is assuming a role of the second controller;

mapping a third cache in a third memory of the third controller to a virtual address space of the first controller; and

writing the first cache entry to the third cache of the third controller.

16. The method of claim 13, further comprising:

detecting, by the second controller, that the first controller has experienced a failure; and

writing, by the second controller, a second copy of the first cache entry to a third cache in at least one of the first physical drive or a second physical drive, wherein the third cache is a second copy of the first cache.

17. The method of claim 16, wherein detecting that the first controller has failed comprises:

writing, by the second controller, a heartbeat message to an area of the first memory that is accessible to the first controller; and

determining, by the second controller, that a response message has not been written to the area of the first memory within a threshold time period.

18. The method of claim 16, further comprising:

detecting that the first controller has recovered from the failure;

reading the second copy of the first cache from the third cache;

overwriting contents of the first cache with the second copy of the first cache; and

writing the second copy of the first cache to the second cache.

19. The method of claim 18, further comprising:

determining, based on a location of the commit pointer, that the first cache entry has been committed;

writing the first metadata update to at least one of the first metadata table or a first metadata table cache for the first metadata table;

writing the second metadata update to at least one of the second metadata table or a second metadata table cache for the second metadata table; and

freeing one or more areas of the first cache and the second cache on which the first cache entry was stored.

20. The method of claim 19, further comprising performing the following prior to writing the first metadata update and the second metadata update:

reading pages of a third cache located on one or more of the plurality of physical drives, each of the pages comprising a plurality of cache entries, wherein the third cache is larger than the first cache and the second cache; and

writing a plurality of additional metadata updates from the plurality of cache entries in the pages of the third cache to at least one of the first metadata table or the second metadata table.

21. The method of claim 18, wherein the first cache is a circular log comprising a plurality of pages, the method further comprising:

determining a starting page of the plurality of pages based on a start page identifier and an end page of the plurality of pages based on an end page identifier, wherein metadata updates included in the plurality of pages are performed sequentially starting from metadata updates in the start page.

22. The method of claim 16, further comprising:

assigning, by the second controller, the virtual drive to a third controller;

notifying the third controller of a location of the third cache in at least one of the first physical drive or the second physical drive; and

reading, by the third controller, the second copy of the first cache entry from the third cache.

23. The method of claim 13, further comprising:

receiving, by the first controller, a message from the second controller indicating an imminent loss of power; and

writing a second copy of the first cache entry to a third cache in at least one of the first physical drive or a second physical drive, wherein the third cache is a second copy of the first cache.

24. The method of claim 13, wherein the first controller is an input/output (IO) controller of a storage server and the second controller is a management controller of the storage server.

25. The method of claim 13, further comprising:

determining that the first VBA is already mapped to a second PBA; and

generating a third metadata update for the second metadata table, wherein the third metadata update indicates that the second PBA is invalid, and wherein the first cache entry further comprises the third metadata update.