System and method for granular deduplication

• US 9,733,836 B1
• Filed: 02/05/2016
• Issued: 08/15/2017
• Est. Priority Date: 02/11/2015
• Status: Active Grant

• Current Assignee: Innovations In Memory LLC
• Original Assignee: Violin Memory Incorporated
• Inventors: Garg, Amit, Stoakes, Timothy, Ratna, Vikas
• Primary Examiner(s): Verbrugge, Kevin
• Application Number:
  US15/017,046
• Time in Patent Office: 557 Days
• Field of Search: None
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD

Abstract

A method of managing a memory system is described, the memory system having a processor with a client interface, a buffer memory, and a storage memory. A LUN (logical unit) is exposed to the client, the LUN having a memory size, and QoS specification. When the processor receives a command from the client to write data to the LUN, determining if the QoS specification includes deduplication and: if the QoS does not include deduplication, processing the command by storing the data in the storage memory and creating metadata uniquely referencing a storage location of the data that is written; or if the QoS includes deduplication, determine if deduplication is possible while meeting a latency value as specified in the QoS specification and performing deduplication.

Claims

1. A method Of managing a memory system, comprising:

providing a memory system with a processor having a client interface, a buffer memory, and a storage memory;

allocating a logical unit (LUN) to the client, the LUN having a memory size, and QoS specification;

receiving a command from the client to write data to the LUN; and

determining if the QoS specification includes deduplication and;

if the QoS does not include deduplication, process the command storing the data in the storage memory and creating metadata uniquely referencing a storage location of the data that is written;

or

if the QoS includes deduplication, determine if deduplication is possible while meeting a latency value as specified in the QoS specification and;

if deduplication is possible, performing deduplication and creating metadata relating a user address to a physical location where a copy of the data that is written is stored;

or,

if deduplication is not possible, storing undeduplicated data in a backing LUN associated with a deduplicated LUN, and

draining undeduplicated data stored in the backing LUN when deduplication is possible by performing deduplication.

2. A method of managing a memory system, comprising:

providing a memory system with a processor having a client interface, a buffer memory, and a storage memory;

maintaining a metadata relating a hash value to a location in the memory system where the data is stored;

associating a plurality of logical units (LUN), with a SIR (single instance repository), each LUN being exposed to a user and having a stub LUN (SL) relating a user logical block address (LBA) to a hash value of data or to a location in the memory system where the data is stored,

wherein hash values comprise hash values that are referenced by at least one of a plurality of LBAs in a LUN or by a plurality of LUNs; and

the hash values are garbage collected by;

determining whether the hash value is referenced by one or more LUNs and;

in accordance with a policy determined by user quality of service requirements (QoS),

evicting the hash value and associated stored data if the hash value is not referenced by any LUN;

or

evicting the hash value and associated stored data if the hash value is not referenced by any LUN for more than one garbage collection cycle;

or

evicting the hash value if the hash value is referenced by a LUN and the hash value has not been referenced for a predetermined period of time;

maintaining the hash value and associated stored data if the hash value is referenced by more than one LUN.

3. The method of claim 2 wherein, the metadata is stored in the SIR.