Performance improvement of a capacity optimized storage system using a performance segment storage system and a segment storage system

• US 10,013,167 B2
• Filed: 08/13/2015
• Issued: 07/03/2018
• Est. Priority Date: 09/22/2009
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Patterson, R. Hugo
• Primary Examiner(s): Rigol, Yaima
• Application Number:
  US14/825,845
• Publication Number:
  US 20160034200A1
• Time in Patent Office: 1,055 Days
• Field of Search: None
• CPC Class Codes: G06F 12/121 using replacement algorithms
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0679 Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays

Abstract

A system for storing data includes a performance storage unit, a performance storage transfer manager, a segment storage system, and a performance segment storage unit. The performance storage unit is for storing a data stream or a data block in. The data stream or the data block includes one or more data items. The performance storage transfer manager manages a transfer of the one or more data items to be automatically stored in the segment storage system over a network. The segment storage system is for storing a stored data item of the one or more data items as a set of segments. The performance segment storage unit is for storing the set of segments in the event that the stored data item has been stored using the segment storage system.

Claims

1. A system for storing data, comprising:

a performance storage unit configured to store a data stream or a data block, wherein the data stream or the data block comprises one or more data items and wherein the performance storage unit responds efficiently to requests for the data stream or the data block to be stored and retrieved;

a performance storage transfer manager associated with the performance storage unit, wherein the performance storage transfer manager manages a transfer of the one or more data items to be automatically stored in a segment storage system over a network, wherein the performance storage transfer manager is configured to determine whether a stored item of the one or more data items is to be stored in a segment storage system over the network in connection with a set of deduplicated segments and wherein the performance storage transfer manager also manages a transfer of the set of deduplicated segments back to the performance storage unit to improve response performance of the performance storage unit;

the segment storage system configured to store a stored data item of the one or more data items as a set of segments over the network in response to the performance storage transfer manager determining a stored item of the one or more data items is to be stored in a segment storage system, wherein the segment storage system deduplicates the set of segments; and

a performance segment storage unit configured to store the set of segments in the event that the stored data item has been stored using the segment storage system, wherein the performance segment storage unit stores the set of deduplicated segments received from the segment storage system over the network in the event the stored data item is required to be available for high performance access, wherein the performance storage transfer manager manages a transfer of the set of deduplicated segments between the performance segment storage unit and performance storage unit.

2. A system as in claim 1, wherein the performance storage unit removes the stored data item after that the stored data item is stored by the segment storage system.

3. A system as in claim 1, wherein the segment storage system is replicated using a replica segment storage system.

4. A system as in claim 3, wherein the replica segment storage system stores a copy of a snapshot of the stored data item.

5. A system as in claim 3, wherein the replication of the segment storage system occurs according to a replication protocol.

6. A system as in claim 1, wherein the performance storage unit comprises one or more of the following:

a flash memory, a fiber channel storage device, a persistent memory, a block addressable memory, or a file system organized memory.

7. A system as in claim 1, wherein a snapshot is performed according to a snapshot protocol.

8. A system as in claim 1, wherein a second stored data item of the one or more data items is not captured using any snapshot.

9. A system as in claim 1, wherein a snapshot is taken using a log of changes to the one or more data items stored by the performance storage unit.

10. A system as in claim 9, wherein the performance storage unit removes the stored data item once the stored data item is stored by taking the snapshot on the segment storage system.

11. A system as in claim 9, wherein the segment storage system stores the snapshot using variable length segments.

12. A system as in claim 1, wherein the segment storage system has content addressable segments.

13. A system as in claim 12, wherein the content addressable segments are addressed using a fingerprint of a segment.

14. A system as in claim 1, wherein a data representation in the performance storage unit is not the same as the data representation in the segment storage system.

15. A system as in claim 14, wherein the data representation in the performance storage unit comprises an uncompressed data representation and the data representation in the segment storage system comprises a compressed data representation.

16. A system as in claim 14, wherein the data representation in the segment storage system comprises a content addressable data representation.

17. A system as in claim 1, wherein the performance segment storage unit includes data stored using a compressed data representation.

18. The system of claim 1, wherein the segment storage system stores a fingerprint associated with at least one of the one or more data items.

19. A method for storing data, comprising:

storing a data stream or a data block in a performance storage unit, wherein the data stream or the data block comprises one or more data items and wherein the performance storage unit responds efficiently to requests for the data stream or the data block to be stored and retrieved;

managing, using a performance storage transfer manager associated with the performance storage unit, a transfer of the one or more data items to be automatically stored in a segment storage system over a network, wherein the managing of the transfer includes determining whether a stored item of the one or more data items is to be stored in a segment storage system over the network in connection with a set of deduplicated segments;

managing, using the performance storage transfer manager, a transfer of the set of deduplicated segments back to the performance storage unit to improve response performance of the performance storage unit;

storing a stored data item of the one or more data items as a set of segments using the segment storage system over the network in response to the performance storage transfer manager determining a stored item of the one or more data items is to be stored in a segment storage system, wherein the storing of the stored data item includes deduplicating the set of segments; and

in the event that the stored data item has been stored using the segment storage system, storing the set of segments in a performance segment storage unit over the network, including determining that the stored data item is to be accessible with high performance, and in the event that it is determined that the stored data item is to be accessible with high performance, storing the set of deduplicated segments received from the segment storage system over the network, wherein the performance storage transfer manager manages a transfer of the set of deduplicated segments between the performance segment storage unit and performance storage unit.

20. A computer program product for storing data, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:

storing a data stream or a data block in a performance storage unit, wherein the data stream or the data block comprises one or more data items and wherein the performance storage unit responds efficiently to requests for the data stream or the data block to be stored and retrieved;

managing, using a performance storage transfer manager associated with the performance storage unit, a transfer of the one or more data items to be automatically stored in a segment storage system over a network, wherein the managing of the transfer includes determining whether a stored item of the one or more data items is to be stored in a segment storage system over the network in connection with a set of deduplicated segments;

managing, using the performance storage transfer manager, a transfer of the set of deduplicated segments back to the performance storage unit to improve response performance of the performance storage unit;

storing a stored data item of the one or more data items as a set of segments using the segment storage system over the network in response to the performance storage transfer manager determining a stored item of the one or more data items is to be stored in a segment storage system, wherein the storing of the stored data item includes deduplicating the set of segments; and

in the event that the stored data item has been stored using the segment storage system, storing the set of segments in a performance segment storage unit over the network, including determining that the stored data item is to be accessible with high performance, and in the event that it is determined that the stored data item is to be accessible with high performance, storing the set of deduplicated segments received from the segment storage system over the network, wherein the performance storage transfer manager manages a transfer of the set of deduplicated segments between the performance segment storage unit and performance storage unit.