Storage system

• US 10,656,836 B2
• Filed: 09/17/2018
• Issued: 05/19/2020
• Est. Priority Date: 11/20/2017
• Status: Active Grant

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Akutsu, Hiroaki, Yamamoto, Akira
• Primary Examiner(s): Torres, Joseph D
• Application Number:
  US16/133,039
• Publication Number:
  US 20190155684A1
• Time in Patent Office: 610 Days
• CPC Class Codes: G06F 11/1048 using arrangements adapted for a specific error detection or correction feature
• CPC Class Codes: G06F 12/0246 in block erasable memory, e.g. flash memory
• CPC Class Codes: G06F 2212/401 Compressed data
• CPC Class Codes: G06F 2212/403 Error protection encoding, e.g. using parity or ECC codes
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0638 Organizing or formatting or addressing of data
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0679 Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]

Abstract

In write processing of a data set group to be written to be one or more data sets to be written, a storage system performs encoding processing including processing for generating a data model showing regularity of the data set group to be written and having one or more input values as an input and the data set group as an output. In the write processing, the storage system writes the data model generated in the encoding processing and associated with a key of the data set group to be written.

Claims

1. A storage system comprising:

a storage unit including at least one of a memory unit to be one or more memories and a PDEV (non-volatile physical storage device) unit to be one or more PDEVs; and

a processor unit to be one or more processors connected to the storage unit,

wherein, in write processing of a data set group to be written to be one or more data sets to be written, the processor unit

performs encoding processing including processing for generating a data model showing regularity of the data set group to be written and having one or more input values as an input and the data set group as an output (W1), and

writes the data model generated in the encoding processing and associated with a key of the data set group to be written to the storage unit (W2).

2. The storage system according to claim 1, wherein

for each data set group, a key of the data set group includes an address of the data set group, and

in read processing of a data set group to be read, the processor unit

specifies a data model corresponding to a key including an address of a read source from the storage unit (R1),

performs decoding processing including processing for restoring a data set group using the specified data model (R2), and

returns the data set group restored in the decoding processing (R3).

3. The storage system according to claim 2, wherein

the encoding processing in (W1) includes processing for generating difference information to be a difference between the data set group to be written and a data set group output from the data model,

in (W2), the processor unit writes the data model associated with the key of the data set group to be written and the generated difference information to the storage unit, and

the data set group restored in the decoding processing in (R2) is a data set group obtained by reflecting the difference information associated with the data model in the data set group output from the data model.

4. The storage system according to claim 2, wherein

the encoding processing in (W1) includes processing for generating error information to be information based on an error between the data set group to be written and a data set group output from the data model and an allowable error,

in (W2), the processor unit writes the data model associated with the key of the data set group to be written and the generated error information to the storage unit, and

the data set group restored in the decoding processing in (R2) is a data set group obtained by correcting the data set group output from the data model using the error information associated with the data model.

5. The storage system according to claim 4, wherein

the storage unit includes a memory unit to be one or more memories,

the processor unit provides a front-end layer writing the input data set group to be written to the memory unit, and

the front-end layer

reads a data model of a data set group to be read to the memory unit through a back-end layer writing the data model of the data set group to be written to the PDEV unit to be one or more PDEVs, and

performs (R2).

6. The storage system according to claim 4, wherein

for each data model, the processor unit compresses the error information and writes the error information to the storage unit,

for each data model,

the error information includes one or more correction values corresponding to one or more output data sets to be one or more data sets forming a data set group output from the data model,

when a correction error increases, a correction value corresponding to the increased correction error approaches a predetermined value and/or when the allowable error increases, each of one or more correction values approaches the predetermined value,

an output data set in which an error is within a range of the allowable error is corrected on the basis of the error and the range of the allowable error and an error of the corrected output data set is the correction error for the output data set,

the processor unit performs at least one of first error adjustment processing and second error adjustment processing regularly or irregularly,

for at least one data model stored in the storage unit, the first error adjustment processing includes

processing for increasing the correction error of an output data set having the correction error among one or more output data sets to be one or more data sets forming a data set group output from the data model, within the range of the allowable error, according to a predetermined rule,

processing for decompressing the error information corresponding to the data model,

processing for updating the decompressed error information on the basis of the increased correction error, and

processing for compressing the updated error information and writing the error information, and

for at least one data model stored in the storage unit, the second error adjustment processing includes

processing for increasing the allowable error corresponding to the data model,

processing for decompressing error information corresponding to the data model,

processing for updating the decompressed error information on the basis of the increased allowable error, and

processing for compressing the updated error information and writing the error information.

7. The storage system according to claim 6, wherein

the processor unit

performs matching determination on whether or not a matching data model to be a data model in which a matching degree with the data model of the data set group to be written is equal to or more than a first threshold value has been stored in the storage unit,

in the case where a result of the matching determination is true,

the processor unit associates, in the write processing, instead of (W2), the matching data model with the key of the data set group to be written, or

deletes any one of the data model of the data set group to be written and the matching data model from the storage unit, and

the first threshold value is a variable value and is smaller when the allowable error is larger.

8. The storage system according to claim 1, wherein the processor unit

performs matching determination on whether or not a matching data model to be a data model in which a matching degree with the data model of the data set group to be written is equal to or more than a first threshold value has been stored in the storage unit, and

in the case where a result of the matching determination is true,

associates the matching data model with a key of the data set group to be written, or

deletes any one of the data model of the data set group to be written and the matching data model from the storage unit, instead of (W2), in the write processing.

9. The storage system according to claim 8, wherein

in the case where the result of the matching determination is false,

if there is a similar data model to be a data model in which a matching degree with the data model of the data set group to be written is less than the first threshold value and equal to or more than a second threshold value, the processor unit generates a data model including a model ID of the similar data model and difference data between the similar data model and the data model of the data set group to be written, as the data model of the data set group to be written.

10. The storage system according to claim 1, wherein

in search processing, the processor unit

searches a target data model to be a data model conforming to a search condition and returns a key associated with the found target data model.

11. The storage system according to claim 1, wherein

each data model is based on values obtained by normalizing a maximum value and a minimum value in a data set group corresponding to the data model.

12. The storage system according to claim 1, wherein

when an I/O load for an address of a write destination of the data set group to be written is less than a predetermined value or a load of the processor unit is less than the predetermined value, the processor unit performs (W1) for the data set group to be written.

13. The storage system according to claim 1, further comprising:

an edge node unit to be one or more edge nodes connected to a core node unit to be one or more core nodes,

wherein the edge node unit has the processor unit and the storage unit, and

the processor unit

performs transmission determination to be determination on whether or not the same data model as the data model of the data set group to be written has been transmitted to the core node unit,

transmits the key of the data set group and the data model to the core node unit when a result of the transmission determination is false, and

transmits the key of the data set group and a model ID of the data model to the core node unit when the result of the transmission determination is true.

14. A storage control method comprising:

in write processing of a data set group to be written to be one or more data sets to be written,

performing encoding processing including processing for generating a data model showing regularity of the data set group to be written and having one or more input values as an input and the data set group as an output (W1); and

writing the data model generated in the encoding processing and associated with a key of the data set group to be written (W2).

15. A computer program for causing a computer to execute:

in write processing of a data set group to be written to be one or more data sets to be written,

performing encoding processing including processing for generating a data model showing regularity of the data set group to be written and having one or more input values as an input and the data set group as an output (W1); and

writing the data model generated in the encoding processing and associated with a key of the data set group to be written (W2).