Data type and topological data categorization and ordering for a mass storage system

• US 6,678,788 B1
• Filed: 05/26/2000
• Issued: 01/13/2004
• Est. Priority Date: 05/26/2000
• Status: Expired due to Term

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: OʼConnell, Mark Andrew
• Primary Examiner(s): Sparks, Donald
• Assistant Examiner(s): NAMAZI, MEHDI
• Application Number:
  US09/579,671
• Time in Patent Office: 1,327 Days
• Field of Search: 707/102, 707/200-206, 711/114
• US Class Current: 711/114
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/2089 Redundant storage control functionality
• CPC Class Codes: G06F 12/1027 using associative or pseudo-associative address translation means, e.g. translation look-aside buffer [TLB]
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: Y10S 707/99952 Coherency, e.g. same view to multiple users
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

The storage space of a RAID striped mass storage system is topologically organized as a plurality of basic units of storage space for storing data items in data blocks. A topological data formatter includes a write data buffer for each data type and an initial data classifier initially categorizes each data item as a structured data type having defined data characteristics or a general data type having variable data characteristics and writes each structured data type data item into a corresponding type buffer. A topological data classifier topologically categorizes each general data type data item as a full-basic unit data type forming data blocks or as a partial-basic unit data type forming partial data blocks and writes each data item into corresponding full-basic type buffer or partial-basic type buffer. Upon a write to storage space, the initial data classifier orders structured data type data items into data block groups corresponding to a basic storage units and writes the data block groups into corresponding areas of storage space. The topological data classifier re-executes the topological classification of full-basic and partial-basic unit data type data items, orders the data items into full-basic or partial-basic unit data block groups, and writes the data block groups into corresponding areas of storage space.

Claims

1. In a mass storage system including a mass storage space for storing data items of a plurality of data types, each data item containing data of a corresponding data type and each data type being defined by the characteristics of the information represented by the data, and wherein the storage space is topologically organized as a plurality of basic units of storage space wherein each basic unit of storage space contains storage space for a predetermined number of data blocks of predetermined sizes, a method for storing the data in the storage space, comprising the steps of:

categorizing each data item to be written into the storage space as a member of a data type, including

performing an initial categorization of each data item to identify whether a data item is a member of

one of a plurality of one or more structured data types, having defined data characteristics or

a least one general data type having variable data characteristics, and

writing each data item that is a member of the one or more structured data types into a corresponding type buffer, and

performing a topological categorization of each data item that is a member of at least one general data type and identifying whether each data item that is a member of at least one general data type is

a full-basic unit data type wherein the data of the data items form one or more data block groups wherein each data block group conforms to the basic unit of storage space, or

a partial-basic unit data type wherein the data of the data items form one or more data block groups wherein each data block group differs from the basic unit of storage space, and

writing each data item that is a full-basic unit data type into a full-basic type buffer, and

writing each data item that is a partial-basic unit data type into a partial-basic type buffer,

upon performing a write of the data items to the mass storage,

reading each data item from the corresponding type buffer, and

for each data item of a structured data type, ordering the data of the data items into one or more data block groups wherein each data block group corresponds to a basic unit of storage, and

writing the one or more data block groups of each structured data type into a corresponding data type area of the storage space,

for each data item of the full-basic unit type and of the partial-basic unit type,

re-executing the topological classification of each data item and re-categorizing each data item as a full-basic unit data type or as a partial-basic unit data type, and

ordering the data of each data item of a full-basic unit data type and the data of each data item of a partial-basic unit data type into one or more corresponding full-basic unit data block groups or one or more corresponding partial-basic unit data block groups wherein each data block group corresponds to a basic unit of storage, and

writing the full-basic unit data block groups and the partial-basic unit data block groups into corresponding data type areas of the storage space.

2. The method of claim 1 for storing data in a storage space, further comprising the step of:

upon writing each data item into a corresponding type buffer, and when the data of a data item being written into a type buffer and the data of one or more data items residing in the type buffer form a full-basic unit of storage space, combining the data items forming a full-basic unit of storage space.

3. The method of claim 1 for storing data in a storage space, further comprising the step of:

returning all data items forming partial-basic unit data block groups to the corresponding type buffers for re-ordering into full-basic unit data block groups in a subsequent write to the storage space.

4. The method of claim 1 for storing data in a storage space, wherein:

each basic unit of storage space is a stripe of a striped mass storage system and wherein each stripe contains storage space for a predetermined number of data blocks.

5. The method of claim 4 for storing data in a storage space, wherein:

the mass storage system is a RAID technology storage system and wherein each stripe further includes at least one data block for storing data recovery information.

6. The method of claim 1 for storing data in a storage space, wherein:

the mass storage space is structured into a plurality of data partitions wherein each data partition corresponds to a data type and is used to store data of the corresponding data type.

7. In a mass storage system including a mass storage space for storing data items of a plurality of data types, each data item containing data of a corresponding data type and each data type being defined by the characteristics of the information represented by the data, and wherein the storage space is topologically organized as a plurality of basic units of storage space wherein each basic unit of storage space contains storage space for a predetermined number of data blocks of predetermined sizes, a topological data formatter for storing the data in the storage space, comprising:

a write data buffer corresponding to each data type;

an initial data classifier for

data type categorizing of each data item to be written into the storage space as a member of a data type, including

performing an initial categorization of each data item to identify whether a data item is a member of a structured data type having defined data characteristics or a general data type having variable data characteristics, and

writing each data item that is a member of a structured data types into a corresponding type buffer, and

a topological data classifier for

topological categorizing of each data item that is a member of a general data type and identifying whether each data item that is

a member of a general data type is a full-basic unit data type wherein the data of the data items form one or more data block groups conforming to the basic unit of storage space or

a partial-basic unit data type wherein the data of the data items form one or more data block groups differing from the basic unit of storage space,

writing each data item that is a full-basic unit data type into a full-basic type buffer, and

writing each data item that is a partial-basic unit data type into a partial-basic type buffer, wherein

upon performing a write of the data items to the mass storage,

the initial data classifier reads each data item from the corresponding type buffer and,

for each data item of a structured data type, orders the data of the data items into one or more data block groups wherein each data block group corresponds to a basic unit of storage and writes the one or more data block groups of each structured data type into a corresponding data type area of the storage space, and

for each data item of the full-basic unit type and of the partial-basic unit type, the topological data classifier re-executes the topological classification of each data item as a full-basic unit data type or as a partial-basic unit data type,

orders the data of each data item of a full-basic unit data type and the data of each data item of a partial-basic unit data type into one or more corresponding full-basic unit data block groups or one or more corresponding partial-basic unit data block groups, and

writes the full-basic unit data block groups and the partial-basic unit data block groups into corresponding data type areas of the storage space.

8. The topological data formatter of claim 7 wherein:

upon writing each data item into a corresponding type buffer, the topological data formatter combines the data of a data item being written into a type buffer and the data of one or more data items residing in the type buffer to form a full-basic unit of storage space.

9. The topological data formatter of claim 7, wherein:

upon performing a write of the data items to the mass storage, the topological data formatter returns all data items forming partial-basic unit data block groups to the corresponding type buffers for re-ordering into full-basic unit data block groups in a subsequent write to the storage space.

10. The topological data formatter of claim 7, wherein:

each basic unit of storage space is a stripe of a striped mass storage system and each stripe contains storage space for a predetermined number of data blocks.

11. The topological data formatter of claim 10, wherein:

the mass storage system is a RAID technology storage system and each stripe further includes at least one data block for storing data recovery information.

12. The topological data formatter of claim 7, wherein:

the mass storage space is structured into a plurality of data partitions wherein each data partition corresponds to a data type and is used to store data of the corresponding data type.

13. A topological data formatter for writing data blocks of data items of a plurality of data types into a storage space of a mass storage system topologically organized as basic units of storage space, comprising:

a write data buffer for each data type,

an initial data classifier for

initially categorizing each data item as a structured data type having defined data characteristics or a general data type having variable data characteristics and

writing each structured data type data item into a corresponding type buffer, and

ordering the structured data type data items into data block groups corresponding to a basic storage units and writing the data block groups into corresponding areas of storage space, and

a topological data classifier for

topologically categorizing each general data type data item as a full-basic unit data type forming data blocks or as a partial-basic unit data type and

writing each general data type data item into a corresponding full-basic type buffer or partial-basic type buffer, and

re-executing the topological classification of buffered full-basic and partial-basic unit data type data items,

ordering the data items into full-basic or partial-basic unit data block groups, and

writing the data block groups into corresponding areas of storage space.

14. A method for topologically formatting and writing data blocks of data items of a plurality of data types into a storage space of a mass storage system topologically organized as basic units of storage space, comprising the steps of:

categorizing each data item as a structured data type having defined data characteristics or a general data type having variable data characteristics and

writing each structured data type data item into a corresponding type buffer,

topologically categorizing each general data type data item as a full-basic unit data type forming data blocks or as a partial-basic unit data type and

writing each general data type data item into a corresponding full-basic type or partial-basic type buffer,

re-executing the topological classification of buffered full-basic and partial-basic unit data type data items,

ordering the buffered data items into data block groups, and

writing the data block groups into areas of storage space corresponding to the data types of the data items.