System and method for disk mapping and data retrieval

• US 6,418,509 B1
• Filed: 11/09/2000
• Issued: 07/09/2002
• Est. Priority Date: 09/24/1990
• Status: Expired due to Term

• Current Assignee: EMC Corporation (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Vishlitzky, Natan, Alterescu, Bruno, Yanai, Moshe, Castel, Daniel
• Primary Examiner(s): BRAGDON, REGINALD GLENWOOD
• Application Number:
  US09/711,212
• Time in Patent Office: 607 Days
• Field of Search: 710/68, 711/112, 711/202, 711/4
• US Class Current: 711/112
• CPC Class Codes: G06F 11/2066 Optimisation of the communication load
• CPC Class Codes: G06F 11/2071 using a plurality of controllers
• CPC Class Codes: G06F 11/2074 Asynchronous techniques
• CPC Class Codes: G06F 11/2079 Bidirectional techniques
• CPC Class Codes: G06F 11/2082 Data synchronisation
• CPC Class Codes: G06F 2003/0697 device management, e.g. handlers, drivers, I/O schedulers
• CPC Class Codes: G06F 3/0601 Interfaces specially adapted for storage systems
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0661 Format or protocol conversion arrangements
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: G11B 20/1217 on discs
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 69/26 Special purpose or proprietary protocols or architectures network applications for proprietary or special purpose networking environments H04L67/12
• CPC Class Codes: H04L 69/40 for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection management of faults, events, alarms or notifications in data switching networks H04L41/06
• CPC Class Codes: H04L 9/40 Network security protocols
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation

Abstract

An apparatus and method for disk mapping and data retrieval includes a data storage medium on which has been stored a plurality of data records. Each record includes at least a record identification portion, for uniquely identifying each record from among the plurality of data records. The apparatus builds a record locator table in high speed semiconductor memory which comprises the unique record identifiers for the records on the storage medium as well as a record locator index generated by the apparatus, which indicates the address of the data record on the storage medium. Data retrieval is facilitated by first searching the record locator table in high speed semiconductor memory for a requested data record. Utilizing the record locator index associated with the requested data record, the system directly H: accesses the requested data record on the storage medium thereby minimizing storage medium search time. Also disclosed is an apparatus and method for converting CKD formatted data records to FBA formatted disk drives and for building and compressing the “count” portion of the CKD data formatted record into a record locator table.

Claims

1. A method of operating a data storage controller for storing information from variable-length data records on one or more fixed-block architecture (FBA) disk drives, said method comprising:

the data storage controller receiving a plurality of the variable-length data records, each of the variable-length data records including at least a record identification portion and a data portion;

the data storage controller storing at least data of each of the variable-length data records on said one or more FBA disk drives;

the data storage controller maintaining an index of record locator information specifying locations where data of the variable-length data records are stored on said one or more disk drives;

wherein said index is accessible for obtaining record locator information of a specified variable-length data record for access to data of the specified variable-length data record on said one or more disk drives; and

the data storage controller compressing the record identification portions of at least some of the variable-length data records to produce compressed record identification portions;

wherein the index of record locator information includes the compressed record identification portions.

2. A method of operating a data storage controller for storing information from count-key-data (CKD) variable-length data records on one or more fixed-block architecture (FBA) disk drives, said method comprising:

the data storage controller receiving a plurality of the CKD variable-length data records, each of ate CKD variable-length data records including at least a record identification portion and a data portion;

the data storage controller storing at least data of each of the CKD variable-length data records on said one or more FBA disk drives;

the data storage controller maintaining an index of record locator information specifying locations where data of the CKD variable-length data records are stored on said one or more disk drives;

wherein said index is accessible for obtaining record locator information of a specified CKD variable-length data record for access to data of the specified CKD variable-length data record on said one or more disk drives; and

the data storage controller compressing the record identification portions of at least some of the CKD variable-length data records to produce compressed record identification portions;

wherein the index of record locator information includes the compressed record identification portions.

3. A data storage controller for storing information from variable-length data records on one or more fixed-block architecture (FBA) disk drives, said data storage controller comprising:

a record receiver for receiving a plurality of the variable-length data records, each of the variable-length data records including at least a record identification portion and a data portion;

a data record storer for storing at least data of the variable-length data records on said one or more FBA disk drives;

a memory containing an index of record locator information specifying locations where data of the variable-length data records are stored on said one or more disk drives;

wherein said index is accessible for obtaining record locator information of a specified variable-length data record for access to da of the specified variable-length data record on said one or more disk drives; and

a data compressor for compressing at least the record identification portion of at least some of the variable-length data records to produce compressed record identification portions;

wherein the index of record locator information includes the compressed record identification portions.

4. A data storage controller for storing information from count-key-data (CKD) variable-length data records on one or more fixed-block architecture (FBA) disk drives, said data storage controller comprising:

a record receiver for receiving a plurality of the CKD variable-length data records, each of the CKD variable-length data records including at least a record identification portion and a data portion;

a data record storer for storing at least data of the CKD variable-length data records on said one or more FBA disk drives;

a memory containing an index of record locator information specifying locations where data of the CKD variable-length data records are stored on said one or more disk drives;

wherein said index is accessible for obtaining record locator information of a specified variable-length data record for access to data of the specified variable-length data record on said one or more disk drives; and

a data compressor for compressing at least the record identification portion of at least some of the variable-length data records to produce compressed record identification portions;

wherein the compressed record identification portions are included in said index.

5. A method of operating a data storage controller for storing information from variable-length data records on one or more fixed-block architecture (FBA) disk drives, said method comprising:

the data storage controller receiving a plurality of the variable-length data records, each of the variable-length data records including at least a record identification portion and a data portion;

the data storage controller storing at least data of each of the variable-length data records on said one or more FBA disk drives;

the data storage controller maintaining an index of record locator information specifying locations where data of the variable-length data records are stored on said one or more disk drives;

wherein said index is accessible for obtaining record locator information of a specified variable-length data record for access to data of the specified variable-length data record on said one or more disk drives; and

the data storage controller compressing the record identification portions of at least some of the variable-length data records to produce compressed record identification portions;

wherein the record identification portions of said at least some of the variable-length data records are included in said index, the storage controller compresses the index, and wherein the data storage controller compresses the record identification portions of said at least some of the variable length records to produce the compressed record identification portions when the storage controller compresses the index.

6. The method as claimed in claim 5, wherein the storage controller compresses the index when the index becomes full.

7. A method of operating a data storage controller for storing information from count-key-data (CKD) variable-length data records on one or more fixed-block architecture (FBA) disk drives, said method comprising:

the data storage controller receiving a plurality of the CKD variable-length data records, each of the CKD variable-length data records including at least a record identification portion and a data portion;

the data storage controller storing at least data of each of the CKD variable-length data records on said one or more FBA disk drives;

the data storage controller maintaining an index of record locator information specifying locations where data of the CKD variable-length data records are stored on said one or more disk drives;

wherein said index is accessible for obtaining record locator information of a specified CKD variable-length data record for access to data of the specified CKD variable-length data record on said one or more disk drives; and

the data storage controller compressing the record identification portions of at least some of the CKD variable-length data records to produce compressed record identification portions;

wherein the record identification portions of said at least some of the CKD variable-length data records are included in said index, the storage controller compresses the index, and wherein the data storage controller compresses the record identification portions of said at least some of the CKD variable-length data records to produce the compressed record identification portions when the storage controller compresses the index.

8. The method as claimed in claim 7, wherein the storage controller compresses the index when the index becomes full.

9. A data storage controller for storing information from variable-length data records on one or more fixed-block architecture (FBA) disk drives, said data storage controller comprising:

a record receiver for receiving a plurality of the variable-length data records, each of the variable-length data records including at least a record identification portion and a data portion;

a data record storer for storing at least data of the variable-length data records on said one or more FBA disk drives;

a memory containing an index of record locator information specifying locations where data of the variable-length data records are stored on said one or more disk drives;

wherein said index is accessible for obtaining record locator information of a specified variable-length data record for access to data of the specified variable-length data record on said one or more disk drives; and

a data compressor for compressing at least the record identification portion of at least some of the variable-length data records to produce compressed record identification portions;

wherein the record identification portions of said at least some of the variable-length data records are included in said index, and wherein the data compressor is operative for compressing the index.

10. A data storage controller for storing information from count-key-data (CKD) variable-length data records on one or more fixed-block architecture (FBA) disk drives, said data storage controller comprising:

a record receiver for receiving a plurality of the CKD variable-length data records, each of the CKD variable-length data records including at least a record identification portion and a data portion;

a data record storer for storing at least data of the CKD variable-length data records on said one or more FBA disk drives;

a memory containing an index of record locator information specifying locations where data of the CKD variable-length data records are stored on said one or more disk drives;

wherein said index is accessible for obtaining record locator information of a specified variable-length data record for access to data of the specified variable-length data record on said one or more disk drives; and

a data compressor for compressing at least the record identification portion of at least some of the variable-length data records to produce compressed record identification portions;

wherein the record identification portions of said at least some of the CKD variable-length data records are included in said index, and wherein the data compressor is operative for compressing the index.