System and method for disk mapping and data retrieval

• US 5,909,692 A
• Filed: 05/06/1997
• Issued: 06/01/1999
• Est. Priority Date: 09/24/1990
• Status: Expired due to Term

• Current Assignee: EMC Corporation (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Vishlitzky, Natan, Alterescu, Bruno, Yanai, Moshe, Castel, Daniel
• Primary Examiner(s): BRAGDON, REGINALD GLENWOOD
• Application Number:
  US08/851,701
• Time in Patent Office: 756 Days
• Field of Search: 711/112, 711/113, 711/4, 711/202
• US Class Current: 711/4
• CPC Class Codes: G06F 11/2066 Optimisation of the communication load
• CPC Class Codes: G06F 11/2071 using a plurality of controllers
• CPC Class Codes: G06F 11/2074 Asynchronous techniques
• CPC Class Codes: G06F 11/2079 Bidirectional techniques
• CPC Class Codes: G06F 11/2082 Data synchronisation
• CPC Class Codes: G06F 2003/0697 device management, e.g. handlers, drivers, I/O schedulers
• CPC Class Codes: G06F 3/0601 Interfaces specially adapted for storage systems
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0661 Format or protocol conversion arrangements
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: G11B 20/1217 on discs
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 69/26 Special purpose or proprietary protocols or architectures network applications for proprietary or special purpose networking environments H04L67/12
• CPC Class Codes: H04L 69/40 for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection management of faults, events, alarms or notifications in data switching networks H04L41/06
• CPC Class Codes: H04L 9/40 Network security protocols
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation

Abstract

An apparatus and method for disk mapping and data retrieval includes a data storage medium on which has been stored a plurality of data records. Each record includes at least a record identification portion, for uniquely identifying each record from among the plurality of data records. The apparatus builds a record locator table in high speed semiconductor memory which comprises the unique record identifiers for the records on the storage medium as well as a record locator index generated by the apparatus, which indicates the address of the data record on the storage medium. Data retrieval is facilitated by first searching the record locator table in high speed semiconductor memory for a requested data record. Utilizing the record locator index associated with the reqested data record, the system directly accesses the requested data record on the storage medium thereby minimizing storage medium search time. Also disclosed is an apparatus and method for converting CKD formatted data records to FBA formatted disk drives and for building and compressing the "count" portion of the CKD data formatted record into a record locator table.

Claims

1. Apparatus for retrieving a requested variable-length data record stored on at least one fixed block architecture (FBA) formatted storage medium comprising:

means for receiving a plurality of variable-length data records, each of said variable-length data records including at least a record identification portion and a data portion;

means, responsive to said means for receiving, for transforming and storing said plurality of variable-length data records on said at least one FBA formatted storage medium;

means, responsive to said means for transforming and storing, for generating a corresponding plurality of record locator indices, each of said corresponding plurality of record locator indices being associated with one of said plurality of variable-length data records and a corresponding record identification portion, for uniquely identifying the location of each of said plurality of variable-length data records stored on said at least one FBA formatted storage medium;

memory storage means, for storing at least said plurality of record locator indices and the associated record identification portions in a record locator table;

means for requesting access to said requested variable-length data record, and for providing the record identification portion corresponding to said requested variable-length data record;

means, responsive to said means for requesting access, for searching said record locator table stored in said memory storage means, for locating an associated one of said record locator indices corresponding to said requested variable-length data record; and

record retrieval means, responsive to said means for searching, for retrieving from said at least one FBA formatted storage medium said requested variable-length data record as directed by said associated one of said record locator indices corresponding to said requested variable-length data record.

2. The apparatus as claimed in claim 1, wherein said means for receiving includes means for receiving said plurality of variable-length data records in count-key-data (CKD) format.

3. The apparatus as claimed in claim 1, wherein said at least one FBA formatted storage medium has a plurality of fixed-length blocks of storage locations, and said means for transforming and storing said plurality of variable-length data records on said at least one FBA formatted storage medium includes means for aligning a beginning of each of said data portions of each of said variable-length data records on a beginning of a respective one of said fixed-length blocks of storage locations.

4. The apparatus as claimed in claim 3, wherein said associated one of said record locator indices corresponding to said requested variable-length data record specifies the respective one of said fixed-length blocks of storage locations storing a beginning portion of said requested variable-length data record.

5. The apparatus as claimed in claim 4, wherein said record retrieval means includes means responsive to information in said record locator table for computing a number of the fixed-length blocks of storage locations that are read to retrieve all of the data portion for said requested variable-length data record.

6. The apparatus as claimed in claim 1, wherein the record identification portion contains a record number portion that usually identifies a record number from among a number of sequentially numbered data records, and wherein the apparatus includes means for determining whether the record number portion of the record identification portion identifies a record number from among a number of sequentially numbered data records.

7. The apparatus as claimed in claim 6, including means for incrementing a record number for a next expected data record by one.

8. The apparatus as claimed in claim 1, wherein said record locator table includes a track-level record locator table, and wherein said means for searching said record locator table includes means for computing an address for said requested variable-length data record from length information stored in said track-level record locator table for intervening variable-length data records in a track including said requested variable-length data record, said intervening variable-length data records occurring in the track before said requested variable-length data record.

9. The apparatus as claimed in claim 1, wherein said memory storage means includes semiconductor memory, said at least one FBA formatted storage medium includes a plurality of FBA formatted disk drives, and said record retrieval means and said means for transforming and storing retrieves and stores at least one variable-length data record in a cache in said semiconductor memory;

one of said record locator indices is associated with said at least one variable-length data record and includes means for indicating that the variable-length data record is in cache, and includes a pointer to a memory address of said at least one variable-length data record in cache, and includes means for indicating that the variable-length data record in cache is awaiting writing to said at least one FBA formatted storage medium.

10. The apparatus as claimed in claim 1, wherein said record retrieval means retrieves the requested variable-length data record from said at least one FBA formatted storage medium with no intervening search of said at least one FBA formatted storage medium.

11. The apparatus as claimed in claim 1, wherein said at least one FBA formatted storage medium includes an FBA formatted disk drive.

12. The apparatus as claimed in claim 1, wherein said at least one FBA formatted storage medium includes semiconductor memory.

13. A method for transforming and storing variable-length data records onto at least one fixed block architecture (FBA) formatted storage medium, and for retrieving a requested data record stored on said at least one FBA formatted storage medium, comprising the steps of:

receiving a plurality of variable-length data records, each of said variable-length data records including at least a record identification portion and a data portion;

storing to said at least one FBA formatted storage medium, said plurality of variable-length data records, said step of storing including;

transforming said plurality of variable-length data records to an FBA format, and storing the plurality of data records in the FBA format on said at least one FBA formatted storage medium;

generating a plurality of record locator indices;

each of the plurality of record locator indices corresponding to one of the plurality of data records, for uniquely identifying the location of each of the plurality of data records stored on said at least one FBA formatted storage medium;

forming a record locator table including the plurality of the record locator indices and record identification portions; and

storing said record locator table in memory;

searching said record locator table stored in memory, for locating one of the record locator indices corresponding to said requested data record; and

retrieving from said at least one FBA formatted storage medium, said requested data record as directed by said one of the record locator indices corresponding to said requested data record.

14. The method as claimed in claim 13, wherein said record locator table includes a track-level record locator table, and wherein said searching said record locator table includes computing an address for said requested data record from length information stored in said track-level record locator table for intervening data records in a track including said requested data record, said intervening data records occurring in the track before said requested data record.

15. The method as claimed in claim 13, wherein said at least one FBA formatted storage medium has fixed-length blocks of storage locations, and said transforming said plurality of variable-length data records to an FBA format includes aligning a beginning of each of said data portions of each of said variable-length data records on a beginning of a respective one of said fixed-length blocks of storage locations.

16. The method as claimed in claim 13, wherein said requested data record is retrieved from said at least one FBA formatted storage medium with no intervening search of said at least one FBA formatted storage medium.

17. A method for transforming and storing variable-length CKD formatted data records onto one or more fixed block architecture (FBA) disk drives, and for retrieving a requested data record stored on one or more of said fixed block architecture disk drives, comprising the steps of:

receiving a plurality of variable-length CKD formatted data records, each of said variable-length CKD formatted data records including at least a record identification portion and a data portion;

storing to said one or more fixed block architecture (FBA) disk drives, said plurality of variable-length CKD formatted data records, said step of storing including;

transforming said plurality of variable-length CKD formatted data records from a CKD format to an FBA format, and storing the plurality of data records in the FBA format on said one or more FBA disk drives;

generating a plurality of record locator indices;

each of the plurality of record locator indices corresponding to one of the plurality of data records, for uniquely identifying the location of each of the plurality of data records stored on said one or more FBA disk drives;

forming a record locator table including the plurality of the record locator indices and record identification portions; and

storing said record locator table in memory;

searching said record locator table stored in memory, for locating one of the record locator indices corresponding to said requested data record; and

retrieving from said one or more FBA disk drives, said requested data record as directed by said one of the record locator indices corresponding to said requested data record.

18. The method as claimed in claim 17, wherein said record locator table includes a track-level record locator table, and wherein said searching said record locator table includes computing an address for said requested data record from length information stored in said track-level record locator table for intervening data records in a track including said requested data record, said intervening data records occurring in the track before said requested data record.

19. The method as claimed in claim 17, wherein said FBA disk drives have fixed-length blocks of storage locations, and said transforming said plurality of variable-length data records from a CKD format to an FBA format includes aligning a beginning of each of said data portions of each of said variable-length data records on a beginning of a respective one of said fixed-length blocks of storage locations.

20. The method as claimed in claim 17, wherein said requested data record is retrieved from said one or more FBA disk drives with no intervening search of said one or more FBA disk drives.

21. Apparatus for retrieving a requested variable-length data record stored on at least one fixed block architecture (FBA) formatted storage medium comprising:

means for receiving a plurality of variable-length data records;

means, responsive to said means for receiving, for transforming and storing said plurality of variable-length data records on said at least one FBA formatted storage medium;

means, responsive to said means for transforming and storing, for generating a corresponding plurality of record locator indices, each of said corresponding plurality of record locator indices being associated with one of said plurality of variable-length data records, for uniquely identifying the location of each of said plurality of variable-length data records stored on said at least one FBA formatted storage medium;

mapping means, for storing at least said plurality of record locator indices in a record locator table, and in response to a request for access to said-requested variable-length data record, for searching said record locator table for obtaining an associated one of said record locator indices corresponding to said requested variable-length data record; and

record retrieval means for retrieving from said at least one FBA formatted storage medium said requested variable-length data record as directed by said associated one of said record locator indices corresponding to said requested variable-length data record.

22. The apparatus as claimed in claim 21, wherein said at least one FDA formatted storage medium has a plurality of fixed-length blocks of storage locations, and said means for transforming and storing said plurality of variable-length data records on said at least one FBA formatted storage medium includes means for aligning a beginning of a data portion of each of said variable-length data records on a beginning of a respective one of said fixed-length blocks of storage locations.

23. The apparatus as claimed in claims 22, wherein said associated one of said record locator indices corresponding to said requested variable-length data record specifies the respective one of said fixed-length blocks of storage locations storing a beginning portion of said requested variable-length data record.

24. The apparatus as claimed in claim 22, wherein said record retrieval means includes means responsive to information in said record locator table for computing a number of the fixed-length blocks of storage locations that are read to retrieve all data for said requested variable-length data record.

25. The apparatus as claimed in claim 21, wherein said record locator table includes a track-level record locator table, and wherein said mapping means includes means for computing an address for said requested variable-length data record from length information stored in said track-level record locator table for intervening variable-length data records in a track including said requested variable-length data record, said intervening variable-length data records occurring in the track before said requested variable-length data record.

26. A method for transforming and storing variable-length data records onto a fixed block architecture (FBA) formatted storage medium, and for retrieving a requested data record stored on said at least one FBA formatted storage medium, comprising the steps of:

receiving a plurality of variable-length data records;

storing to said at least one FBA formatted storage medium, said plurality of variable-length data records, said step of storing including;

transforming said plurality of variable-length data records to an FBA format, and storing the plurality of data records in the FBA format on said at least one FBA formatted storage medium;

generating a plurality of record locator indices;

each of the plurality of record locator indices corresponding to one of the plurality of data records, for uniquely identifying the location of each of the plurality of data records stored on said at least one FBA formatted storage medium; and

forming a record locator table including the plurality of the record locator indices;

searching said record locator table, for locating one of the record locator indices corresponding to said requested data record; and

retrieving from said at least one FBA formatted storage medium, said requested data record as directed by said one of the record locator indices corresponding to said requested data record.

27. The method as claimed in claim 26, wherein said record locator table includes a track-level record locator table, and wherein said searching said record locator table includes computing an address for said requested data record from length information stored in said track-level record locator table for intervening data records in a track including said requested data record, said intervening data records occurring in the track before said requested data record.

28. The method as claimed in claim 26, wherein said at least one FBA formatted storage medium has fixed-length blocks of storage locations, and said transforming said plurality of variable-length data records to an FBA format includes aligning a beginning of a data portion of each of said variable-length data records on a beginning of a respective one of said fixed-length blocks of storage locations.

29. Apparatus for transforming, storing, and mapping variable-length data records on at least one fixed block architecture (FBA) formatted storage medium comprising:

means for receiving a plurality of variable-length data records, each of said variable-length data records including at least a record identification portion and a data portion;

means, responsive to said means for receiving, for transforming and storing said plurality of variable-length data records on said at least one FBA formatted storage medium;

means, responsive to said means for transforming and storing, for generating a corresponding plurality of record locator indices, each of said corresponding plurality of record locator indices being associated with one of said plurality of variable-length data records and a corresponding record identification portion, for uniquely identifying the location of each of said plurality of variable-length data records stored on said at least one FBA formatted storage medium; and

memory storage means, for storing at least said plurality of record locator indices and the associated record identification portions in a record locator table.

30. The apparatus as claimed in claim 29, wherein said at least one FBA formatted storage medium has a plurality of fixed-length blocks of storage locations, and said means for transforming and storing said plurality of variable-length data records on said at least one FBA formatted storage medium includes means for aligning a beginning of each of said data portions of each of said variable-length data records on a beginning of a respective one of said fixed-length blocks of storage locations.

31. The apparatus as claimed in claim 29, wherein the record identification portion contains a record number portion that usually identifies a record number from among a number of sequentially numbered data records, and wherein the apparatus includes means for determining whether the record number portion of the record identification portion identifies a record number from among a number of sequentially numbered data records.

32. The apparatus as claimed in claim 31, including means for incrementing a record number for a next expected data record by one.

33. Apparatus for retrieving data stored on at least one fixed block architecture (FBA) formatted storage medium, said at least one FBA formatted storage medium including a plurality of fixed-length blocks of storage locations, said apparatus comprising:

means for receiving a plurality of variable-length data records, each of said variable-length data records including at least a record identification portion and a data portion;

means, responsive to said means for receiving, for transferring and storing the data portions of said plurality of variable-length data records on said at least one FBA formatted storage medium by aligning a beginning of each of the data portions on a respective one of said fixed-length blocks of storage locations;

means, responsive to said means for transferring and storing, for generating a corresponding plurality of record locator indices, each of said corresponding plurality of record locator indices being associated with one of said plurality of variable-length data records and a corresponding record identification portion, for uniquely identifying the respective one of the fixed-length blocks of storage locations on which is aligned the beginning of the data portion of said one of said plurality of variable-length data records;

memory storage means, for storing at least said plurality of record locator indices and the associated record identification portions in a record locator table;

means for requesting access to a requested variable-length data record, and for providing the record identification portion corresponding to said requested variable-length data record;

means, responsive to said means for requesting access, for searching said record locator table stored in said memory storage means, for locating an associated one of said record locator indices corresponding to said requested variable-length data record; and

record data retrieval means, responsive to said means for searching, for retrieving from said FBA formatted storage medium the data portion of said requested variable-length data record as directed by said associated one of said record locator indices corresponding to said requested variable-length data record.

34. The apparatus as claimed in claim 33, wherein said at least one FBA formatted storage medium includes rotating disk storage, and said memory storage means includes random access semiconductor memory.

35. The apparatus as claimed in claim 33, wherein said means for receiving includes means for receiving said plurality of variable-length data records in count-key-data (CKD) format.

36. The apparatus as claimed in claim 33, wherein said record data retrieval means includes means responsive to information in said record locator table for computing a number of the fixed-length blocks of storage locations that are read to retrieve all of the data portion for said requested variable-length data record.

37. The apparatus as claimed in claim 33, wherein the record identification portion contains a record number portion that usually identifies a record number from among a number of sequentially numbered data records, and wherein the apparatus includes means for determining whether the record number portion of the record identification portion identifies a record number from among a number of sequentially numbered data records.

38. The apparatus as claimed in claim 37, including means for incrementing a record number for a next expected data record by one.

39. The apparatus as claimed in claim 33, wherein said record locator table includes a track-level record locator table, and wherein said means for searching said record locator table includes means for computing an address for said requested variable-length data record using length information stored in said track-level record locator table for intervening variable-length data records in a track including said requested variable-length data record, said intervening variable-length data records occurring in the track before said requested variable-length data record.

40. The apparatus as claimed in claim 33, which further includes means for storing the record locator indices in a compressed format in the record locator table.

41. The apparatus as claimed in claim 33, which further includes means for storing the record identification portions in a compressed format in the memory storage means.

42. Apparatus for retrieving data stored on fixed block architecture (FBA) formatted storage medium, the FBA formatted storage medium including a plurality of fixed-length blocks of storage locations, said apparatus comprising:

a data record receiver for receiving a plurality of variable-length data records, each of said variable-length data records including at least a record identification portion and a data portion;

a data record storer, responsive to the data record receiver, for transferring and storing the data portions of said plurality of variable-length data records on said FBA formatted storage medium by aligning a beginning of each of the data portions on a respective one of said fixed-length blocks of storage locations;

a record locator index generator, responsive to said data record storer, for generating a corresponding plurality of record locator indices, each of said corresponding plurality of record locator indices being associated with one of said plurality of variable-length data records and a corresponding record identification portion, for uniquely identifying the respective one of the fixed-length blocks of storage locations on which is aligned the beginning of the data portion of said one of said plurality of variable-length data records;

memory storage, for storing at least said plurality of record locator indices and the associated record identification portions in a record locator table;

a data record access requester, for requesting access to a requested variable-length data record, and for providing the record identification portion corresponding to said requested variable-length data record;

a record locator table searcher, responsive to said data record access requester, for searching said record locator table stored in said memory storage, for locating an associated one of said record locator indices corresponding to said requested variable-length data record; and

a record data retriever, responsive to said record locator table searcher, for retrieving from said FBA formatted storage medium the data portion of said requested variable-length data record as directed by said associated one of said record locator indices corresponding to said requested variable-length data record.

43. The apparatus as claimed in claim 42, wherein the FBA formatted storage medium is rotating disk storage, and said memory storage is random access semiconductor memory.

44. The apparatus as claimed in claim 42, wherein said data record receiver is adapted for receiving said plurality of variable-length data records in count-key-data (CKD) format.

45. The apparatus as claimed in claim 42, wherein said record data retriever is responsive to information in said record locator table for computing a number of the fixed-length blocks of storage locations that are read to retrieve all of the data portion for said requested variable-length data record.

46. The apparatus as claimed in claim 42, wherein the record identification portion contains a record number portion that usually identifies a record number from among a number of sequentially numbered data records, and wherein the apparatus includes a record number checker for determining whether the record number portion of the record identification portion identifies a record number from among a number of sequentially numbered data records.

47. The apparatus as claimed in claim 46, which includes a record number counter for incrementing a record number for a next expected data record by one.

48. The apparatus as claimed in claim 42, wherein said record locator table includes a track-level record locator table, and wherein said record locator table searcher includes an address computer for computing an address for said requested variable-length data record using length information stored in said track-level record locator table for intervening variable-length data records in a track including said requested variable-length data record, said intervening variable-length data records occurring in the track before said requested variable-length data record.

49. The apparatus as claimed in claim 42, further including a data compressor for compressing and storing the record locator indices in a compressed format in the record locator table.

50. The apparatus as claimed in claim 42, further including a data compressor for compressing and storing the record identification portions in a compressed format in the memory storage.

51. A method of retrieving data stored on fixed block architecture (FBA) formatted storage medium, the FBA formatted storage medium including a plurality of fixed-length blocks of storage locations, said method including the steps of:

receiving a plurality of variable-length data records, each of said variable-length data records including at least a record identification portion and a data portion;

transferring and storing the data portions of said plurality of variable-length data records on said FBA formatted storage medium by aligning a beginning of each of the data portions on a respective one of said fixed-length blocks of storage locations;

generating a corresponding plurality of record locator indices, each of said corresponding plurality of record locator indices being associated with one of said plurality of variable-length data records and a corresponding record identification portion, for uniquely identifying the respective one of the fixed-length blocks of storage locations on-which is aligned the beginning of the data portion of said one of said plurality of variable-length data records;

storing at least said plurality of record locator indices and the associated record identification portions in a record locator table;

requesting access to a requested variable-length data record, and for providing the record identification portion corresponding to said requested variable-length data record;

searching said record locator table, for locating an associated one of said record locator indices corresponding to said requested variable-length data record; and

retrieving from said FBA formatted storage medium the data portion of said requested variable-length data record as directed by said associated one of said record locator indices corresponding to said requested variable-length data record.

52. The method as claimed in claim 51, wherein the FBA formatted storage medium is rotating disk storage, and said record locator table is stored in random access semiconductor memory.

53. The method as claimed in claim 51, which includes receiving said plurality of variable-length data records in count-key-data (CKD) format.

54. The method as claimed in claim 51, which includes using information in said record locator table for computing a number of the fixed-length blocks of storage locations that are read to retrieve all of the data portion for said requested variable-length data record.

55. The method as claimed in claim 51, wherein the record identification portion contains a record number portion that usually identifies a record number from among a number of sequentially numbered data records, and wherein the method includes determining whether the record number portion of the record identification portion identifies a record number from among a number of sequentially numbered data records.

56. The method as claimed in claim 55, which includes incrementing a record number for a next expected data record by one.

57. The method as claimed in claim 51, wherein said record locator table includes a track-level record locator table, and wherein the method includes searching said record locator table by computing an address for said requested variable-length data record using length information stored in said track-level record locator table for intervening variable-length data records in a track including said requested variable-length data record, said intervening variable-length data records occurring in the track before said requested variable-length data record.

58. The method as claimed in claim 51, which includes storing the record locator indices in a compressed format in the record locator table.

59. The method as claimed in claim 51, which includes storing the record identification portions in a compressed format.

60. Apparatus for retrieving data stored on at least one fixed block architecture (FBA) formatted storage medium comprising:

means for receiving a plurality of variable-length data records, each of said variable-length data records including at least a record identification portion and a data portion;

means, responsive to said means for receiving, for transforming and storing at least the data portions of said plurality of variable-length data records on said at least one FBA formatted storage medium;

means, responsive to said means for transforming and storing, for generating a corresponding plurality of record locator indices, each of said corresponding plurality of record locator indices being associated with one of said plurality of variable-length data records and a corresponding record identification portion, for uniquely identifying the location of each of the data portions of said plurality of variable-length data records stored on said at least one FBA formatted storage medium;

memory storage means, for storing at least said plurality of record locator indices in a record locator data structure, the record locator indices being stored at locations different from the locations of the data portions of their associated variable-length data records;

means for requesting access to a requested variable-length data record, and for providing the record identification portion corresponding to said requested variable-length data record;

means, responsive to said means for requesting access, for searching said record locator data structure stored in said memory storage means, for locating an associated one of said record locator indices corresponding to said requested variable-length data record; and

record retrieval means, responsive to said means for searching, for retrieving from said at least one FBA formatted storage medium at least the data portion of said requested variable-length data record as directed by said associated one of said record locator indices corresponding to said requested variable-length data record.

61. The apparatus as claimed in claim 60, wherein said means for receiving includes means for receiving said plurality of variable-length data records in count-key-data (CKD) format.

62. The apparatus as claimed in claim 60, which further includes means for compressing a count portion of each variable-length data record to produce a compressed count portion of said each variable-length data record, and wherein the memory storage means includes means for storing the compressed count portion of each variable-length data record in the record locator data structure.

63. The apparatus as claimed in claim 60, which further includes means for compressing a portion of each variable-length data record to produce a compressed portion of said each variable-length data record, and wherein the memory storage means includes means for storing the compressed portion of said each variable-length data record in the record locator data structure.

64. The apparatus as claimed in claim 63, wherein the data portion of said each variable-length data record stored in said at least one FBA formatted storage medium is not compressed, and wherein the means for transforming and storing includes means for storing the compressed portion of said each variable-length data record in said at least one FBA formatted storage medium in association with the data portion of said each variable-length data record.

65. The apparatus as claimed in claim 60, wherein said at least one FBA formatted storage medium has a plurality of fixed-length blocks of storage locations, and said means for transforming and storing said plurality of variable-length data records on said at least one FBA formatted storage medium includes means for aligning a beginning of each of said data portions of each of said variable-length data records on a beginning of a respective one of said fixed-length blocks of storage locations.

66. The apparatus as claimed in claim 60, wherein the record identification portion contains a record number portion that usually identifies a record number from among a number of sequentially numbered data records, and wherein the apparatus includes means for determining whether the record number portion of the record identification portion identifies a record number from among a number of sequentially numbered data records.

67. The apparatus as claimed in claim 60, wherein said record retrieval means retrieves at least the data portion of the requested variable-length data record from said at least one FBA formatted storage medium with no intervening search of said at least one FBA formatted storage medium.

68. A method for storing and retrieving data stored on at least one fixed block architecture (FBA) formatted storage medium, said method comprising:

receiving a plurality of variable-length data records, each of said variable-length data records including at least a record identification portion and a data portion;

storing to said at least one FBA formatted storage medium, at least the data portions of said plurality of variable-length data records, said storing including;

transforming at least said data portions of said plurality of variable-length data records to an FBA format, and storing at least said data portions of said plurality of data records in the FBA format on said at least one FBA formatted storage medium;

generating a plurality of record locator indices;

each of the plurality of record locator indices corresponding to one of the plurality of data records, for uniquely identifying the location of each of the data portions of said plurality of data records stored on said at least one FBA formatted storage medium;

forming a record locator data structure including the plurality of the record locator indices, the record locator data structure being stored in memory at a location different from the location of the data portions of the plurality of data records stored on said at least one FBA formatted storage medium;

searching said record locator data structure stored in memory, for locating one of the record locator indices corresponding to a requested data record; and

retrieving from said at least one FBA formatted storage medium, at least the data portion of said requested data record as directed by said one of the record locator indices corresponding to said requested data record.

69. The method as claimed in claim 68, which includes receiving said plurality of variable-length data records in count-key-data (CKD) format.

70. The method as claimed in claim 69, which further includes compressing a count portion of each variable-length data record to produce a compressed count portion of said each variable-length data record, and storing the compressed count portion of said each variable-length data record in the record locator data structure.

71. The method as claimed in claim 68, which further includes compressing a portion of each variable-length data record to produce a compressed portion of said each variable-length data record, and storing the compressed portion of said each variable-length data record in the record locator data structure.

72. The method as claimed in claim 71, wherein the data portion of said each variable-length data record stored in said at least one FBA formatted storage medium is not compressed, and wherein the method includes storing the compressed portion of said each variable-length data record in said at least one FBA formatted storage medium in association with the data portion of said each variable-length data record.

73. The method as claimed in claim 68, wherein said at least one FBA formatted storage medium has a plurality of fixed-length blocks of storage locations, and wherein said transforming and storing said plurality of variable-length data records on said at-least one FBA formatted storage medium includes aligning a beginning of each of said data portions of each of said variable-length data records on a beginning of a respective one of said fixed-length blocks of storage locations.

74. The method as claimed in claim 68, wherein the record identification portion contains a record number portion that usually identifies a record number from among a number of sequentially numbered data records, and wherein the method includes determining whether the record number portion of the record identification portion identifies a record number from among a number of sequentially numbered data records.

75. The method as claimed in claim 68, wherein the data portion of the requested variable-length data record is retrieved from said at least one FBA formatted storage medium with no intervening search of said at least one FBA formatted storage medium.

76. Apparatus for retrieving data stored on at least one fixed block architecture (FBA) formatted storage medium comprising:

a data record receiver, for receiving a plurality of variable-length data records, each of said variable-length data records including at least a record identification portion and a data portion;

a data record storer, responsive to the data record receiver, for transforming and storing at least the data portions of said plurality of variable-length data records on said at least one FBA formatted storage medium;

a record locator index generator, responsive to said data record storer, for generating a corresponding plurality of record locator indices, each of said corresponding plurality of record locator indices being associated with one of said plurality of variable-length data records and a corresponding record identification portion, for uniquely identifying the location of each of the data portions of said plurality of variable-length data records stored on said at least one FBA formatted storage medium;

memory storage, for storing at least said plurality of record locator indices in a record locator data structure, the record locator indices being stored at locations different from the locations of the data portions of their associated variable-length data records;

a record locator index searcher, responsive to a request for at least the data portion of a requested variable-length data record, for searching said record locator data structure stored in said memory storage means, for locating an associated one of said record locator indices corresponding to the requested variable-length data record; and

a data retriever, responsive to said record locator index searcher, for retrieving from said at least one FBA formatted storage medium at least the data portion of said requested variable-length data record as directed by said associated one of said record locator indices corresponding to said requested variable-length data record.

77. The apparatus as claimed in claim 76, wherein said data receiver is adapted to receive said plurality of variable-length data records in count-key-data (CKD) format.

78. The apparatus as claimed in claim 77, which includes a data compressor for compressing a count portion of each variable-length data record to produce a compressed count portion of said each variable-length data record, and wherein the compressed count-portion of said each variable-length, data record is stored in the record locator data structure.

79. The apparatus as claimed in claim 76, which includes a data compressor for compressing a portion of each variable-length data record to produce a compressed portion of said each variable-length data record, and wherein the compressed portion of said each variable-length data record is stored in the record locator data structure.

80. The apparatus as claimed in claim 79, wherein the data portion of said each variable-length data record stored in said at least one FBA formatted storage medium is not compressed, and wherein the data record storer stores the compressed portion of said each variable-length data record in said at least one FBA formatted storage medium in association with the data portion of said each variable-length data record.

81. The apparatus as claimed in claim 76, wherein said at least one FBA formatted storage medium has a plurality of fixed-length blocks of storage locations, and the data record storer transforms and stores at least the data portions of said plurality of variable-length data records on said at least one FBA formatted storage medium by aligning a beginning of each of said data portions of each of said variable-length data records on a beginning of a respective one of said fixed-length blocks of storage locations.

82. The apparatus as claimed in claim 76, wherein the record identification portion contains a record number portion that usually identifies a record number from among a number of sequentially numbered data records, and wherein the data record storer determines whether the record number portion of the record identification portion identifies a record number from among a number of sequentially numbered data records.

83. The apparatus as claimed in claim 76, wherein said record retrieval means retrieves at least the data portion of the requested variable-length data record from said at least one FBA formatted storage medium with no intervening search of said at least one FBA formatted storage medium.