System and Method for an Efficient Database Storage Model Based on Sparse Files

• US 20150234841A1
• Filed: 02/20/2014
• Published: 08/20/2015
• Est. Priority Date: 02/20/2014
• Status: Abandoned Application

• Current Assignee: Futurewei Technologies Incorporated (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Futurewei Technologies Incorporated (Huawei Investment & Holding Co., Ltd.)
• Inventors: Varakur, Gangavara Prasad, Hebert, Jacques Earl
• Application Number:
  US14/185,516
• Publication Number:
  US 20150234841A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2282 Tablespace storage structures; Management thereof

Abstract

Embodiments are provided herein for an efficient database storage model, which utilizes sparse file features to efficiently store and retrieve data. The embodiments provide database algorithms that utilize the file system abstraction layer to hide the complexity of managing disk space while providing the database a linear and contiguous logical address space for holding multiple database objects. An embodiment method includes pre-allocating, in a logical sparse file, a plurality of segments fixed in size and contiguous at fixed offsets. Upon receiving a command to write database objects to the segments, the database objects are mapped to the segments in a database catalog. The method further includes interfacing with a file system to initialize storage medium space for writing the data objects to the segments at the fixed offsets.

Claims

1. A method by a database system engine for database storage operations, the method comprising:

pre-allocating, in a logical sparse file, a plurality of segments fixed in size and contiguous at fixed offsets;

receiving a command to write database objects to the segments;

mapping the database objects to the segments in a database catalog; and

interfacing with a file system to initialize storage medium space for writing the data objects to the segments at the fixed offsets.

2. The method of claim 1, wherein the segments are pre-allocated in the logical sparse file without initializing the storage medium space for the segments.

3. The method of claim 1, wherein the database objects are mapped in the database catalog to the segments using indices indicating object IDs of the database objects and object segment indices in relation to the object IDs in the database catalog.

4. The method of claim 3 further comprising upon a command to delete the database objects or free the segments, initializing to zero the indices indicating the object IDs and the object segment indices.

5. The method of claim 1 further comprising:

calculating page locations of subsegments in the segments according to defined page offset and subsegment size; and

assigning the database objects to the subsegments at the page locations.

6. The method of claim 1, wherein the segments are larger is size than the database objects, and wherein the database objects start at the fixed offsets of the segments in the logical sparse file.

7. The method of claim 1, wherein the database system engine is an application programming interface that interacts with the file system for managing storage medium operations for the logical sparse file.

8. The method of claim 1 further comprising updating, using atomicity, consistency, isolation, and durability (ACID) transactions, database metadata maintained in the database catalog and data and metadata written into the segments in the logical sparse file.

9. The method of claim 1 further comprising upon receiving a command to write the database objects to the segments, initializing, at a file system engine, storage medium space for writing the data objects to segments starting at the fixed offsets.

10. The method of claim 1 further comprising updating, using atomicity, consistency, isolation, and durability (ACID) transactions, a mapping of the segments to disjoint physical disk extents.

11. The method of claim 1 further comprising marinating, in a journal file, metadata of the file system.

12. A method by a database system engine for database storage operations, the method comprising:

provisioning a collection file including a plurality of segments having a fixed size and separated by fixed offsets;

adding a collection file object ID (COID) for the collection file in an entry of a tablespace catalog;

initializing, for each one of the segments of the collection file, an object ID (OID) and an object segment index (OSEG) in an entry in a collection catalog; and

adding, to the entry in the collection catalog, the COID and a collection segment index indicating a location of the segment in the collection file.

13. The method of claim 12 further comprising:

receiving a command to write a database object to a segment of the segments in the collection file, the database object assigned an OID value;

calculating an OSEG value in relation to the OID value for the segment by dividing a page offset by a subsegment size defined for the segments;

calculating a page location in the segment as the page offset modulo the subsegment size; and

searching the collection catalog for an entry that matches the OID value and the OSEG value.

14. The method of claim 13 further comprising upon finding an entry in the collection catalog that matches the OID value and the OSEG value, performing a page write to the segment at the page location.

15. The method of claim 13 further comprising upon finding no entry in the collection catalog that matches the OID value and the OSEG value, searching the collection catalog for an entry indicating an unassigned segment and including the initialized OID and OSEG.

16. The method of claim 15 further comprising upon finding the entry indicating an unassigned segment, assigning the unassigned segment to the database object by setting the OID value and the OSEG value in the entry;

and

performing a page write to the segment at the page location in the collection file.

17. The method of claim 16 adding a segment format indicating a format of the segment, wherein the format of the segment is data only, initialization data only, data and a free space map, or a combination of data, a free space map, and a visibility map.

18. The method of claim 15 further comprising upon finding no entry indicating an unassigned segment and including the initialized OID and OSEG, reporting that there is not disk space available in the collection file.

19. The method of claim 12 further comprising:

receiving a command to free, in the collection file, all segments assigned to a database object with a given OID value;

searching the collection catalog for each entry that matches the OID value;

upon finding an entry that matches the OID value, sending a system call to de-allocate a segment at an offset in the collection file corresponding to the collection segment index in the entry; and

reinitializing the OID and the OSEG in the entry of the collection catalog.

20. A management component for database storage operations, the management component comprising:

at least one processor; and

a non-transitory computer readable storage medium storing programming for execution by the at least one processor, the programming including instructions to;

pre-allocate, in a logical sparse file, a plurality of segments fixed in size and contiguous at fixed offsets;

receive a command to write database objects to the segments;

map the database objects to the segments in a database catalog; and

interface with a file system component to initialize storage medium space for writing the data objects to the segments at the fixed offsets.

21. The management component of claim 20, wherein the programming includes further instructions to initialize storage medium space for writing the data objects to segments starting at the fixed offsets after pre-allocating the segments in the logical sparse file and after receiving the command to writhe the database objects to the segments.

22. The management component of claim 20, wherein the instructions to pre-allocate the segments in the logical sparse file includes instructions to pre-allocate the segments in the logical sparse file without initializing the storage medium space for the segments.

23. The management component of claim 20, wherein the instructions to map the database objects to the segments in the database catalog include instruction to add, in the database catalog, indices indicating object IDs of the database objects and object segment indices in relation to the object IDs in the logical sparse file.

24. The management component of claim 20, wherein the segments include subsegments fixed in size and contiguous at fixed subsegment offsets.

25. The management component of claim 20, wherein the database catalog is maintained in a non-volatile storage medium.