SCALABLE BLOB STORAGE INTEGRATED WITH SCALABLE STRUCTURED STORAGE

• US 20100106695A1
• Filed: 10/24/2008
• Published: 04/29/2010
• Est. Priority Date: 10/24/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: SRIVASTAV, SHASHWAT, BHARDWAJ, DEEPALI, NAGY, ALEXANDER FELSOBUKI, CALDER, BRADLEY GENE, NILAKANTAN, NIRANJAN, WANG, JU, WU, XINRAN
• Granted Patent: US 8,620,884 B2
• Time in Patent Office: Days
• US Class Current: 707/696
• CPC Class Codes: G06F 16/2255 Hash tables
• CPC Class Codes: G06F 16/24561 Intermediate data storage techniques for performance improvement

Abstract

Embodiments of the present invention relate to systems, methods and computer storage media for facilitating the structured storage of binary large objects (Blobs) to be accessed by an application program being executed by a computing device. Generally, the structured storage of Blobs includes a primary structured storage index for indexing Blobs, a secondary hash index that is integrated into the structured storage system, a Blob log stream, and a Blob data stream for storing blocks that include the Blob data. In an embodiment, a block is created and written to a Blob store along with a block list. The block list facilitates the locating of one or more blocks that store the Blob data. In this embodiment, a primary structured storage index and a secondary hash index are updated to facilitate efficient access of the Blob in a structured storage system.

Claims

1. One or more computer storage media having a data structure embodied thereon for structured storage of one or more Binary Large Objects (Blobs) to be accessed by an application program being executed by a computing device, comprising:

a primary structured storage index for indexing the one or more Blobs in a structured storage system;

a secondary hash index for maintaining a hash index that facilitates efficient lookup of a Blob primary key, wherein the secondary hash index includes one or more primary keys associated with the one or more Blobs; and

a Blob store for storing the one or more Blobs indexed by the primary structured storage index.

2. The computer storage media of claim 1, wherein the Blob store is a Blob log stream that includes one or more blocks for storing the one or more Blobs.

3. The computer storage media of claim 2, wherein the Blob log stream includes a record for updates made to the primary structured storage index.

4. The computer storage media of claim 3, wherein the one or more blocks and the record are written to the Blob log stream in a single write.

5. The computer storage media of claim 3, wherein a table server that loads a partition that includes at least one of the one or more Blobs, reconstructs a memory table for the table server based, in part, on the Blob log stream.

6. The computer storage media of claim 1, wherein the primary structured storage index includes at least one Blob property column.

7. The computer storage media of claim 6, wherein at least one Blob property includes information corresponding to at least one selection from the following,

a pointer to the Blob associated with the Blob property,

a pointer to a Temporary Block List (TBL), wherein the TBL maintains a list of one or more uncommitted blocks corresponding to the Blob associated with the Blob property,

a pointer to a Committed Block List (CBL), wherein the CBL maintains a list of one or more committed blocks corresponding to the Blob associated with the Blob property,

a last modified time for the TBL,

a size of the Blob associated with the Blob property,

a creation time of the Blob associated with the Blob property, and

metadata of the Blob associated with the Blob property.

8. The computer storage media of claim 1, wherein the secondary hash index is eagerly updated after a memory table of a partition served from a table server is updated.

9. The computer storage media of claim 1, wherein the secondary hash index is lazily updated after a memory table of a partition served from a table server is written to a file table of the partition.

10. The computer storage media of claim 9, wherein a queue associated with the secondary hash index includes an indication of additional updates pending for the secondary hash index, wherein the secondary hash index is useable when the queue is empty.

11. The computer storage media of claim 1, wherein the secondary hash index facilitates a single in-memory lookup of one of the one or more Blobs.

12. One or more computer storage media having computer-executable instructions embodied thereon for performing a method for writing a Binary Large Object (Blob) in a structured storage system to one or more computer storage media, the method comprising:

receiving the Blob;

creating one or more blocks for storing the Blob;

writing the one or more blocks into a Blob log stream;

creating a Temporary Block List (TBL) that includes the one or more blocks;

writing the TBL into the Blob log stream;

updating a primary structured storage index to index the Blob in a structured storage system, wherein the Blob is indexed by one or more keys; and

updating a secondary hash index for maintaining a secondary hash index of the primary structured storage index to facilitate an in memory mapping between the one or more keys and a pointer to the Blob.

13. The method of claim 12 further comprising:

reading the TBL to identify at least one of the one or more blocks storing the Blob;

creating a Committed Block List (CBL) that includes the at least one of the one or more block storing the Blob; and

updating the primary structured storage index to include a location identifier of the CBL.

14. The method of claim 13, wherein the one or more keys are primary keys comprised of at least a primary key for a row corresponding to the Blob and a Blob property of the Blob.

15. The method of claim 12, wherein a partition server determined dynamically inline the Blob with row data of the structured storage system.

16. The method of claim 12 wherein the one or more blocks and the TBL are written to the Blob log stream as a single write.

17. The method of claim 12, wherein the updating of the primary structured storage index further comprises updating a memory table of a partition served by a table server of the structured storage system, wherein the memory table is updated to indicate a change to a row of the primary structured storage index that is associated with the Blob.

18. The method of claim 12 further comprising setting a dirty bit in the primary structured storage index and the secondary hash index, wherein the dirty bit indicates information stored in a Blob header is inconsistent with information stored in the primary structured index.

19. The method of claim 12 further comprising loading a partition on a table server, wherein the partition is loaded based, in part, on replaying the Blob log stream and a row update log.

20. One or more computer storage media having a data structure embodied thereon for structured storage of one or more Binary Large Objects (Blobs) to be accessed by an application program being executed by a computing device, comprising:

a primary structured storage index for indexing the one or more Blobs in a structured storage system, wherein the primary structured storage index includes row data for each of the one or more Blobs;

the row data includes metadata corresponding to each of the one or more Blobs and the row data includes a pointer corresponding to each of the one or more Blobs;

a secondary hash index that is integrated into the structured storage system for maintaining a secondary hash index of the primary structured storage index, wherein the secondary hash index provides a single hash lookup for the one or more Blobs;

a Blob log stream that includes one or more blocks for storing the one or more Blobs at least until the primary structured storage index is checkpointed; and

a Blob data stream that includes the one or more blocks of the Blob log stream after the primary structured storage index is checkpointed.