Method and apparatus for database

• US 9,811,548 B2
• Filed: 03/01/2013
• Issued: 11/07/2017
• Est. Priority Date: 04/30/2012
• Status: Active Grant

• Current Assignee: MongoDB, Inc., Tihgtdb, Inc.
• Original Assignee: Tightdb, Inc. (MongoDB, Inc.)
• Inventors: Stigsen, Alexander, Christiansen, Bjarne, Reinhold, Lasse
• Primary Examiner(s): KUDDUS, DANIEL A
• Application Number:
  US14/398,109
• Publication Number:
  US 20150088844A1
• Time in Patent Office: 1,712 Days
• Field of Search: 707703, 707741, 707615, 707655, 707748, 707769, 707809, 707812, 707826, 707829
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/221 Column-oriented storage; Management thereof
• CPC Class Codes: G06F 16/2246 Trees, e.g. B+trees
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 8/30 Creation or generation of source code

Abstract

A method for representing a database in a computer readable medium. Accessing a first table in the database. The first table has rows of data, each row of data having one or more columns. Storing a first representation of the first table in a volatile computer memory in a column-oriented format, the column-oriented format comprising at least one B-tree for at least one column, wherein the B-tree is composed of at least one array and the at least one B-tree includes references to a plurality of arrays, each array of the plurality of arrays comprised of data corresponding to the rows of one column of the table, wherein the storing further comprising storing data corresponding to the rows in a packed format. Storing a second representation of the database on the computer-readable medium, the second representation substantially identical to the first representation.

Claims

1. A method for representing a database in a computer readable medium, the method comprising:

accessing a first table in the database, the first table having a plurality of rows of data, each row of data having one or more columns;

storing a first representation of the first table in a volatile computer memory in a column-oriented format, the column-oriented format comprising at least one B-tree for at least one column,

wherein the at least one B-tree is composed of at least one array and the at least one B-tree includes references to a plurality of arrays, each array of the plurality of arrays is comprised of data corresponding to rows of one column of the first table,

wherein the storing further comprises reducing an amount of storage space required when storing the first representation of the first table to improve storage efficiency by storing data of a particular column corresponding to the rows in a packed format using a power of two amount of space to store each datum and adaptively increasing the power of two amount of space to store each datum in the particular column when a datum is to be written to the particular column that has a value that requires an increased amount of space for storage to reduce use of unnecessary storage space; and

storing a second representation of the first table on a computer-readable medium, wherein the second representation is substantially identical to the first representation, without serialization of the first representation of the first table to an intermediate form, without deserialization from the intermediate form into the second representation, and without basic header information, metadata or other versioning information necessarily being identical.

2. The method of claim 1, further comprising

performing one or more database operations on the first representation while maintaining a database atomicity, consistency, isolation and durability guarantees on the database and the second representation in the computer-readable medium, the first representation and second representation remaining substantially identical at a successful completion of each transaction.

3. The method of claim 2, wherein the performing of the one or more database operations comprises a retrieval operation locating a matching row using one of (i) one of AND pattern matching instructions and exclusive OR (XOR) pattern matching instructions and (ii) simple instructions multiple data (SIMD) instructions applied to multiple row datum of a single column simultaneously.

4. The method of claim 2, wherein the performing of the one or more database operations comprises a copy-on-modify behavior with modifications being appended as copied arrays to an end of the first representation of the first table.

5. The method of claim 4, wherein the database is versioned across transactions via lookback to earlier arrays in the first representation.

6. The method of claim 1, wherein the storing of the second representation does not include any serialization or boxing of the second representation.

7. The method of claim 1, wherein the storing of the second representation is accomplished by mapping the volatile computer memory to a portion of the computer-readable medium.

8. The method of claim 1, wherein the packed format comprises using a power of two amount of space to store each datum.

9. A method of extending a definition of a first variable used in a first computer program, wherein the first computer program is encoded in a programming language, and the first computer program is stored in a computer-readable medium, the method comprising during execution of the first computer program on a computer system:

determining from the definition of the first variable a schema, wherein the schema describes a plurality of columns of a first table of a database;

defining a first representation of the first table in a computer-readable memory coupled to a first computer, the first representation comprising a column-oriented format further comprising at least one B-tree for at least one column,

wherein the at least one B-tree is composed of at least one array and the at least one B-tree includes references to a plurality of arrays, each array of the plurality of arrays is comprised of data corresponding to rows of one column of the first table,

wherein the defining further comprises reducing an amount of storage space required when storing data of a particular column to improve storage efficiency by storing data of the particular column corresponding to the rows in a packed format using a power of two amount of space to store each datum and adaptively increasing the power of two amount of space to store each datum in the particular column when a datum is to be written to the particular column that has a value that requires an increased amount of space for storage to reduce use of unnecessary storage space,

wherein the first representation supports a database transaction on the first table indicated by the first computer program, and

wherein the first representation provides database atomicity, consistency, isolation and durability guarantees on completion of the database transaction.

10. The method of claim 9, wherein the first representation is thread-safe across access by a second computer program operating on the database.

11. The method of claim 9, further comprising extending a definition of a second variable in a second computer program by repeating the determining and defining on the second variable, the second variable referring to the database.

12. The method of claim 11, wherein the first computer program and the second computer program are written in different programming languages.

13. The method of claim 9, wherein the first computer program is written in a general purpose computer programming language.

14. The method of claim 9, further comprising a retrieval operation for a single record in the database and wherein the retrieval operation is performed within a time defined by (n log n) with low constant overhead, where n represents a size of an input for the retrieval operation.

15. The method of claim 9, further comprising receiving a search having at least two criteria and executing the search using a zig-zag loop to identify a row matching the at least two criteria.

16. A non-transitory computer-readable storage medium storing instructions for representing a database in a volatile computer memory, the instructions which when executed by a processor cause the processor to:

access a first table in the database, the first table having a plurality of rows of data, each row of data having one or more columns;

store a first representation of the first table in the volatile computer memory in a column-oriented format, the column-oriented format comprising at least one B-tree for at least one column,

wherein the at least one B-tree is composed of at least one array and the at least one B-tree includes references to a plurality of arrays, each array of the plurality of arrays is comprised of data corresponding to rows of one column of the first table,

wherein the storing further comprises reducing an amount of storage space required when storing the first representation of the first table to improve storage efficiency by storing data of a particular column corresponding to the rows in a packed format using a power of two amount of space to store each datum and adaptively increasing the power of two amount of space to store each datum in the particular column when a datum is to be written to the particular column that has a value that requires an increased amount of space for storage to reduce use of unnecessary storage space; and

store a second representation of the first table on a computer-readable medium, wherein the second representation is substantially identical to the first representation, without serialization of the first representation of the first table to an intermediate form and without deserialization from the intermediate form into the second representation, without basic header information, metadata or other versioning information necessarily being identical.

17. A system for representing a database in a volatile computer memory, the system comprising:

a processor coupled to the volatile computer memory and a non-transitory computer-readable storage medium;

the non-transitory computer-readable storage medium storing instructions which when executed by a processor cause the processor to;

access a first table in the database, the first table having a plurality of rows of data, each row of data having one or more columns;

store a first representation of the first table in the volatile computer memory in a column-oriented format, the column-oriented format comprising at least one B-tree for at least one column,

wherein the at least one B-tree is composed of at least one array and the at least one B-tree includes references to a plurality of arrays, each array of the plurality of arrays is comprised of data corresponding to rows of one column of the first table,

wherein the storing further comprises reducing an amount of storage space required when storing the first representation of the first table to improve storage efficiency by storing data of a particular column corresponding to the rows in a packed format using a power of two amount of space to store each datum and adaptively increasing the power of two amount of space to store each datum in the particular column when a datum is to be written to the particular column that has a value which requires an increased amount of space for storage to reduce use of unnecessary storage space; and

store a second representation of the first table on a computer-readable medium, wherein the second representation is substantially identical to the first representation, without serialization of the first representation of the first table to an intermediate form, without deserialization from the intermediate form into the second representation, and without basic header information, metadata or other versioning information necessarily being identical.