Global dictionaries using universal primitives

• US 9,275,050 B2
• Filed: 10/10/2012
• Issued: 03/01/2016
• Est. Priority Date: 10/24/2011
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: Apptio (International Business Machines Corporation)
• Inventors: McLachlan, Paul Damien, Elliott, Tavis Dean
• Primary Examiner(s): Mackes, Kris
• Application Number:
  US13/649,019
• Publication Number:
  US 20130103654A1
• Time in Patent Office: 1,238 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/23 Updating
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/245 Query processing

Abstract

Embodiments are directed towards managing data storage and queries within a database system using global dictionaries with universal primitives (UNIPs) to represent non-numeric data within a mixed numeric/non-numeric environment. Common data types are managed within a same global dictionary through dictionaries that are globally used within the database system. At least non-numeric data within mixed data fields may be stored using a UNIP to identify the stored non-numeric data. The UNIP may take advantage of the IEEE-754 standard for floating point data representation by setting a first field within the UNIP to 0x7ff (HEX) to indicate that the data is non-numeric (NaN) and using remaining bits to store typed data, such as a date or unique indirect reference (e.g. a sequence number or file offset to larger piece of data). The UNIP may then replace the data within the database and be used during operations performed on the data.

Claims

1. A network device, comprising:

a database for managing mixed numeric and non-numeric data;

a storage device for storing the database, or other data; and

a processor that is operative to perform actions, including;

reading data within the database having mixed numeric and non-numeric data;

generating one or more global dictionaries for read data having one or more common data types in the database, wherein each read data value for a common data type is copied to a global dictionary;

when it is determined that the read data value for the common data type is non-numeric, performing further actions, including;

creating a universal primitive (UNIP) having a first field indicating that the read data value is non-numeric, and at least a second field that identifies a storage location within the storage device for storing the read data value, wherein the UNIP is assigned to the read non-numeric data value, and

replacing the read non-numeric data value within the database with the created UNIP, such that subsequent queries within the database manages queries using the UNIPs that replaced the read non-numeric data values having the common data types; and

when it is determined that the read value for the common data type'"'"'s values is numeric, replacing the common data type'"'"'s values with one or more keys for the global dictionary.

2. The network device of claim 1, wherein the UNIP is implemented using a Institute of Electrical and Electronics Engineers (IEEE) Standard for Floating-Point Arithmetic (IEEE-754) by setting a first field within the UNIP to 0x7ff (HEX) to indicate that the data is non-numeric (NaN).

3. The network device of claim 1, wherein the one or more global dictionaries are employed for managing each of a plurality of UNIPs created for the read non-numeric data values determined within the database.

4. The network device of claim 1, wherein the read non-numeric data value identified by the UNIP is encrypted and a substitution of the UNIP is used to remove identifiable information from the database.

5. The network device of claim 1, wherein the processor that is operative to perform actions, further including performing at least one database operation on data within the database, wherein the operation is directly performed on the UNIP rather than on the read non-numeric data value represented by the UNIP.

6. The network device of claim 1, wherein the processor that is operative to perform actions, including:

when the read data is non-numeric and is assigned to a previously created UNIP to represent it'"'"'s read non-numeric data value, replacing this read non-numeric data value within the database with the previously created UNIP.

7. The network device of claim 1, wherein at least a portion of the UNIP is encrypted.

8. A system, comprising:

a storage device that stores a database having stored therein mixed numeric and non-numeric data;

one or more processors operative to perform actions, including;

reading data within the database having mixed numeric and non-numeric data;

generating one or more global dictionaries for read data having one or more common data types in the database, wherein a read data value for a common data type is copied to a global dictionary;

when it is determined that the read data value for the common data type is non-numeric, performing further actions, including;

creating a universal primitive (UNIP) having a first field indicating that the read data value is non-numeric, and at least a second field that identifies a storage location within the storage device for storing the read data value, wherein the UNIP is assigned to the read non-numeric data value, and

replacing the read non-numeric data value within the database with the created UNIP, such that subsequent queries within the database manages queries using the UNIPs that replaced the read non-numeric data values having the common data types; and

when it is determined that the read value for the common data type'"'"'s values is numeric, replacing the common data type'"'"'s values with one or more keys for the global dictionary.

9. The system of claim 8, wherein the UNIP is implemented using a Institute of Electrical and Electronics Engineers (IEEE) Standard for Floating-Point Arithmetic (IEEE-754) by setting a first field within the UNIP to 0x7ff (HEX) to indicate that the data is non-numeric (NaN).

10. The system of claim 8, wherein the global dictionary is employed for managing each of a plurality of UNIPs created for the read non-numeric data values determined within the database.

11. The system of claim 8, wherein the storage address further comprises a first address to the storage location and an offset address from the first address.

12. The system of claim 8, wherein the processors are operative to perform actions, further including performing at least one database operation on data within the database, and wherein the operation is directly performed on the UNIP rather than on the non-numeric data represented by the UNIP.

13. The system of claim 8, wherein the processors are operative to perform actions, including:

when the read data is non-numeric and is assigned to a previously created UNIP to represent it'"'"'s read non-numeric data value, replacing the read non-numeric data value within the database with the previously created UNIP.

14. The system of claim 8, wherein the read non-numeric data value includes at least one of an alpha-numeric name, a date, or a currency indicator.

15. An apparatus comprising a non-transitory computer readable medium, having computer-executable instructions stored thereon, that in response to execution by a computing device, cause the computing device to perform operations, comprising:

reading data within the database having mixed numeric and non-numeric data;

generating one or more global dictionaries for read data having one or more common data types in the database, wherein a read data value for a common data type is copied to a global dictionary;

when it is determined that the read data value for the common data type is non-numeric, performing further actions, including;

creating a universal primitive (UNIP) having a first field indicating that the read data value is non-numeric, and at least a second field that identifies a storage location within the storage device for storing the read data value, wherein the UNIP is assigned to the read non-numeric data value, and

replacing the read non-numeric data value within the database with the created UNIP, such that subsequent queries within the database manages queries using the UNIPs that replaced the read non-numeric data values having the common data types; and

when it is determined that the read value for the common data type'"'"'s values is numeric, replacing the common data type'"'"'s values with one or more keys for the global dictionary.

16. The apparatus of claim 15, wherein the UNIP is implemented using a Institute of Electrical and Electronics Engineers (IEEE) Standard for Floating-Point Arithmetic (IEEE-754) by setting a first field within the UNIP to 0x7ff (HEX) to indicate that the data is non-numeric (NaN).

17. The apparatus of claim 15, wherein the global dictionary is employed for managing each of a plurality of UNIPs created for the read non-numeric data value determined within the database.

18. The apparatus of claim 15, wherein the storage address further comprises a first address to the storage location and an offset address from the first address.

19. The apparatus of claim 15, wherein the computing device to perform operations, further comprising:

performing at least one database operation on data within the database, wherein the operation is directly performed on the UNIP rather than on the read non-numeric data value represented by the UNIP.

20. The apparatus of claim 15, wherein the computing device to perform operations, further comprising:

when the read data value is non-numeric and is assigned to a previously created UNIP to represent the read non-numeric data value, replacing the read non-numeric data value within the database with the previously created UNIP.