Dynamic data management

• US 9,569,511 B2
• Filed: 07/26/2012
• Issued: 02/14/2017
• Est. Priority Date: 08/25/2011
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: Morin, Raymond
• Primary Examiner(s): Owyang, Michelle
• Application Number:
  US13/559,409
• Publication Number:
  US 20130054642A1
• Time in Patent Office: 1,664 Days
• Field of Search: 707/758, 707769-770
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2282 Tablespace storage structures; Management thereof
• CPC Class Codes: G06F 16/24544 Join order optimisation
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/25 Integrating or interfacing systems involving database management systems
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 16/3331 Query processing
• CPC Class Codes: G06F 16/3337 Translation of the query language, e.g. Chinese to English
• CPC Class Codes: G06F 16/80 of semi-structured data, e.g. markup language structured data such as SGML, XML or HTML content-based retrieval of web data G06F16/95

Abstract

An interface for users to gain access and manipulate unstructured data is provided. In response to receiving a user query associated with a first database format, a system can request unstructured data associated with a second database format from a second database. The unstructured data can include a set of data groups where each data group has a set of values. Each value can be associated with a different tag. To generate a structured database, some embodiments can determine the number of data groups and the number of unique tags across the data groups and populate the table with data from the unstructured data. Subsequently, the system can apply the user query to the table to obtain a query result and transmit the query result to the user device.

Claims

1. A method for accessing data, the method comprising:

receiving, with a server having one or more processors, a user query formatted according to a relational database query language utilized for a first database of a first database format, wherein the user query is utilized against unstructured data and structured data in the first database, wherein the unstructured data is stored in a second database of distributed, non-relational, semi-structured database format and utilizes a different query language than the first database and the user query;

requesting, with the one or more processors and in response to the user query, the structured data from the first database;

receiving, from the first database and with the one or more processors, in response to the requesting, the structured data from the first database;

requesting, with the one or more processors and in response to the user query, the unstructured data from the second database using a unstructured data request created from the user query, wherein the unstructured data request is formatted according to the different query language;

receiving, from the second database, with the one or more processors, in response to the requesting, the unstructured data including at least a set of data groups, each data group having a set of one or more values, each value of a respective data group being associated with a different tag;

determining, in response to receiving the unstructured data, with the one or more processors, a first number of data groups in the unstructured data;

determining, in response to receiving the unstructured data, with the one or more processors, a second number of unique tags across the data groups of the unstructured data;

creating, with the one or more processors, a table in the first database format, wherein the number of columns of the table is based on the second number of unique tags, and the number of rows of the table corresponds to the first number of data groups;

generate a populated table by populating, with the one or more processors, each row of the table with the unstructured data of a respective data group by adding each value to the column corresponding to the associated tag;

creating, with the one or more processors, a joined table comprising data of the populated table and the structured data from the first database;

applying, with the one or more processors, the user query to the joined table to obtain a query result; and

transmitting, with the one or more processors, the query result to the user device.

2. The method of claim 1, wherein the unstructured data is time series-based, wherein the user query for unstructured data includes receiving a specified time interval for the unstructured data.

3. The method of claim 1, wherein the table includes a set of entries and wherein populating each row of the table includes inserting a null value in a subset of entries that does not have a corresponding value from the unstructured data.

4. The method of claim 1 wherein the joined table is malleable based on function calls in the first database format.

5. The method of claim 1, wherein the unstructured data is log data from an event log.

6. The method of claim 1, wherein each of the set of data groups is associated with a timestamp.

7. The method of claim 1 further comprising:

determining a data type for each data in the set of data group; and

verifying that the data type for each data in each column of the table.

8. A non-transitory computer readable storage medium encoded with program instructions that, when executed, are configurable to cause one or more processors to:

receive a user query formatted according to a relational database query language utilized for a first database of a first database format, wherein the user query is utilized against unstructured data and structured data in the first database, wherein the unstructured data is stored in a second database of distributed, non-relational, semi-structured database format and utilizes a different query language than the first database and the user query;

request, in response to the user query, the structured data from the first database;

receive, from the first database and in response to the request, the structured data from the first database;

request, in response to the user query, the unstructured data from the second database using a unstructured data request created from the user query, wherein the unstructured data request is formatted according to the different query language;

receive, from the second database, in response to the requesting, the unstructured data including at least a set of data groups, each data group having a set of one or more values, each value of a respective data group being associated with a different tag;

determine, in response to receiving the unstructured data, a first number of data groups in the unstructured data;

determine, in response to receiving the unstructured data, a second number of unique tags across the data groups of the unstructured data;

create a table in the first database format, wherein the number of columns of the table is based on the second number of unique tags, and the number of rows of the table corresponds to the first number of data groups;

generate a populate table by populate each row of the table with the unstructured data of a respective data group by adding each value to the column corresponding to the associated tag;

create a joined table comprising data of the populated table and the structured data from the first database;

apply the user query to the joined table to obtain a query result; and

transmit the query result to the user device.

9. The non-transitory computer readable storage medium of claim 8, wherein the unstructured data is time series-based, wherein receiving the user query for unstructured data includes a specified time interval for the unstructured data.

10. The non-transitory computer readable storage medium of claim 8, wherein the populated table includes a set of entries and wherein populating each row of the table includes inserting a null value in a subset of entries that does not have a corresponding value from the unstructured data.

11. The non-transitory computer readable storage medium of claim 8, wherein the joined table is malleable based on function calls in the first database format.

12. The non-transitory computer readable storage medium of claim 8, wherein the unstructured data is log data from an event log.

13. A system comprising:

one or more processors; and

a memory device including instructions that, when executed by the one or more processors, are configurable to cause the system to;

receive a user query formatted according to a relational database query language utilized for a first database of a first database format, wherein the user query is utilized against unstructured data and structured data in the first database, wherein the unstructured data is stored in a second database of distributed, non-relational, semi-structured database format and utilizes a different query language than the first database and the user query;

request, in response to the user query, the structured data from the first database;

receive, from the first database and in response to the request, the structured data from the first database;

request, in response to the user query, the unstructured data from the second database using a unstructured data request created from the user query, wherein the unstructured data request is formatted according to the second database;

receive, from the second database, in response to the requesting, the unstructured data including at least a set of data groups, each data group having a set of one or more values, each value of a respective data group being associated with a different tag;

determine, in response to receiving the unstructured data, a first number of data groups in the unstructured data;

determine, in response to receiving the unstructured data, a second number of unique tags across the data groups of the unstructured data;

create a table in the first database format, wherein the number of columns of the table is based on the second number of unique tags, and the number of rows of the table corresponds to the first number of data groups;

generate a populated table by populate each row of the table with the unstructured data of a respective data group by adding each value to the column corresponding to the associated tag;

create a joined table comprising data of the populated table and the structured data from the first database;

apply the user query to the joined table to obtain a query result; and

transmit the query result to the user device.

14. The system of claim 13, wherein the unstructured data is time series-based, wherein receiving the user query for unstructured data includes a specified time interval for the unstructured data.

15. The system of claim 13, wherein the populated table includes a set of entries and wherein populating each row of the table includes inserting a null value in a subset of entries that does not have a corresponding value from the unstructured data.

16. The system of claim 13, wherein the unstructured data is log data from an event log.