Management of structured, non-structured, and semi-structured data in a multi-tenant environment

• US 10,642,863 B2
• Filed: 05/27/2015
• Issued: 05/05/2020
• Est. Priority Date: 05/27/2015
• Status: Active Grant

• Current Assignee: Datto LLC
• Original Assignee: Kaseya International Limited (Kaseya Limited)
• Inventors: Fischer, Mark, Khot, Prakash, Arcari, Daniel Philip
• Primary Examiner(s): Gortayo, Dangelino N
• Application Number:
  US14/722,403
• Publication Number:
  US 20160350303A1
• Time in Patent Office: 1,805 Days
• Field of Search: 707763, 707722, 707759, 707760, 707769, 707770
• CPC Class Codes: G06F 16/2452 Query translation
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 16/256 in federated or virtual databases
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 16/284 Relational databases
• CPC Class Codes: G06F 16/9024 Graphs; Linked lists G06F16/9027 takes precedence

Abstract

Disclosed is a network of systems that includes plural disparate storage systems that store user data, the disparate storage systems including NoSQL server databases that provide storage and retrieval of data modeled in forms besides tabular relations used in relational databases, and index storage system, a relational graph storage system and one or more data storage query platforms in communication with the plural disparate storage that have queries produced in a modeling language that abstracts application programmer functionality from network functionality.

Claims

1. A network of systems (network), the network comprising:

one or more systems that execute applications that generate standard syntax wrapped query language queries that are executable against data objects that are instances of data elements, and which data objects include metadata that describe a structure and a type of data within the data objects, with each of the standard syntax wrapped query language queries having a parameter that is an identifier to a specific one of the data objects;

one or more data storage query platforms that produce No-SQL queries from the received standard syntax wrapped query language queries, with the one or more data storage query platforms in communication with the one or more systems and with one or more No-SQL storage systems that store and retrieve data, with at least one of the data storage query platforms, (the data storage query platform) configured to;

receive a first one of the generated standard syntax wrapped query language queries (received standard syntax wrapped query language query);

determine from the identifier a platform type corresponding to the one or more No-SQL storage systems on which the received query is executable;

translate the received standard syntax wrapped query language query into a native language query that is executable on the determined one or more No-SQL storage systems, according to query semantics specific to the determined platform type; and

send the translated native language query to the determined one or more No-SQL storage systems.

2. The network of claim 1 wherein the identifier in the parameter of the standard syntax wrapped query is one or more query filters other than the key, which one or more filter elements cause the one or more data storage query platforms to conduct a full index search.

3. The network of claim 1 wherein at least some of the one or more systems are further configured to:

execute an event driven computing platform that produces workflows that are saved in one or more of the plural disparate storage systems, with the workflows configured to operate on events that are received over event channels.

4. The network of claim 1 wherein at least some of the one or more systems are further configured to:

produce queries for the data storage query platform from instances written in a domain specific modeling language.

5. The network of claim 4 wherein the one or more client devices are further configured to produce from the instances, platform independent code and artifact files that describe a logical structure of a programming entity.

6. The network of claim 1 wherein at least some of the one or more systems are further configured to:

execute a programming structure platform that produces an extension structure that includes a manifest structure and executable code for remote execution on a managed device in a second network.

7. The network of claim 1 wherein the one or more systems are configured to:

execute an event driven computing platform to produce workflows from routines or routine chains according received user-supplied designations; and

store the produced workflow in one or more of the disparate storage systems.

8. The network of claim 1 wherein the one or more systems are configured to:

receive executable code;

generate a manifest structure, from source code files associated with the executable code, the manifest structure including a document that is a human readable manifest and a machine readable manifest that is version of the human readable manifest;

produce an extension comprising the manifest structure and the received executable code;

apply a digital signature to the extension by using a private key.

9. The network of claim 1 wherein when the data storage query platform persists the specific data object, the data storage query platform determines an appropriate No-SQL data store based on the specific data object'"'"'s metadata by storing data attributes in the plural No-SQL storage systems in one or more of NoSQL server databases as structured data and in one or more index storage systems, as an index-able document.

10. The network of claim 9 wherein when the data storage query platform persists the specific data object that contains a reference to a relationship graph, the data storage query platform determines an appropriate data store based on the specific data object'"'"'s metadata, and by further storing the relationship graph in one or more of the plural No-SQL storage systems that include one or more relational graph storage systems.

11. The network of claim 9 wherein when the data storage query platform persists the specific data object that contains a reference to a binary large object (BLOB), the data storage query platform determines an appropriate data store based on the specific data object'"'"'s metadata, and by further storing the BLOB object in one or more of the plural No-SQL storage systems that include one or more binary large object (BLOB) storage systems.

12. The network of claim 1 wherein when the data storage query platform retrieves data in response to the standard syntax wrapped query language query (query), the data storage query platform determines an appropriate data store from which to retrieve the data based on the specific data object'"'"'s metadata by applying a key specified in the query to retrieve the data from one or more of the plural No-SQL storage systems that include one or more of NoSQL server databases as a lookup.

13. The network of claim 12 wherein when the query specifies any query filters other than the key, the data storage query platform retrieves the data from one or more of the plural No-SQL storage systems that include an index storage entity using the filter element(s) for a full index search.

14. The network of claim 13 wherein one or more of the plural No-SQL storage systems include one or more relational graph storage systems and/or one or more binary large object (BLOB) storage systems, and when the specific data object'"'"'s metadata indicates the data element contains references to a relationship graph and/or a binary large object (BLOB), the data storage query platform executes subsequent queries against either the one or more relational graph storage systems and/or the one or more binary large object (BLOB) storage systems.

15. The network of claim 1, further comprising:

the one or more of the plural No-SQL storage systems.

16. The network of claim 15 wherein the one or more plural No-SQL storage systems include one or more of NoSQL server databases, one or more of index storage systems, and one or more relational graph storage systems.

17. The network of claim 16 wherein the NoSQL server databases store and retrieve data that are modeled in forms other than tabular relationship models used in relational databases.

18. A method of operating a network of systems, the method comprising:

receiving by one or more data storage query platforms, a standard syntax wrapped query language query that is received from a system executing an application, which system is within an application level of the network, with the standard syntax wrapped query language query being executing against data objects that are instances of data elements, and which data objects include metadata that describe a structure and a type of data within the data objects, and with the standard syntax wrapped query language query having a parameter an identifier to a specific one of the data objects;

producing by the one or more data storage query platforms one or more No-SQL queries from the received standard syntax wrapped query language queries, with the one or more data storage query platforms in communication with the computing system and with one or more No-SQL storage systems that store and retrieve data, with at least one of the data storage query platforms, (the data storage query platform) producing the one or more No-SQL queries by;

determining from the identifier in the metadata in the received one of the standard syntax wrapped query language queries, a platform type from which the received the standard syntax wrapped query language query is executable;

translating the received standard syntax wrapped query language query into a native language query that is executable on the determined one or more No-SQL storage systems, according to query semantics specific to the determined No-SQL storage system type; and

sending by the computing system, the translated native language query to the determined one or more No-SQL storage systems in communication with the data storage query platform.

19. The method of claim 18 further comprising:

executing by one or more client devices an event driven computing process to produce workflows that are saved in one or more of the No-SQL storage systems, with the workflows configured to operate on events that are received over event channels.

20. The method of claim 18 further comprising:

executing by one or more client devices a programming structure process that produces an extension structure that includes a manifest structure and executable code for remote execution on a managed device in a second network.

21. The method of claim 18 wherein when the data storage query platform persists the specific data object, the data storage query platform determines an appropriate No-SQL data store based on the specific data object'"'"'s metadata by storing data attributes in the plural No-SQL storage systems in one or more of NoSQL server databases as structured data and in one or more index storage systems, as an indexable document.