Systems and methods for operating a server-side data abstraction layer

• US 10,331,740 B2
• Filed: 02/10/2015
• Issued: 06/25/2019
• Est. Priority Date: 02/10/2014
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Low, Yucheng, Gu, Haijie, Wang, Ping, Samanas, Evan, Raman, Sethu, Guestrin, Carlos
• Primary Examiner(s): Gortayo, Dangelino N
• Application Number:
  US14/619,020
• Publication Number:
  US 20150227569A1
• Time in Patent Office: 1,596 Days
• Field of Search: 707713, 707602, 707718, 707756, 707708, 707737, 707777, 707798
• CPC Class Codes: G06F 16/211 Schema design and management
• CPC Class Codes: G06F 16/221 Column-oriented storage; Management thereof
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/2343 Locking methods, e.g. distributed locking or locking implementation details
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27
• CPC Class Codes: G06F 16/2453 Query optimisation
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: G06F 16/9024 Graphs; Linked lists G06F16/9027 takes precedence
• CPC Class Codes: G06F 16/958 Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking

Abstract

A method receives a first request from a client object at a device. The first request specifies a data source. In response to the first request, the method uploads data from the data source, stores the data as a plurality of first columns, and instantiates a first server object that provides access to the first columns. The method later receives a second request from the client object. The second request specifies a transformation of the data. In response to the second request, the method stores one or more additional columns and instantiates a second server object that provides access to the additional columns and one or more of the first columns. Each of the additional columns is constructed from the first columns according to the requested transformation, and each of the additional columns includes a plurality of data values all having the same data type.

Claims

1. A method of operating a server-side data abstraction layer, comprising:

at a server system having one or more processors, non-volatile memory, and volatile memory storing one or more programs configured for execution by the one or more processors;

receiving a first request from a first client object at a first client device, wherein the first request specifies a data source;

in response to the first request;

uploading data from the specified data source;

storing the data as a plurality of first columns in the non-volatile memory; and

instantiating a first server object that provides access to the first columns, wherein each column of the plurality of first columns comprises a plurality of data values all having the same data type;

receiving a second request from the first client object at the first client device, wherein the second request specifies a transformation of the data; and

in response to the second request;

constructing, from the first columns according to the requested transformation, one or more additional columns comprising a plurality of data values all having the same data type;

storing the one or more additional columns in the volatile memory; and

instantiating a second server object that provides access to the additional columns at the volatile memory and one or more of the first columns at the non-volatile memory.

2. The method of claim 1, wherein the data source is selected from the group consisting of:

a CSV file stored on the first client device;

a CSV file stored in the non-volatile memory of the server system;

a CSV file stored at a remote location specified by a URL;

a flat file stored at the first client device; and

a result set retrieved from an SQL database using an SQL query.

3. The method of claim 1, further comprising:

receiving a request from the first client object to read the transformed data;

in response to the request to read the transformed data, retrieving the corresponding additional columns and one or more first columns from the non-volatile storage and transmitting the retrieved additional columns and one or more first columns to the first client device.

4. The method of claim 1, further comprising:

receiving a request from a client-side graph object at the first client device to use the transformed data, wherein the request specifies whether to use the transformed data as vertices or edges;

in response to the request, building a server-side graph object corresponding to the client-side graph object, the server-side graph object using the transformed data, and not transmitting any of the transformed data to the client-side graph object, wherein the server-side graph object comprises a set of vertices and a set of edges, each edge connecting a pair of vertices.

5. The method of claim 1, wherein each of the first columns is stored as a distinct file in the non-volatile memory.

6. The method of claim 1, wherein each of the first columns has the same number N of data values.

7. The method of claim 6, wherein at least one of the first columns has at least one data value that is missing.

8. The method of claim 6, wherein the transformation constructs a second column of the additional columns using a formula, wherein for each i in {1, 2, . . . , N}, the formula computes the ith data value of the second column using the ith data values of one or more of the first columns.

9. The method of claim 1, wherein the server system comprises a plurality of servers, each with a one or more processors, non-volatile memory, and volatile memory storing one or more programs configured for execution by the respective one or more processors.

10. The method of claim 1, further comprising:

receiving a request from a second client object at a second client device to build a corresponding second server object whose data comes from the data source as specified by the first request at the first client device;

determining that the data for the second server object is already stored as the first columns in the non-volatile memory;

updating metadata for the second server object to access the first columns, thereby providing access to the requested data without re-uploading the data from the specified data source.

11. A server system, comprising one or more servers, each having:

one or more processors;

non-volatile memory; and

volatile memory storing one or more programs configured for execution by the one or more processors, the one or more programs comprising instructions that cause the one or more processors to perform operations including;

receiving a first request from a first client object at a first client device, wherein the first request specifies a data source;

in response to the first request;

uploading data from the specified data source;

storing the data as a plurality of first columns in the non-volatile memory; and

instantiating a first server object that provides access to the first columns, wherein each column of the plurality of first columns comprises a plurality of data values all having the same data type;

receiving a second request from the first client object at the first client device, wherein the second request specifies a transformation of the data; and

in response to the second request;

constructing, from the first columns according to the requested transformation, one or more additional columns comprising a plurality of data values all having the same data type;

storing the one or more additional columns in the volatile memory; and

instantiating a second server object that provides access to the additional columns at the volatile memory and one or more of the first columns at the non-volatile memory.

12. The server system of claim 11, wherein the data source is selected from the group consisting of:

a CSV file stored on the first client device;

a CSV file stored in the non-volatile memory of the server system;

a CSV file stored at a remote location specified by a URL;

a flat file stored at the first client device; and

a result set retrieved from an SQL database using an SQL query.

13. The server system of claim 11, the one or more programs further comprising instructions that cause the one or more processors to perform operations including:

receiving a request from a client-side graph object at the first client device to use the transformed data, wherein the request specifies whether to use the transformed data as vertices or edges;

in response to the request, building a server-side graph object corresponding to the client-side graph object, the server-side graph object using the transformed data, and not transmitting any of the transformed data to the client-side graph object, wherein the server-side graph object comprises a set of vertices and a set of edges, each edge connecting a pair of vertices.

14. The server system of claim 11, wherein each of the first columns is stored as a distinct file in the non-volatile memory.

15. The server system of claim 11, wherein each of the first columns has the same number N of data values.

16. The server system of claim 15, wherein at least one of the first columns has at least one data value that is missing.

17. The server system of claim 15, wherein the transformation constructs a second column of the additional columns using a formula, wherein for each i in {1, 2, . . . , N}, the formula computes the ith data value of the second column using the ith data values of one or more of the first columns.

18. A non-transitory, computer readable storage medium storing one or more programs configured for execution by one or more processors of a server system having non-volatile memory and volatile memory, the one or more programs comprising instructions that cause the one or more processors to perform operations including:

receiving a first request from a first client object at a first client device, wherein the first request specifies a data source;

in response to the first request;

uploading data from the specified data source;

storing the data as a plurality of first columns in the non-volatile memory; and

instantiating a first server object that provides access to the first columns, wherein each column of the plurality of first columns comprises a plurality of data values all having the same data type;

receiving a second request from the first client object at the first client device, wherein the second request specifies a transformation of the data; and

in response to the second request;

constructing, from the first columns according to the requested transformation, one or more additional columns comprising a plurality of data values all having the same data type;

storing the one or more additional columns in the volatile memory; and

instantiating a second server object that provides access to the additional columns at the volatile memory and one or more of the first columns at the non-volatile memory.

19. The non-transitory computer readable storage medium of claim 18, the one or more programs further comprising instructions for:

receiving a request from a client-side graph object at the first client device to use the transformed data, wherein the request specifies whether to use the transformed data as vertices or edges;

in response to the request, building a server-side graph object corresponding to the client-side graph object, the server-side graph object using the transformed data, and not transmitting any of the transformed data to the client-side graph object, wherein the server-side graph object comprises a set of vertices and a set of edges, each edge connecting a pair of vertices.

20. The non-transitory computer readable storage medium of claim 18, wherein each of the first columns has the same number N of data values.

21. The non-transitory computer readable storage medium of claim 20, wherein the transformation constructs a second column of the additional columns using a formula, wherein for each i in {1, 2, . . . , N}, the formula computes the ith data value of the second column using the ith data values of one or more of the first columns.