UNIVERSAL DATA STORAGE SYSTEM THAT MAINTAINS DATA ACROSS ONE OR MORE SPECIALIZED DATA STORES

• US 20150169757A1
• Filed: 12/12/2013
• Published: 06/18/2015
• Est. Priority Date: 12/12/2013
• Status: Active Grant

• Current Assignee: Netflix, Inc.
• Original Assignee: Netflix, Inc.
• Inventors: Kalantzis, Christos, Singh, Shyam
• Granted Patent: US 9,430,539 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/213 with details for schema evolution support
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database

Abstract

In an embodiment, a method comprises receiving an input defining organization of one or more universal data elements; generating a universal schema that defines organization of the one or more universal data elements based on the input; associating the universal schema with a specialized plug-in, wherein the specialized plug-in maps the one or more universal data elements with data elements in one or more specialized databases maintained by a specialized data store; receiving, from a computer, a request to receive data stored in the specialized data store and comprising parameters based on the universal schema; transforming the request using the specialized plug-in to produce a specialized query; sending the specialized query to the specialized data store to execute the specialized query; receiving a specialized result set from the specialized data store; sending, to the computer, a result set based on specialized result set.

Claims

1. A method comprising:

receiving a first input defining organization of one or more universal data elements;

generating a first universal schema that defines organization of the one or more universal data elements based on the first input;

associating the first universal schema with a first specialized plug-in, wherein the first specialized plug-in maps the one or more universal data elements into data elements a specialized database in a first specialized data store;

receiving, from a first computer, a first request to receive data stored in the first specialized data store and comprising parameters based on the first universal schema;

transforming the first request using the first specialized plug-in associated with the first universal schema to produce a specialized query;

sending the specialized query to the first specialized data store to execute the specialized query;

receiving a first specialized result set from the first specialized data store;

sending, to the first computer, a result set based on specialized result set;

wherein the method is performed by one or more second computers.

2. The method of claim 1 comprising:

receiving the first request from the first computer in an HTTP request;

sending the result set to the first computer using an HTTP response.

3. The method of claim 1, wherein the first request is a write request that includes a payload, and the method comprises storing the payload in the first specialized data store according to the first specialized plug-in.

4. The method of claim 1, wherein the first request is an update request that includes a payload, and the method comprises storing the payload in the first specialized data store according to the first specialized plug-in.

5. The method of claim 1, wherein the first request is a delete request, and the method comprises deleting data from the first specialized data store according to the first specialized plug-in.

6. The method of claim 1, wherein the first request is a read request, the first specialized data store includes a specialized schema that defines organization in the first specialized data store of one or more data elements that correspond to the one or more universal data elements, and the first specialized result set is formatted based the specialized schema, and the method comprises transforming the first specialized result set to produce the result set using the first specialized plug-in, wherein the result set is formatted based on the first universal schema.

7. The method of claim 6 comprising:

receiving a second input defining how one or more second universal data elements are organized;

generating a second universal schema that defines how the one or more second universal data elements are organized;

associating the second universal schema with a second specialized plug-in, wherein the second specialized plug-in defines how the one or more universal data elements are organized in a second specialized data store;

receiving a second request with parameters based on the first universal schema to receive data stored in the second specialized data store;

transforming the second request using the second specialized plug-in associated with the second universal schema to produce a second specialized query;

sending the second specialized query to the second specialized data store to execute the second specialized query;

receiving a second specialized result set from the second specialized data store;

transforming the second specialized result set to produce a second universal result set using the second specialized plug-in, wherein the second universal result set is formatted based on the second universal schema;

sending the second universal result set to the remote client computer that the second request was received from.

8. The method of claim 1 comprising:

receiving a second input defining how one or more second universal data elements are organized;

generating a second universal schema that defines how the one or more second universal data elements are organized;

associating the second universal schema with a second specialized plug-in, wherein the second specialized plug-in defines how the one or more universal data elements are organized in a second specialized data store;

receiving the first request with parameters based on the first universal schema and the second universal schema to receive data stored in the first specialized data store and the second specialized data store;

transforming the first request to produce a second specialized query using the second specialized plug-in;

sending the second specialized query to the second specialized data store to execute the second specialized query;

receiving a second specialized result set from the second specialized data store;

transforming the first specialized result set to a first universal result set using the first specialized plug-in, wherein the first universal result set is formatted based on the first universal schema;

transforming the second specialized result set to a second universal result set using the second specialized plug-in, wherein the second universal result set is formatted based on the second universal schema;

joining the first specialized result set and the second specialized result set to produce the result set.

9. The method of claim 1 comprising:

receiving second input defining one or more operations to perform on a universal result set if the first request includes a particular parameter;

receiving the first request with the particular parameter;

transforming the first specialized result set to the universal result set using the first specialized plug-in;

performing the one or more operations on the universal result set to produce the result set.

10. The method of claim 1 comprising:

receiving an instruction to replicate new data sent to the first specialized data store to a second specialized data store, wherein the first specialized data store and the second specialized data store are a same type of data store;

determining that the first request is not a read request and in response sending the specialized query to the second specialized data store.

11. The method of claim 1 comprising:

receiving an instruction to replicate new data sent to the first specialized data store to a second specialized data store, wherein the first specialized data store is a different type of data store than the second specialized data store, and in response;

associating the first universal schema with a second specialized plug-in, wherein the second specialized plug-in defines how the one or more universal data elements are organized in the second specialized data store;

determining that the first request is not a read request and in response;

transforming the first request using the second specialized plug-in associated with the first universal schema to produce a second specialized query;

sending the second specialized query to the second specialized data store to execute the second specialized query.

12. The method of claim 1 comprising:

associating the first universal schema with a second specialized plug-in;

transforming the first request using the first specialized plug-in associated with the first universal schema to produce a first specialized query to retrieve a first set of results from the first specialized data store;

wherein the first specialized query comprises one or more instructions that are compatible with the first specialized data store;

transforming the first request using the second specialized plug-in associated with the first universal schema to produce a second specialized query to retrieve a second set of results from a second specialized data store;

wherein the second specialized query comprises one or more instructions that are compatible with the second specialized data store;

wherein the second specialized query is different than the first specialized query;

receiving the first specialized result set from the first specialized data store based on the first specialized query;

receiving a second specialized result set from the second specialized data store based on the second specialized query;

transforming the first specialized result set to a first universal result set, wherein the first universal result set is based on the first universal schema;

transforming the second specialized result set to a second universal result set, wherein the second universal result set is based on the first universal schema;

joining the first universal result set and the second universal result set to produce the result set, which is a universal result set that is based on the first universal schema.

13. The method of claim 1 comprising associating the first universal schema with the first specialized plug-in, which defines one or more methods for transforming a universal request into one or more specialized queries that create, read, update, and delete one or more data items in the first specialized data store.

14. A method comprising:

receiving an input defining organization of one or more universal data elements;

generating a universal schema that defines organization of the one or more universal data elements based on the input;

associating the universal schema with a first specialized plug-in, wherein the first specialized plug-in defines how the one or more universal data elements are organized in a first specialized data store;

receiving a command to copy data in the first specialized data store to a second specialized data store;

associating the universal schema with a second specialized plug-in, wherein the second specialized plug-in defines organization of the one or more universal data elements in the second specialized data store;

generating a first request to receive a set of data from the first specialized data store;

transforming the first request using the first specialized plug-in to produce a first specialized query;

sending the first specialized query to the first specialized data store to execute the first specialized query;

receiving a specialized result set from the first specialized data store;

transforming the specialized result set using the first specialized plug-in to produce a universal result set, wherein the universal result set is formatted based on the universal schema;

generating a second specialized query, using the second specialized plug-in, to store the universal result set in the second specialized data store;

sending the second specialized query to the second specialized data store;

wherein the method is performed by one or more second computers.

15. One or more non-transitory computer-readable media storing one or more sequences of instructions which, when executed by one or more computing devices, cause:

receiving a first input defining organization of one or more universal data elements;

generating a first universal schema that defines organization of the one or more universal data elements based on the first input;

associating the first universal schema with a first specialized plug-in, wherein the first specialized plug-in maps the one or more universal data elements into data elements a specialized database in a first specialized data store;

receiving, from a first computer, a first request to receive data stored in the first specialized data store and comprising parameters based on the first universal schema;

transforming the first request using the first specialized plug-in associated with the first universal schema to produce a specialized query;

sending the specialized query to the first specialized data store to execute the specialized query;

receiving a first specialized result set from the first specialized data store;

sending, to the first computer, a result set based on specialized result set.

16. The one or more non-transitory computer-readable media of claim 15 comprising sequences of instructions which when executed cause:

receiving the first request from the first computer in an HTTP request;

sending the result set to the first computer using an HTTP response.

17. The one or more non-transitory computer-readable media of claim 15, wherein the first request is a write request that includes a payload, and comprising sequences of instructions which when executed cause storing the payload in the first specialized data store according to the first specialized plug-in.

18. The one or more non-transitory computer-readable media of claim 15, wherein the first request is an update request that includes a payload, and comprising sequences of instructions which when executed cause storing the payload in the first specialized data store according to the first specialized plug-in.

19. The one or more non-transitory computer-readable media of claim 15, wherein the first request is a delete request, and comprising sequences of instructions which when executed cause deleting data from the first specialized data store according to the first specialized plug-in.

20. The one or more non-transitory computer-readable media of claim 15, wherein the first request is a read request, the first specialized data store includes a specialized schema that defines organization in the first specialized data store of one or more data elements that correspond to the one or more universal data elements, and the first specialized result set is formatted based the specialized schema, and comprising sequences of instructions which when executed cause transforming the first specialized result set to produce the result set using the first specialized plug-in, wherein the result set is formatted based on the first universal schema.

21. The one or more non-transitory computer-readable media of claim 20 comprising sequences of instructions which when executed cause:

receiving a second input defining how one or more second universal data elements are organized;

generating a second universal schema that defines how the one or more second universal data elements are organized;

associating the second universal schema with a second specialized plug-in, wherein the second specialized plug-in defines how the one or more universal data elements are organized in a second specialized data store;

receiving a second request with parameters based on the first universal schema to receive data stored in the second specialized data store;

transforming the second request using the second specialized plug-in associated with the second universal schema to produce a second specialized query;

sending the second specialized query to the second specialized data store to execute the second specialized query;

receiving a second specialized result set from the second specialized data store;

transforming the second specialized result set to produce a second universal result set using the second specialized plug-in, wherein the second universal result set is formatted based on the second universal schema;

sending the second universal result set to the remote client computer that the second request was received from.

22. The one or more non-transitory computer-readable media of claim 15 comprising sequences of instructions which when executed cause:

receiving a second input defining how one or more second universal data elements are organized;

generating a second universal schema that defines how the one or more second universal data elements are organized;

associating the second universal schema with a second specialized plug-in, wherein the second specialized plug-in defines how the one or more universal data elements are organized in a second specialized data store;

receiving the first request with parameters based on the first universal schema and the second universal schema to receive data stored in the first specialized data store and the second specialized data store;

transforming the first request to produce a second specialized query using the second specialized plug-in;

sending the second specialized query to the second specialized data store to execute the second specialized query;

receiving a second specialized result set from the second specialized data store;

transforming the first specialized result set to a first universal result set using the first specialized plug-in, wherein the first universal result set is formatted based on the first universal schema;

transforming the second specialized result set to a second universal result set using the second specialized plug-in, wherein the second universal result set is formatted based on the second universal schema;

joining the first specialized result set and the second specialized result set to produce the result set.

23. The one or more non-transitory computer-readable media of claim 15 comprising sequences of instructions which when executed cause:

receiving second input defining one or more operations to perform on a universal result set if the first request includes a particular parameter;

receiving the first request with the particular parameter;

transforming the first specialized result set to the universal result set using the first specialized plug-in;

performing the one or more operations on the universal result set to produce the result set.

24. The one or more non-transitory computer-readable media of claim 15 comprising sequences of instructions which when executed cause:

receiving an instruction to replicate new data sent to the first specialized data store to a second specialized data store, wherein the first specialized data store and the second specialized data store are a same type of data store;

determining that the first request is not a read request and in response sending the specialized query to the second specialized data store.

25. The one or more non-transitory computer-readable media of claim 15 comprising sequences of instructions which when executed cause:

receiving an instruction to replicate new data sent to the first specialized data store to a second specialized data store, wherein the first specialized data store is a different type of data store than the second specialized data store, and in response;

associating the first universal schema with a second specialized plug-in, wherein the second specialized plug-in defines how the one or more universal data elements are organized in the second specialized data store;

determining that the first request is not a read request and in response;

transforming the first request using the second specialized plug-in associated with the first universal schema to produce a second specialized query;

sending the second specialized query to the second specialized data store to execute the second specialized query.

26. The one or more non-transitory computer-readable media of claim 15 comprising sequences of instructions which when executed cause:

associating the first universal schema with a second specialized plug-in;

transforming the first request using the first specialized plug-in associated with the first universal schema to produce a first specialized query to retrieve a first set of results from the first specialized data store;

wherein the first specialized query comprises one or more instructions that are compatible with the first specialized data store;

transforming the first request using the second specialized plug-in associated with the first universal schema to produce a second specialized query to retrieve a second set of results from a second specialized data store;

wherein the second specialized query comprises one or more instructions that are compatible with the second specialized data store;

wherein the second specialized query is different than the first specialized query;

receiving the first specialized result set from the first specialized data store based on the first specialized query;

receiving a second specialized result set from the second specialized data store based on the second specialized query;

transforming the first specialized result set to a first universal result set, wherein the first universal result set is based on the first universal schema;

transforming the second specialized result set to a second universal result set, wherein the second universal result set is based on the first universal schema;

joining the first universal result set and the second universal result set to produce the result set, which is a universal result set that is based on the first universal schema.

27. The one or more non-transitory computer-readable media of claim 15 comprising sequences of instructions which when executed cause associating the first universal schema with the first specialized plug-in, which defines one or more methods for transforming a universal request into one or more specialized queries that create, read, update, and delete one or more data items in the first specialized data store.

28. One or more non-transitory computer-readable media storing one or more sequences of instructions which, when executed by one or more computing devices, cause:

receiving an input defining organization of one or more universal data elements;

generating a universal schema that defines organization of the one or more universal data elements based on the input;

associating the universal schema with a first specialized plug-in, wherein the first specialized plug-in defines how the one or more universal data elements are organized in a first specialized data store;

receiving a command to copy data in the first specialized data store to a second specialized data store;

associating the universal schema with a second specialized plug-in, wherein the second specialized plug-in defines organization of the one or more universal data elements in the second specialized data store;

generating a first request to receive a set of data from the first specialized data store;

transforming the first request using the first specialized plug-in to produce a first specialized query;

sending the first specialized query to the first specialized data store to execute the first specialized query;

receiving a specialized result set from the first specialized data store;

transforming the specialized result set using the first specialized plug-in to produce a universal result set, wherein the universal result set is formatted based on the universal schema;

generating a second specialized query, using the second specialized plug-in, to store the universal result set in the second specialized data store;

sending the second specialized query to the second specialized data store.