System and method for storing and processing database requests

• US 10,025,873 B2
• Filed: 04/17/2015
• Issued: 07/17/2018
• Est. Priority Date: 04/18/2014
• Status: Active Grant

• Current Assignee: Walmart Apollo, LLC (WalMart Inc.)
• Original Assignee: Walmart Apollo, LLC (WalMart Inc.)
• Inventors: Jackson, Richard, Frerking, John R.
• Primary Examiner(s): Pham, Tuan A
• Application Number:
  US14/689,493
• Publication Number:
  US 20150302037A1
• Time in Patent Office: 1,187 Days
• CPC Class Codes: G06F 16/972 Access to data in other repository systems, e.g. legacy data or dynamic Web page generation
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]

Abstract

A system is provided that is adapted to service web-based database service requests. In one implementation, a database service is provided for servicing web-originated service requests. In one implementation, virtual computer systems may be used to service requests in a more reliable manner. Different operating modes may be configured for backup redundancy and the database service may be scaled to meet service requests for a particular application. Also, methods are provided for exchanging timestamp information among web service transaction systems to reduce the amount of processing capability and bandwidth for ensuring database consistency. Further, a NoSQL Key/Value (KVS) database is provided that provides one or more features that may be beneficial for cloud service, Internet-based, and other types of database applications.

Claims

1. A system for processing database requests, the system comprising:

a server including a processor, configured to receive and process database requests, the server comprising a plurality of components comprising;

a plurality of virtual computer systems adapted to receive database requests including full keys or generic keys;

a logical storage system coupled to the plurality of virtual computer systems, wherein each of the plurality of virtual computer systems shares a common storage that is adapted to store the received database requests;

a request handler element adapted to distribute a first database request to at least one of the plurality of virtual computer systems for processing;

a component of the plurality of components adapted to set record retention at a row level,

wherein the component adapted to set record retention at a row level is responsive to a specification of a time to live value within the first database request,

wherein the at least one of the plurality of virtual computing systems is configured to (i) receive a Uniform Resource Identifiers (URI) associated with the first database request, including a generic key, (ii) retrieve one or more records associated with the generic key, (iii) in response to retrieving a plurality of records associated with the generic key, receive an HTTP Status Text including a full key corresponding to at least one record of the plurality of records stored in the logical storage system in response to processing the first database request, (iv) move the full key from the HTTP Status Text to the URI to access the record using the full key included in the HTTP Status Text,

wherein the at least one of the plurality of virtual computing systems is configured to receive a second database request, determine the second database request is of a certain type, asynchronously process the second database request, write results of processing the second database requests to a file and transmit a response indicating the file name.

2. The system according to claim 1, wherein the database requests include at least one of a group comprising:

a representational state transfer (REST) service request; and

a Simple Object Access Protocol (SOAP) service request.

3. The system according to claim 1, wherein the database requests include at least one of a group of requests comprising:

a POST request;

a GET request;

a PUT request; and

a DELETE request.

4. The system according to claim 3, wherein the database is accessed in a read-only mode wherein the GET request is accessed using HTTP without a username/password information.

5. The system according to claim 3, wherein the database is accessed responsive to the database requests using HTTPS and Secure Sockets Layer/Transport Layer Security (SSL/TLS).

6. The system according to claim 1, wherein the server configured to receive and process database requests is operable in at least one of a group of operating modes, the group comprising:

an active/single mode;

an active/standby mode; and

an active/active mode.

7. The system according to claim 1, wherein the server configured to receive and process database requests is operable in at least one of a group of operating modes, the group comprising:

a basic mode; and

a query mode.

8. The system according to claim 7, wherein in the basic mode, access to a row of information in the database is performed using a full key value.

9. The system according to claim 7, wherein in the basic mode, a table is browsed sequentially using a full or generic key value.

10. The system according to claim 7, further comprising a component configured to return a set of database fields from the database responsive to a view indicator defining the set of database fields.

11. The system according to claim 10, wherein the view information is specified within at least one of the plurality of received database requests.

12. The system according to claim 7, further comprising a component configured to set record retention at a row level of the database.

13. The system according to claim 12, wherein record retention information is specified within at least one of the plurality of received database requests.

14. The system according to claim 12, wherein record retention information is specified within at least one of the plurality of received database requests.

15. The system according to claim 7, further comprising a component configured to set security at a field level of the database.

16. The system according to claim 15, wherein the database includes a data structure specifying one or more users having field level access to information stored in the database.

17. The system according to claim 16, wherein the database is adapted to control access to the database responsive to the received database requests at a field level of the database based on one or more user identifiers.

18. The system according to claim 7, further comprising a component configured to perform a query of the database in an offload processing mode responsive to an indicator defining the offload processing mode.

19. The system according to claim 7, further comprising a component responsive to a database request that does not specify a table name.

20. The system according to claim 7, further comprising a component responsive to a database request that joins information from a specified table.

21. The system according to claim 1, wherein the database request includes a request type identifier.

22. The system according to claim 21, wherein the request type identifier includes at least one of a group of request type identifiers, the group comprising:

an indicator that specifies that the database request includes a precise key; and

an indicator that specifies that the database request includes a generic key.

23. The system according to claim 22, wherein the indicator that specifies that the database request includes the generic key also indicates that database records retrieved from the database by generating a query including at least one operator of a group of operators and the generic key, the group of operators comprising:

a greater than or equal to relationship operator;

a greater than relationship operator;

a less than or equal to relationship operator; and

a less than relationship operator.

24. The system according to claim 1, further comprising a component responsive to a trigger event that defines a database operation.

25. The system according to claim 24, wherein the component responsive to the trigger event may perform at least one of a group comprising a synchronization of data with another database service, a logging of data to a logging service, a receipt of information to the database, and a publication of data to an external destination.

26. The system according to claim 24, wherein the trigger event may be specified as a synchronous or an asynchronous operation.

27. The system according to claim 1, wherein the database is a NoSQL key/value store database.

28. The system according to claim 1, further comprising a load balancing element adapted to distribute the database requests among a plurality of server systems.

29. The system according to claim 1, wherein the plurality of virtual computer systems are located within a partition.

30. The system according to claim 29, wherein a database application is assigned to a particular partition.

31. The system according to claim 1, further comprising transaction server components that are adapted to process database transactions.

32. The system according to claim 1, wherein the logical storage system further comprises a common database shared by the plurality of virtual servers upon which database requests are transacted.

33. The system according to claim 1, further comprising an entity that monitors an expiration of a database record associated with at least one database request.

34. The system according to claim 33, further comprising corresponding entities that execute among at least two of the plurality of virtual computer systems, the entities being adapted to compare timestamps associated with the database record associated with the at least one database request.

35. The system according to claim 34, wherein the at least two of the plurality of virtual computer systems execute within separate computer systems.

36. The system according to claim 33, wherein the entity is adapted to delete the database record associated with the at least one database request.

37. The system according to claim 1, wherein at least two of the virtual computer systems are located in different data centers.

38. A method for processing database requests, the method comprising out of:

receiving, by a load sharing entity, a plurality of database requests including full keys or generic keys, from one or more client systems;

storing, in a common storage location, the received plurality of database requests;

assigning at least one virtual computer system to process at least a first one of the plurality of database requests stored in the common storage location;

setting record retention at a row level;

including a time to live value within the at least first one of the database requests;

receiving, via the at least one virtual computer system, a Uniform Resource Identifiers (URI) associated with the first one of the database requests, including a generic key;

retrieving, via the at least one virtual computer system, a plurality of records associated with the generic key;

in response to retrieving the plurality of records associated with the generic key, receiving, via the at least one virtual computer system, an HTTP Status Text including a full key corresponding to at least one record of the plurality of records stored in the common storage system in response to processing the at least first one of the database requests,

moving, via the at least one virtual computer system, the full key from the HTTP Status Text to the URI to access the at least one record using the full key included in the HTTP Status Text;

providing, via the at least one virtual computer system, a first response including the at least one record, to the one or more client systems that generated the at least first one of the plurality of database requests;

receiving, via the at least one virtual computer system, an at least second database request;

determining, via the at least one virtual computer system, the at least second database request is of a certain type;

asynchronously processing, via the at least one virtual computer system, the at least second database request;

writing, via the at least one virtual computer system, results of processing the at least second, via the at least one virtual computer system, database requests to a file; and

transmitting, via the at least one virtual computer system, a second response indicating the file name the one or more client systems that generated the at least second one of the plurality of database requests.

39. The method according to claim 38, wherein the at least one of the plurality of database requests includes at least one of a group comprising:

a representational state transfer (REST) service request; and

a Simple Object Access Protocol (SOAP) service request.

40. The method according to claim 38, wherein the database requests include at least one of a group of requests comprising:

a POST request;

a GET request;

a PUT request; and

a DELETE request.

41. The method according to claim 40, further comprising an act of accessing the database in a read-only mode wherein the GET request is accessed using HTTP without a username/password information.

42. The method according to claim 40, further comprising an act of accessing the database responsive to receiving database requests using HTTPS and Secure Sockets Layer/Transport Layer Security (SSL/TLS).

43. The method according to claim 38, further comprising an act of operating the server configured to receive and process database requests in at least one of a group of operating modes, the group comprising:

an active/single mode;

an active/standby mode; and

an active/active mode.

44. The method according to claim 38, further comprising an act of operating the server to receive and process database requests, the server being operable in at least one of a group of operating modes, the group comprising:

a basic mode; and

a query mode.

45. The system according to claim 44, wherein in the basic mode, access to a row of information in the database is performed using a full key value.

46. The method according to claim 44, wherein in the basic mode, browsing a table sequentially using a full or generic key value.

47. The method according to claim 44, further comprising an act of returning a set of database fields from the database responsive to a view indicator defining the set of database fields.

48. The method according to claim 47, wherein the view information is specified within at least one of the plurality of received database requests.

49. The method according to claim 44, further comprising an act of setting record retention at a row level of the database.

50. The method according to claim 49, wherein record retention information is specified within at least one of the plurality of received database requests.

51. The method according to claim 49, further comprising an act of setting record retention responsive to received record retention information within at least one of the plurality of received database requests.

52. The method according to claim 44, further comprising an act of setting security at a field level of the database.

53. The method according to claim 52, wherein the database includes a data structure specifying one or more users having field level access to information stored in the database.

54. The method according to claim 53, wherein the database is adapted to control access to the database responsive to the received database requests at a field level of the database based on one or more user identifiers.

55. The method according to claim 44, further comprising an act of performing a query of the database in an offload processing mode responsive to an indicator defining the offload processing mode.

56. The method according to claim 44, further comprising an act of responding to a database request that does not specify a table name.

57. The method according to claim 44, further comprising an act of responding to a database request that joins information from a specified table.

58. The method according to claim 38, wherein the database request includes a request type identifier.

59. The method according to claim 58, wherein the request type identifier includes at least one of a group of request type identifiers, the group comprising:

an indicator that specifies that the database request includes a precise key; and

an indicator that specifies that the database request includes a generic key.

60. The method according to claim 59, wherein the indicator that specifies that the database request includes the generic key also indicates that database records retrieved from the database are to be retrieved by generating a query including at least one operator of a group of operators specified in the database request and the generic key, the group of operators comprising:

a greater than or equal to relationship operator;

a greater than relationship operator;

a less than or equal to relationship operator; and

a less than relationship operator.

61. The method according to claim 38, further comprising an act of responding to a trigger event that defines a database operation.

62. The method according to claim 61, further comprising an act of performing, responsive to the trigger event, at least one of a group comprising a synchronization of data with another database service, a logging of data to a logging service, a receipt of information to the database, and a publication of data to an external destination.

63. The method according to claim 61, wherein the trigger event may be specified as a synchronous or an asynchronous operation.

64. The method according to claim 38, wherein the database is a NoSQL key/value store database.

65. The method according to claim 38, further comprising an act of distributing the plurality of database requests among a group of virtual computer systems.

66. The method according to claim 65, wherein the group of virtual computer systems are located within a partition.

67. The method according to claim 66, further comprising an act of assigning a database application to the partition.

68. The method according to claim 66, further comprising an act of sharing, by the group of virtual computer systems, the common storage location that stores the plurality of database requests.