Network-attached storage devices

• US 20080288498A1
• Filed: 05/14/2008
• Published: 11/20/2008
• Est. Priority Date: 05/14/2007
• Status: Abandoned Application

• Current Assignee: Dataupia Corp.
• Original Assignee: Dataupia Corp.
• Inventors: Hinshaw, Foster D., Chiu, John, Bingham, Timothy J., Potter, Alan, Harris, Craig S., Chen, Yongmin
• Application Number:
  US12/152,420
• Publication Number:
  US 20080288498A1
• Time in Patent Office: Days
• US Class Current: 707/8
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Systems and methods for managing data within a networked computer environment including application servers and a plurality of storage devices utilize a collection of service modules, each separately addressable from and in communication with the application servers and the storage devices and providing system-wide services to the application servers and the storage devices.

Claims

1. A system for managing data within a networked computer environment, the system comprising:

one or more application servers, each capable of running one or more applications thereon;

a plurality of storage devices each in communication with at least one of the application servers and comprising a plurality of physical storage disks for storing data supportive of the applications; and

a plurality of service modules, separately addressable from and in communication with the application servers and the storage devices, each service module providing system-wide services to each of the application servers and each of the storage devices.

2. The system of claim 1 wherein the application servers comprise database management host servers.

3. The system of claim 2 wherein the application servers are embedded within adapters between the database management host servers and the plurality of storage devices.

4. The system of claim 1 wherein the storage devices comprise network-attached storage devices.

5. The system of claim 1 wherein the service modules comprise one or more of a transaction services module, a root services module, a blade services module, a lock manager services module, a catalog service module and a database services module.

6. The system of claim 5 wherein the application servers communicate with the service modules via an application programming interface.

7. The system of claim 5 wherein the transaction services module provides system-wide unique transaction ids for transactions being processed by the system.

8. The system of claim 5 wherein the locking services module provides locking of data records stored on the storage devices.

9. The system of claim 8 wherein locking comprises one or more of read-only locking, write-locking, and transaction locking.

10. The system of claim 5 wherein the root services module provides information to the application server comprising the physical and logical location of each of the service modules.

11. The system of claim 10 wherein the physical and logical location of each of the service modules is transmitted to the application server in response to a User Datagram Protocol broadcast.

12. The system of claim 5 wherein the blade services module provides information to the application server comprising addressing information for the storage devices.

13. The system of claim 12 wherein the blade service module further provides information to application server comprising instances of databases attributed to each of the storage devices and whether the instances are operational.

14. The system of claim 12 wherein the addressing information comprises one or more of an IP address, port number, host id and a connection string.

15. The system of claim 5 wherein the blade services module provides information to the application server comprising information regarding the health of the storage devices

16. The system of claim 5 wherein the database services module provides information to the application server comprising catalog information.

17. The system of claim 16 wherein the catalog information comprises a mapping of logical data locations to physical locations on the storage devices.

18. The system of claim 17 wherein the mapping comprises (i) a mapping of tables to the one or more storage devices on which data allocated to the tables are physically stored, and (ii) a description of how the data is allocated among the one or more storage devices.

19. The system of claim 1 further comprising an application-independent adapter residing on the application server and facilitating the transmission of data transaction messages directly to the storage devices.

20. The system of claim 19 further comprising an application-specific adapter for facilitating messaging between the applications and the application-independent adapter.

21. The system of claim 20 wherein the application-independent adapter is configured to combine transaction results received from each of the storage devices.

22. The system of claim 1 wherein each storage device further comprises query transaction services for processing queries submitted by the application server, the query transaction services comprising query parsing services, query planning services, query execution services and query result aggregation services.

23. The system of claim 22 wherein the queries comprise one or more of insert transactions, update transactions, delete transactions and retrieval transactions.

24. The system of claim 1 further comprising a separately-addressable transaction recording module for receiving messages from each of the storage devices requesting the creation of a persistent record indicating the completion of a transaction at the storage devices.

25. A method for executing a transaction in a data processing system, the method comprising:

receiving, at a centralized transaction service module, a request from an application to perform a transaction involving data stored on one or more of a plurality of storage devices;

associating a unique transaction ID to the request;

establishing direct connections between the application and each of a plurality of storage devices and transmitting the request to perform the transaction thereon;

executing the transaction at one or more of the storage devices;

transmitting results of the transaction to the application;

transmitting a prepare transaction message from the application to each of the storage devices;

creating durable records based on the results of the transaction at each storage device;

transmitting a confirmation message from the storage devices to the application indicating the creation of the durable records;

transmitting the unique transaction ID from the application to a global transaction recording service, indicating the transaction has been completed at each storage device; and

creating a persistent record at the global transaction recording service indicating the transaction has been committed.

26. The method of claim 25 wherein the transaction comprises one of a record insert transaction, a record update transaction, a record delete transaction or a record retrieve transaction.

27. The method of claim 25 wherein the unique transaction IDs are retrieved from a central data store.

28. The method of claim 25 wherein the durable records are created in non-volatile storage.

29. The method of claim 25 further comprising transmitting instructions from the application directly to each of the storage devices to release resources previously allocated to the transaction.

30. A system for managing data within a networked computer environment, the system comprising:

one or more application servers each capable of running one or more applications thereon;

a plurality of storage devices each in communication with at least one of the application servers and comprising a plurality of physical storage disks for storing data supportive of the applications;

an application-independent adapter residing on the application server and facilitating the transmission of data transaction messages directly to the storage devices; and

an application-specific adapter for facilitating messaging between the applications and the application-independent adapter.

31. The system of claim 30 wherein the applications comprise database management systems.

32. The system of claim 30 wherein the storage devices comprise network attached storage devices.

33. The system of claim 30 wherein the applications use different syntaxes for transmitting the data transaction messages to the storage devices, and the application-independent adapter translates the different syntaxes into a common message syntax, thereby allowing the system to service data requests from heterogeneous applications using common data storage devices.

34. The system of claim 30 wherein the application-independent adapter is configured to parse data transaction messages into message components.

35. The system of claim 34 wherein the message components comprise a where clause from a structured query language query identifying a subset of the storage devices to which the data transaction commands are to be sent.

36. The system of claim 34 wherein the message components comprise a from clause from a structured query language query identifying a subset of tables from which to select data records, thereby facilitating the transmission of the data transaction command to the storage devices on which the subset of tables resides.

37. The system of claim 30 wherein the application-independent adapter is configured to determine a distribution of data among the storage devices, thereby facilitating the delivery of data transaction messages to a storage device having data related to the data transaction message.

38. The system of claim 30 wherein the application-independent adapter is configured to combine transaction results received from each of the storage devices.

39. The system of claim 30 further comprising a plurality of service modules, separately addressable from and in communication with the application-independent adapter, each service module providing system-wide services to each of the application servers via the application-independent adapter.

40. A method for migrating data from a symmetric multiprocessing data store to a massively parallel processing data store, the method comprising the steps of:

defining a view within the symmetric multiprocessing data store wherein the view refers to a table within the massively parallel processing data store; and

issuing a single data transaction message against the symmetric multiprocessing data store whereby execution of the data transaction message selects records from the symmetric multiprocessing data store and inserts the selected records into a table within the massively parallel processing data store.

41. The method of claim 40 wherein the massively parallel processing data store comprises a plurality of network attached storage devices.

42. The method of claim 41 wherein the table is distributed among more than one of the plurality of network attached storage devices.

43. For use with a system for managing data within a networked computer environment, the system comprising one or more application servers, each capable of running one or more applications thereon, and a plurality of storage devices each in communication with at least one of the application servers and comprising a plurality of physical storage disks for storing data supportive of the applications, a plurality of service modules each separately addressable from and in communication with the application servers and the storage devices, each service module comprising means for providing system-wide services to each of the application servers and each of the storage devices.

44. The service modules of claim 43 wherein the application servers are embedded within adapters between the database management host servers and the plurality of storage devices.

45. The service modules of claim 43 wherein the service modules comprise one or more of a transaction services module, a root services module, a blade services module, a lock manager services module, a catalog service module and a database services module.

46. The service modules of claim 43 further comprising an application-independent adapter residing on the application server and facilitating the transmission of data transaction messages directly to the storage devices.

47. The service modules of claim 46 further comprising an application-specific adapter for facilitating messaging between the applications and the application-independent adapter.

48. The service modules of claim 43 wherein each storage device further comprises query transaction services for processing queries submitted by the application server, the query transaction services comprising query parsing services, query planning services, query execution services and query result aggregation services.

49. The service modules of claim 48 wherein the queries comprise one or more of insert transactions, update transactions, delete transactions and retrieval transactions.

50. The service modules of claim 43 further comprising a separately-addressable transaction recording module for receiving messages from each of the storage devices requesting the creation of a persistent record indicating the completion of a transaction at the storage devices.

51. For use with a system for managing data within a networked computer environment, the system comprising one or more application servers, each capable of running one or more applications thereon, and a plurality of storage devices each in communication with at least one of the application servers and comprising a plurality of physical storage disks for storing data supportive of the applications:

an application-independent adapter residing on the application server and facilitating the transmission of data transaction messages directly to the storage devices; and

an application-specific adapter for facilitating messaging between the applications and the application-independent adapter.

52. The adapters of claim 51 wherein the application-independent adapter is configured to determine a distribution of data among the storage devices, thereby facilitating the delivery of data transaction messages to a storage device having data related to the data transaction message.

53. The adapters of claim 51 wherein the application-independent adapter is configured to combine transaction results received from each of the storage devices.

54. The adapters of claim 51 further comprising a plurality of service modules, separately addressable from and in communication with the application-independent adapter, each service module providing system-wide services to each of the application servers via the application-independent adapter.