System and method for reading and writing to big data storage formats

• US 10,169,601 B2
• Filed: 11/18/2015
• Issued: 01/01/2019
• Est. Priority Date: 11/18/2015
• Status: Active Grant

• Current Assignee: American Express Travel Related Services Company, Inc. (American Express Company)
• Original Assignee: American Express Travel Related Services Company, Inc. (American Express Company)
• Inventors: Arasan, Ravi, Arguello, Carmen Patricia, Bose, Sandeep, Meyer, Matthew Kent, Ponnambalam, Marimuthu, Ramasamy, Gurusamy, Seideman, Jeremy D., Torkzahrani, Mohammad, Varier, Roopesh R.
• Primary Examiner(s): Lewis, Cheryl
• Application Number:
  US14/944,979
• Publication Number:
  US 20170139997A1
• Time in Patent Office: 1,140 Days
• CPC Class Codes: G06F 16/24534 Query rewriting; Transformation
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06F 21/6227 where protection concerns the structure of data, e.g. records, types, queries

Abstract

A system may receive a request for access to a first variable. The request may include a requested action and a variable identifier for the first variable. The request may also have a syntax that is incompatible with first data storage format. The system may parse the request to capture the variable identifier from the request. The system may also look up a location of the first variable in a catalog using the variable identifier. The location of the first variable may include the first data storage format. The system may generate a query to access the first variable. The syntax of the query may be compatible with the first data storage format. The system may then submit the query to the first data storage format. The query may be configured to complete the requested action.

Claims

1. A method comprising:

receiving, by a processor, a request at a single, centralized access point created by an access interface for access to a first variable in a single logical data structure as viewed from an application side, wherein the request comprises a requested action and a variable identifier for the first variable, wherein the request has a first syntax, wherein a first data storage format is incompatible with the first syntax;

parsing, by the processor, the request to capture the variable identifier from the request;

looking up, by the processor, the first variable in a catalog using the variable identifier, wherein the first variable includes a location of the first variable and the first data storage format that maintains the first variable and the variable identifier within the first data storage format;

generating, by the processor, a query to access the first variable, wherein the query comprises a second syntax compatible with the first data storage format; and

submitting, by the processor, the query to access the first data storage format, wherein the query is configured to complete the requested action.

2. The method of claim 1, further including checking, by the processor, an access permission for the first variable in the catalog, wherein the access permission includes at least one of read access, write access or authorization to view information contained in the first variable.

3. The method of claim 1, wherein the variable identifier includes at least one of a variable name, a preferred data storage format, a time, a requester username, a requester user group, or an application ID.

4. The method of claim 1, further comprising formatting, by the processor, a result of the requested action into a requested result format, wherein the result comprises data from the first data storage format.

5. The method of claim 1, further comprising generating, by the processor, a log entry corresponding to the request, wherein the log entry comprises at least one of a user identifier or an application identifier.

6. The method of claim 1, further comprising:

receiving, by the processor, the request for access to a second variable stored in a second data storage format, wherein the second data storage format is different from the first data storage format; and

returning, by the processor, a result of the requested action, wherein the result comprises the first variable from the first data storage format and the second variable from the second data storage format.

7. The method of claim 1, further comprising writing, by the processor, data to the first variable in the first data storage format in response to the request.

8. The method of claim 1, wherein the first variable is associated with an access permission, wherein the access permission is cataloged at at least one of a file level, table level, variable level or data type level.

9. The method of claim 1, wherein the completing the requested action includes providing the first variable in the first data storage format, while the first variable is stored in a second data storage format.

10. A computer-based system, comprising:

a processor; and

a tangible, non-transitory memory configured to communicate with the processor, the tangible, non-transitory memory having instructions stored thereon that, in response to execution by the processor, cause an access interface system to perform operations comprising;

receiving, by the processor, a request at a single, centralized access point created by an access interface for access to a first variable in a single logical data structure as viewed from an application side, wherein the request comprises a requested action and a variable identifier for the first variable, wherein the request has a first syntax, wherein a first data storage format is incompatible with the first syntax;

parsing, by the processor, the request to capture the variable identifier from the request;

looking up, by the processor, the first variable in a catalog using the variable identifier, wherein the first variable includes a location of the first variable and the first data storage format that maintains the first variable and the variable identifier within the first data storage format;

generating, by the processor, a query to access the first variable, wherein the query comprises a second syntax compatible with the first data storage format; and

submitting, by the processor, the query to access the first data storage format, wherein the query is configured to complete the requested action.

11. The computer-based system of claim 10, further comprising formatting, by the processor, a result of the requested action into a requested result format, wherein the result comprises data from the first data storage format.

12. The computer-based system of claim 10, further comprising generating, by the processor, a log entry corresponding to the request, wherein the log entry comprises at least one of a user identifier or an application identifier.

13. The computer-based system of claim 10, further comprising:

receiving, by the processor, the request for access to a second variable stored in a second data storage format, wherein the second data storage format is different from the first data storage format; and

returning, by the processor, a result of the requested action, wherein the result comprises the first variable from the first data storage format and the second variable from the second data storage format.

14. The computer-based system of claim 10, further comprising writing, by the processor, data to the first variable in the first data storage format in response to the request.

15. An article of manufacture including a non-transitory, tangible computer readable storage medium having instructions stored thereon that, in response to execution by a processor, cause the processor to perform operations comprising:

receiving, by the processor, a request at a single, centralized access point created by an access interface for access to a first variable in a single logical data structure as viewed from an application side, wherein the request comprises a requested action and a variable identifier for the first variable, wherein the request has a first syntax, wherein a first data storage format is incompatible with the first syntax;

parsing, by the processor, the request to capture the variable identifier from the request;

looking up, by the processor, the first variable in a catalog using the variable identifier, wherein the first variable includes a location of the first variable and the first data storage format that maintains the first variable and the variable identifier within the first data storage format;

generating, by the processor, a query to access the first variable, wherein the query comprises a second syntax compatible with the first data storage format; and

submitting, by the processor, the query to access the first data storage format, wherein the query is configured to complete the requested action.

16. The article of claim 15, further including:

checking, by the processor, an access permission for the first variable; and

denying, by the processor, the request in response to the access permission.

17. The article of claim 16, wherein the access permission indicates that the request does not have permission to access the first variable.

18. The article of claim 15, further comprising formatting, by the processor, a result of the requested action into a requested result format, wherein the result comprises data from the first data storage format.

19. The article of claim 15, further comprising generating, by the processor, a log entry corresponding to the request, wherein the log entry comprises at least one of a user identifier or an application identifier.

20. The article of claim 15, further comprising writing, by the processor, data to the first variable in the first data storage format in response to the request.