Yarn rest API protection

• US 10,291,602 B1
• Filed: 04/12/2017
• Issued: 05/14/2019
• Est. Priority Date: 04/12/2017
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: BlueTalon, Inc. (Microsoft Corporation)
• Inventors: Sailappan, Sridhar Shanmugam, Arumugam, Dilli Dorai Minnal
• Primary Examiner(s): Jamshidi, Ghodrat
• Application Number:
  US15/486,259
• Time in Patent Office: 762 Days
• CPC Class Codes: H04L 47/82 Miscellaneous aspects
• CPC Class Codes: H04L 63/0281 Proxies
• CPC Class Codes: H04L 63/08 for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32
• CPC Class Codes: H04L 63/10 for controlling access to devices or network resources
• CPC Class Codes: H04L 63/205 involving negotiation or determination of the one or more network security mechanisms to be used, e.g. by negotiation between the client and the server or between peers or by selection according to the capabilities of the entities involved negotiation of communication capabilities H04L69/24
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54

Abstract

Systems, computer program products and methods implementing YARN service protection are described. A reverse proxy in a cluster of computers in a distributed computing system can intercept a request to access a YARN service. The request can be associated with requester credentials. The reverse proxy determines that the request includes a REST API call. The reverse proxy determines, based on authentication configuration information, that the call needs to be authenticated. The reverse proxy authenticates the call based on the requester credentials using an authentication mechanism specified in the configuration information. Upon successful authentication of the call, the reverse proxy makes authorization checks based on specified configuration information. If the authorization checks pass, the reverse proxy forwards the request to a server that provides the YARN service in the cluster. If the authentication or authorization checks fail, the reverse proxy denies the request.

Claims

1. A method, comprising:

intercepting, by a reverse proxy in a cluster of computers in a distributed computing system, a request to access a resource manager service of the distributed computing system;

determining, by the reverse proxy, that the request includes a call that conforms to a REST (Representational State Transfer) API (application programming interface);

determining, by the reverse proxy, that the call has a particular method type that indicates requesting an action to be performed;

determining, by the reverse proxy based on authentication configuration information, whether the call needs to be authenticated, wherein the authentication configuration information includes a list of methods that need to be authenticated and determining whether the call needs to be authenticated comprises determining whether the particular method of the call is included in the list of methods of the authentication configuration information; and

in response to determining that the call needs to be authenticated;

authenticating the call using an authentication mechanism specified in the authentication configuration information,

upon successful authentication of the call, performing authorization checks based on the configuration information, and

upon successful authorization of the call, forwarding the request from the reverse proxy to a server that provides the resource manager service in the cluster,

wherein the reverse proxy includes one or more computer processors.

2. The method of claim 1, wherein the authentication configuration information is stored in a configuration file or a database table, the cluster is a Hadoop cluster, and the resource manager service is a YARN (Yet Another Resource Negotiator) resource manager service.

3. The method of claim 2, wherein the configuration file or database table has a first logical column listing YARN REST API calls and a second logical column storing a respective flag indicating weather each YARN REST API call requires authentication.

4. The method of claim 1, wherein authenticating the call comprises submitting a representation of caller credentials associated with the call by the reverse proxy to a user authentication service and receiving a response indicating that the caller credentials are authenticated by the user authentication service.

5. The method of claim 1, comprising:

receiving, by the reverse proxy from the server providing the resource manager service, a response to the request;

filtering the response based on the authentication and authorization configuration information, including blocking at least a portion of content in the response based on one or more parameters in the configuration information; and

forwarding the filtered response by the reverse proxy to a client device that submitted the request.

6. The method of claim 5, wherein the request is an HTTP (hypertext transfer protocol) request and the response is a JSON (JavaScript Object Notation) object or an XML (Extensible Markup Language) document.

7. The method of claim 1, wherein the authentication configuration information includes an address and port of the server and an address and port of the reverse proxy.

8. The method of claim 1, wherein the list of methods includes a plurality of methods associated with one or more HTTP or HTTPS calls.

9. The method of claim 8, further comprises:

determining that the particular method of the call is not included in the list of methods of the authentication configuration information, and in response;

forwarding the request from the reverse proxy to a server that provides the resource manager service in the cluster without authenticating the call.

10. A non-transitory computer readable storage medium storing instructions executable by a data processing apparatus and upon such execution cause the data processing apparatus to perform operations comprising:

intercepting, by a reverse proxy in a cluster of computers in a distributed computing system, a request to access a resource manager service of the distributed computing system;

determining, by the reverse proxy, that the request includes a call that conforms to a REST (Representational State Transfer) API (application programming interface);

determining, by the reverse proxy, that the call has a particular method type that indicates requesting an action to be performed;

determining, by the reverse proxy based on authentication configuration information, whether the call needs to be authenticated, wherein the authentication configuration information includes a list of methods that need to be authenticated and determining whether the call needs to be authenticated comprises determining whether the particular method of the call is included in the list of methods of the authentication configuration information;

in response to determining that the call needs to be authenticated, performing actions including;

authenticating the call using an authentication mechanism specified in the authentication configuration information;

upon successful authentication of the call, performing authorization checks based on the configuration information; and

upon successful authorization of the call, forwarding the request from the reverse proxy to a server that provides the resource manager service in the cluster; and

in response to determining that the call does not need to be authenticated, forwarding the request from the reverse proxy to the server that provides the resource manager service.

11. The non-transitory computer readable storage medium of claim 10, wherein the authentication configuration information is stored in a configuration file or a database table, the cluster is a Hadoop cluster, and the resource manager service is a YARN (Yet Another Resource Negotiator) resource manager service.

12. The non-transitory computer readable storage medium of claim 11, wherein the configuration file or database table has a first logical column listing YARN REST API calls and a second logical column storing a respective flag indicating weather each YARN REST API call requires authentication.

13. The non-transitory computer readable storage medium of claim 10, wherein authenticating the call comprises submitting a representation of caller credentials associated with the call by the reverse proxy to a user authentication service and receiving a response indicating that the caller credentials are authenticated by the user authentication service.

14. The non-transitory computer readable storage medium of claim 10, the operations comprising:

receiving, by the reverse proxy from the server providing the resource manager service, a response to the request;

filtering the response based on the authentication and authorization configuration information, including blocking at least a portion of content in the response based on one or more parameters in the configuration information; and

forwarding the filtered response by the reverse proxy to a client device that submitted the request.

15. The non-transitory computer readable storage medium of claim 14, wherein the request is an HTTP (hypertext transfer protocol) request and the response is a JSON (JavaScript Object Notation) object or an XML (Extensible Markup Language) document.

16. The non-transitory computer readable storage medium of claim 10, wherein the authentication configuration information includes an address and port of the server and an address and port of the reverse proxy.

17. A system comprising:

one or more computers and one or more storage devices on which are stored instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

intercepting, by a reverse proxy in a cluster of computers in a distributed computing system, a request to access a resource manager service of the distributed computing system;

determining, by the reverse proxy, that the request includes a call that conforms to a REST (Representational State Transfer) API (application programming interface);

determining, by the reverse proxy, that the call has a particular method type that indicates requesting an action to be performed;

determining, by the reverse proxy based on authentication configuration information, whether the call needs to be authenticated, wherein the authentication configuration information includes a list of methods that need to be authenticated and determining whether the call needs to be authenticated comprises determining whether the particular method of the call is included in the list of methods of the authentication configuration information;

in response to determining that the call needs to be authenticated, performing actions including;

authenticating the call using an authentication mechanism specified in the authentication configuration information;

upon successful authentication of the call, performing authorization checks based on the configuration information; and

upon successful authorization of the call, forwarding the request from the reverse proxy to a server that provides the resource manager service in the cluster; and

in response to determining that the call does not need to be authenticated, forwarding the request from the reverse proxy to the server that provides the resource manager service.

18. The system of claim 17, wherein the authentication configuration information is stored in a configuration file or a database table, the cluster is a Hadoop cluster, and the resource manager service is a YARN (Yet Another Resource Negotiator) resource manager service.

19. The system of claim 18, wherein the configuration file or database table has a first logical column listing YARN REST API calls and a second logical column storing a respective flag indicating weather each YARN REST API call requires authentication.

20. The system of claim 17, wherein authenticating the call comprises submitting a representation of caller credentials associated with the call by the reverse proxy to a user authentication service and receiving a response indicating that the caller credentials are authenticated by the user authentication service.

21. The system of claim 17, the operations comprising:

receiving, by the reverse proxy from the server providing the resource manager service, a response to the request;

filtering the response based on the authentication and authorization configuration information, including blocking at least a portion of content in the response based on one or more parameters in the configuration information; and

forwarding the filtered response by the reverse proxy to a client device that submitted the request.

22. The system of claim 17, wherein the authentication configuration information includes an address and port of the server and an address and port of the reverse proxy.