Reply
New Contributor
Posts: 3
Registered: ‎12-12-2018

securing yarn logs

There's no control in place to ensure access to the job tracker portal (YARN logs). The URL is open to all who have knowledge of it.  I am looking for a way to secure these logs URLS (YARN application history, job history, Spark history).  what is best way to go about locking down these URLs to sppecific groups or to force some kind of authentication (provde login credentials) and not have them open to all who is aware of them....

Any thoughts or suggestions of best way to do this?

Posts: 998
Topics: 1
Kudos: 249
Solutions: 126
Registered: ‎04-22-2014

Re: securing yarn logs

@nbts5n2,

 

Usually, UI security is done via Kerberos for YARN and Spark.  If you have enabled Kerberos authentication in your cluster and you have enabled.  If you are using Cloudera Manager, the following can be set to enable SPNEGO authentication for the YARN UI and HDFS UIs:

Enable Kerberos Authentication for HTTP Web-Consoles

 

To provide authorization you can enable ACLs I think and then specify admins via yarn.admin.acl.

 

This documentation may help: 

https://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_yarn_acl.html

 

For Spark see:

https://spark.apache.org/docs/latest/security.html#spark-history-server-acls

and

https://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_yarn_acl.html#concept_yarn_app...

(see the spark section)

Announcements