Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

Ranger cannot record Knox audit log to hdfs

New Contributor

Hello, I would like to ask a question.

I integrated Ranger + Knox + LDAP on my test cluster.

Ranger could records the audit logs into HDFS except the audit log of Knox.

It doesn't have any user or group synchronize problem with LDAP on my cluster.

Let me list up those component version of my cluster:

Ambari 2.2.1

HDP 2.4.2.0

Ranger: 0.5.0.2.4

Knox: 0.6.0.2.4

The following error message is from /var/log/knox/gateway.log

I cannot figure out why it couldn't recognize my cluster nameservice.

My HDFS directory is hdfs://testcluster/ranger/audit

2016-07-06 19:24:18,591 ERROR queue.AuditFileSpool (AuditFileSpool.java:logError(710)) - Error sending logs to consumer. provider=knox.async.batch, consumer=knox.async.batch.hdfs
2016-07-06 19:25:18,669 ERROR provider.BaseAuditHandler (BaseAuditHandler.java:logError(329)) - Error writing to log file.
java.lang.IllegalArgumentException: java.net.UnknownHostException: testcluster
	at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:411)
	at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:311)
	at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:678)
	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:619)
	at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:150)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2653)
	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92)
	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687)
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
	at org.apache.ranger.audit.destination.HDFSAuditDestination.getLogFileStream(HDFSAuditDestination.java:221)
	at org.apache.ranger.audit.destination.HDFSAuditDestination.logJSON(HDFSAuditDestination.java:123)
	at org.apache.ranger.audit.queue.AuditFileSpool.sendEvent(AuditFileSpool.java:890)
	at org.apache.ranger.audit.queue.AuditFileSpool.runDoAs(AuditFileSpool.java:838)
	at org.apache.ranger.audit.queue.AuditFileSpool$2.run(AuditFileSpool.java:759)
	at org.apache.ranger.audit.queue.AuditFileSpool$2.run(AuditFileSpool.java:757)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:360)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1689)
	at org.apache.ranger.audit.queue.AuditFileSpool.run(AuditFileSpool.java:765)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.UnknownHostException: testcluster
	... 22 more

Please help me figure this out.

Thanks

1 ACCEPTED SOLUTION

can you please refer to this document :http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.2/bk_Ranger_Install_Guide/content/save_audits_.... and please see the steps to create the following symbolic links:

Link /etc/hadoop/conf/hdfs-site.xml file to /etc/knox/conf/hdfs-site.xml

Link /etc/hadoop/conf/core-site.xml file to /etc/knox/conf/core-site.xml

More

View solution in original post

6 REPLIES 6

is this HA cluster?

can you please provide the knox audit confguration

can you please refer to this document :http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.2/bk_Ranger_Install_Guide/content/save_audits_.... and please see the steps to create the following symbolic links:

Link /etc/hadoop/conf/hdfs-site.xml file to /etc/knox/conf/hdfs-site.xml

Link /etc/hadoop/conf/core-site.xml file to /etc/knox/conf/core-site.xml

More

New Contributor

After configured by following the document.

It works now.

Thank you very much.

Explorer

Soft linking of hdfs and core site xmls on KNOX Gateway server fixed the UnknownHostException issue.

The versions I was working with are

HDP - 2.5

Ranger - 0.6.0.2.5

Knox - 0.9.0.2.5

Thanks

did it solve the problem ?

Rising Star

Is 'testcluster' the name of the topology file that you've configured via Knox?

You should have a file named 'testcluster.xml' under conf/topologies

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.