Support Questions
Find answers, ask questions, and share your expertise

Who agreed with this topic

Nifi HDFS audit error with HA NameNode

Hello, I've got an error when nifi and nifi-registry try to write audit ranger logs to HDFS.

 

 

2019-09-02 16:00:13,452 ERROR [XXXXXXX_nifi_registry.async.multi_dest.batch_XXXXXXX_nifi_registry.async.multi_dest.batch.hdfs_destWriter] o.a.r.audit.provider.BaseAuditHandler Error writing to log file.
java.lang.IllegalArgumentException: java.net.UnknownHostException: XXXXXXX
        at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:444) ~[hadoop-common-3.0.0.jar:na]
        at org.apache.hadoop.hdfs.NameNodeProxiesClient.createProxyWithClientProtocol(NameNodeProxiesClient.java:132) ~[hadoop-hdfs-client-3.0.0.jar:na]
        at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:355) ~[hadoop-hdfs-client-3.0.0.jar:na]
        at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:289) ~[hadoop-hdfs-client-3.0.0.jar:na]
        at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:163) ~[hadoop-hdfs-client-3.0.0.jar:na]
        at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3288) ~[hadoop-common-3.0.0.jar:na]
        at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:123) ~[hadoop-common-3.0.0.jar:na]
        at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3337) ~[hadoop-common-3.0.0.jar:na]
        at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3305) ~[hadoop-common-3.0.0.jar:na]
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:476) ~[hadoop-common-3.0.0.jar:na]
        at org.apache.ranger.audit.destination.HDFSAuditDestination.getLogFileStream(HDFSAuditDestination.java:284) ~[ranger-plugins-audit-1.1.0.jar:1.1.0]
        at org.apache.ranger.audit.destination.HDFSAuditDestination.access$000(HDFSAuditDestination.java:43) ~[ranger-plugins-audit-1.1.0.jar:1.1.0]
        at org.apache.ranger.audit.destination.HDFSAuditDestination$1.run(HDFSAuditDestination.java:156) ~[ranger-plugins-audit-1.1.0.jar:1.1.0]
        at org.apache.ranger.audit.destination.HDFSAuditDestination$1.run(HDFSAuditDestination.java:153) ~[ranger-plugins-audit-1.1.0.jar:1.1.0]
        at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_112]
        at javax.security.auth.Subject.doAs(Subject.java:422) ~[na:1.8.0_112]
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962) ~[hadoop-common-3.0.0.jar:na]
        at org.apache.ranger.audit.provider.MiscUtil.executePrivilegedAction(MiscUtil.java:523) ~[ranger-plugins-audit-1.1.0.jar:1.1.0]
        at org.apache.ranger.audit.destination.HDFSAuditDestination.logJSON(HDFSAuditDestination.java:153) ~[ranger-plugins-audit-1.1.0.jar:1.1.0]
        at org.apache.ranger.audit.queue.AuditFileSpool.sendEvent(AuditFileSpool.java:879) [ranger-plugins-audit-1.1.0.jar:1.1.0]
        at org.apache.ranger.audit.queue.AuditFileSpool.runLogAudit(AuditFileSpool.java:827) [ranger-plugins-audit-1.1.0.jar:1.1.0]
        at org.apache.ranger.audit.queue.AuditFileSpool.run(AuditFileSpool.java:757) [ranger-plugins-audit-1.1.0.jar:1.1.0]
        at java.lang.Thread.run(Thread.java:745) [na:1.8.0_112]
Caused by: java.net.UnknownHostException: XXXXXXX
        ... 23 common frames omitted

 

 

 

 My namenode is in HA so I set for the configuration xasecure.audit.destination.hdfs.dir the value hdfs://XXXXX/ranger/audit

 

Nifi seams not to accept the name of the cluster as HDFS node ?

Any idea ?

 

I use HDP 3.1 and HDF 3.4.

 

Thank's a lot

 

Mathieu

Who agreed with this topic