Support Questions
Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search

Who agreed with this topic

Nifi HDFS audit error with HA NameNode

couron_mathieu
Explorer

Created on ‎09-02-2019 07:38 AM - last edited on ‎09-02-2019 09:12 AM by Moderator

Hello, I've got an error when nifi and nifi-registry try to write audit ranger logs to HDFS.

 

 

2019-09-02 16:00:13,452 ERROR [XXXXXXX_nifi_registry.async.multi_dest.batch_XXXXXXX_nifi_registry.async.multi_dest.batch.hdfs_destWriter] o.a.r.audit.provider.BaseAuditHandler Error writing to log file.
java.lang.IllegalArgumentException: java.net.UnknownHostException: XXXXXXX
        at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:444) ~[hadoop-common-3.0.0.jar:na]
        at org.apache.hadoop.hdfs.NameNodeProxiesClient.createProxyWithClientProtocol(NameNodeProxiesClient.java:132) ~[hadoop-hdfs-client-3.0.0.jar:na]
        at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:355) ~[hadoop-hdfs-client-3.0.0.jar:na]
        at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:289) ~[hadoop-hdfs-client-3.0.0.jar:na]
        at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:163) ~[hadoop-hdfs-client-3.0.0.jar:na]
        at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3288) ~[hadoop-common-3.0.0.jar:na]
        at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:123) ~[hadoop-common-3.0.0.jar:na]
        at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3337) ~[hadoop-common-3.0.0.jar:na]
        at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3305) ~[hadoop-common-3.0.0.jar:na]
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:476) ~[hadoop-common-3.0.0.jar:na]
        at org.apache.ranger.audit.destination.HDFSAuditDestination.getLogFileStream(HDFSAuditDestination.java:284) ~[ranger-plugins-audit-1.1.0.jar:1.1.0]
        at org.apache.ranger.audit.destination.HDFSAuditDestination.access$000(HDFSAuditDestination.java:43) ~[ranger-plugins-audit-1.1.0.jar:1.1.0]
        at org.apache.ranger.audit.destination.HDFSAuditDestination$1.run(HDFSAuditDestination.java:156) ~[ranger-plugins-audit-1.1.0.jar:1.1.0]
        at org.apache.ranger.audit.destination.HDFSAuditDestination$1.run(HDFSAuditDestination.java:153) ~[ranger-plugins-audit-1.1.0.jar:1.1.0]
        at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_112]
        at javax.security.auth.Subject.doAs(Subject.java:422) ~[na:1.8.0_112]
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962) ~[hadoop-common-3.0.0.jar:na]
        at org.apache.ranger.audit.provider.MiscUtil.executePrivilegedAction(MiscUtil.java:523) ~[ranger-plugins-audit-1.1.0.jar:1.1.0]
        at org.apache.ranger.audit.destination.HDFSAuditDestination.logJSON(HDFSAuditDestination.java:153) ~[ranger-plugins-audit-1.1.0.jar:1.1.0]
        at org.apache.ranger.audit.queue.AuditFileSpool.sendEvent(AuditFileSpool.java:879) [ranger-plugins-audit-1.1.0.jar:1.1.0]
        at org.apache.ranger.audit.queue.AuditFileSpool.runLogAudit(AuditFileSpool.java:827) [ranger-plugins-audit-1.1.0.jar:1.1.0]
        at org.apache.ranger.audit.queue.AuditFileSpool.run(AuditFileSpool.java:757) [ranger-plugins-audit-1.1.0.jar:1.1.0]
        at java.lang.Thread.run(Thread.java:745) [na:1.8.0_112]
Caused by: java.net.UnknownHostException: XXXXXXX
        ... 23 common frames omitted

 

 

 

 My namenode is in HA so I set for the configuration xasecure.audit.destination.hdfs.dir the value hdfs://XXXXX/ranger/audit

 

Nifi seams not to accept the name of the cluster as HDFS node ?

Any idea ?

 

I use HDP 3.1 and HDF 3.4.

 

Thank's a lot

 

Mathieu

870 Views
1
Who agreed with this topic