Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How can I filter the NiFi log to exclude a certain error message?

avatar
Expert Contributor

I am constantly seeing this error message in the nifi-app.log file.

2018-04-16 11:00:36,383 ERROR [nifi.async.batch_nifi.async.batch.hdfs_destWriter] o.a.r.audit.provider.BaseAuditHandler Error writing to log file.
java.io.IOException: No FileSystem for scheme: hdfs
        at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2660)
        at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2667)
        at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
        at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703)
        at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2685)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
        at org.apache.ranger.audit.destination.HDFSAuditDestination.getLogFileStream(HDFSAuditDestination.java:271)
        at org.apache.ranger.audit.destination.HDFSAuditDestination.access$000(HDFSAuditDestination.java:43)
        at org.apache.ranger.audit.destination.HDFSAuditDestination$1.run(HDFSAuditDestination.java:157)
        at org.apache.ranger.audit.destination.HDFSAuditDestination$1.run(HDFSAuditDestination.java:154)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
        at org.apache.ranger.audit.provider.MiscUtil.executePrivilegedAction(MiscUtil.java:524)
        at org.apache.ranger.audit.destination.HDFSAuditDestination.logJSON(HDFSAuditDestination.java:154)
        at org.apache.ranger.audit.queue.AuditFileSpool.sendEvent(AuditFileSpool.java:880)
        at org.apache.ranger.audit.queue.AuditFileSpool.runLogAudit(AuditFileSpool.java:828)
        at org.apache.ranger.audit.queue.AuditFileSpool.run(AuditFileSpool.java:758)
        at java.lang.Thread.run(Thread.java:745)

Since this is a bug related to the version of NiFi I am on, I do not need to see it as it happens very frequently and is cluttering the log file. How can I exclude this so it is not logged? Is there some configuration I can do in the logback.xml file?

1 ACCEPTED SOLUTION

avatar
Super Mentor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
4 REPLIES 4

avatar
Super Mentor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
Expert Contributor

Hi Matt,

I was able to implement these steps and it did exactly what I needed. Thanks!

avatar
New Contributor

Hi @Josh Nicholson

Can you point what is the bug (URL in jira or any other info) related to the error regarding hdfs scheme? What is your version of NiFi?

Thanks,

Bruno

,

Hi Josh, can you point what is the bug (URL in jira or any other info) related to the error regarding hdfs scheme? What is your version of NiFi?

avatar
Rising Star

Another option could be to use Ambari log search