Member since
11-30-2017
44
Posts
6
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1436 | 04-16-2018 07:49 PM | |
1752 | 01-05-2018 02:31 PM |
04-16-2018
07:49 PM
HDF 3.1.1 is not compatible with HDP 2.6.1. I was told by Hortonworks that HDP 3.0 will be compatible and I must wait until its release.
... View more
04-16-2018
07:32 PM
Hi Matt, I was able to implement these steps and it did exactly what I needed. Thanks!
... View more
04-16-2018
05:06 PM
I am constantly seeing this error message in the nifi-app.log file. 2018-04-16 11:00:36,383 ERROR [nifi.async.batch_nifi.async.batch.hdfs_destWriter] o.a.r.audit.provider.BaseAuditHandler Error writing to log file.
java.io.IOException: No FileSystem for scheme: hdfs
at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2660)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2667)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2685)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
at org.apache.ranger.audit.destination.HDFSAuditDestination.getLogFileStream(HDFSAuditDestination.java:271)
at org.apache.ranger.audit.destination.HDFSAuditDestination.access$000(HDFSAuditDestination.java:43)
at org.apache.ranger.audit.destination.HDFSAuditDestination$1.run(HDFSAuditDestination.java:157)
at org.apache.ranger.audit.destination.HDFSAuditDestination$1.run(HDFSAuditDestination.java:154)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at org.apache.ranger.audit.provider.MiscUtil.executePrivilegedAction(MiscUtil.java:524)
at org.apache.ranger.audit.destination.HDFSAuditDestination.logJSON(HDFSAuditDestination.java:154)
at org.apache.ranger.audit.queue.AuditFileSpool.sendEvent(AuditFileSpool.java:880)
at org.apache.ranger.audit.queue.AuditFileSpool.runLogAudit(AuditFileSpool.java:828)
at org.apache.ranger.audit.queue.AuditFileSpool.run(AuditFileSpool.java:758)
at java.lang.Thread.run(Thread.java:745)
Since this is a bug related to the version of NiFi I am on, I do not need to see it as it happens very frequently and is cluttering the log file. How can I exclude this so it is not logged? Is there some configuration I can do in the logback.xml file?
... View more
Labels:
- Labels:
-
Apache NiFi
-
Cloudera DataFlow (CDF)
03-09-2018
01:41 PM
I am currently on HDP 2.6.1, Ambari 2.5.1, and HDF 3.0.1 which is installed on top of HDP. I am ultimately wanting to upgrade HDF to 3.1.1 so I can use the newer features like the Registry. I am following steps here to upgrade. https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.1.1/bk_ambari-upgrade/content/ch_upgrade-hdf-on-hdp.html However, after first upgrading Ambari to 2.6.1.5, and then HDF to 3.1.1, Ambari still shows in the versions that NiFi is running version 1.2.0 when it is actually on 1.5.0. What am I missing here? How can I get the correct version to show in Ambari? Due to the manual process of upgrading the packages, it seems Ambari is no longer aware of what version NiFi is actually on.
... View more
Labels:
- Labels:
-
Apache Ambari
-
Cloudera DataFlow (CDF)
01-23-2018
07:00 PM
@clukasik In the last line of the script, you have alertHost s `hostname`
I am wanting to send the FQDN of the host, but if I put alertHost s `hostname -f`
the monitoring service I am sending the trap to never receives the trap at all. Why is this? Do you have any recommendations on how I can send the FQDN?
... View more
01-19-2018
07:38 PM
You were correct, once I deleted the client certificate I started getting the same error in my title when browsing to it in Chrome. Also I noticed on the UI it now has myuser@realm instead of just myuser. Is there a way change this?
... View more
01-19-2018
06:29 PM
Here's what I see there. 2018-01-19 11:30:24,421 INFO [NiFi Web Server-284700] o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (<myuser@realm><hostname><CN=hostname, OU=myorg, O=mycompany, L=mylocation, ST=mystate, C=US>) GET https:/<hostname>:9091/nifi-api/process-groups/7c84501d-d10c-407c-b9f3-1d80e38fe36a
2018-01-19 11:30:24,421 INFO [NiFi Web Server-284700] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for <myuser>@<realm>
2018-01-19 11:30:24,421 INFO [NiFi Web Server-284700] o.a.n.w.a.c.AccessDeniedExceptionMapper <myuser>@<realm> does not have permission to access the requested resource. No applicable policies could be found. Returning Forbidden response. Edit: I actually was sort of able to resolve this. For some reason the curl command is sending myuser@realm instead of just myuser, which is what I gave access to in Ranger. When I granted myuser@realm access in Ranger, the curl command worked. So the question is why am I able to login to the UI by just granting myuser access but in order to curl I have to give myuser@realm access? Edit 2: I'm thinking it's due to the way I have the pattern mapping configured. I'm guessing when I log into the UI it's using kerberos, and then when I'm running a curl it's using the CN, which is myuser@realm instead of just myuser. Is it possible to configure so I can just use myuser?
... View more
01-19-2018
04:31 PM
First I ran the following command to get an authentication token and verified it this command worked by echoing $token. token=$(curl -k -X POST --negotiate -u : https://<nifi-hostname>:9091/nifi-api/access/kerberos) Then I tried to get one of the processor groups by running this command but got "No applicable policies could be found. Contact the system administrator." curl -k -X GET "https://<nifi-hostname>:9091/nifi-api/process-groups/7c84501d-d10c-407c-b9f3-1d80e38fe36a" -H "Authorization: Bearer $token" I am able to access the NiFi UI and have my given myself access to * for the NiFi Resource Identifier in Ranger. Additionally if I use Google Chrome I am am able to browse to this link and view the JSON, I'm just not able to execute the curl. I executed all of the above commands on the NiFi node I am trying to curl to.
... View more
Labels:
- Labels:
-
Apache NiFi
01-15-2018
03:21 PM
Hi Timothy, In the second link you provided, it says "We plan to provide a benchmarking and performance test template to include in the build, which allows users to easily test their system and to identify where bottlenecks are, and at which point they might become a factor. This template should also make it easy for system administrators to make changes and to verify the impact." Do you know happen to know how far away this is? It's exactly what I'm looking for. Thanks, Josh
... View more
01-05-2018
02:31 PM
1 Kudo
It appears there are none at this moment. Here is an open JIRA regarding this. https://issues.apache.org/jira/browse/NIFI-213
... View more
- « Previous
- Next »