Member since
09-29-2015
871
Posts
721
Kudos Received
255
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2723 | 12-03-2018 02:26 PM | |
1793 | 10-16-2018 01:37 PM | |
3181 | 10-03-2018 06:34 PM | |
1905 | 09-05-2018 07:44 PM | |
1503 | 09-05-2018 07:31 PM |
03-15-2017
04:47 PM
I don't really know anything about HUNK so I can't speak to why it would be slower, maybe it hit some threshold where theres finally enough smaller files under the NiFi data that now it is slowing down? Just wanted to mention that after your previous post I asked a couple of people about the possibility of partially appended data during an error scenario, and the consensus seemed to be that HDFS wouldn't let this happen. So you might be fine just appending to one file per hour or day from NiFi.
... View more
03-13-2017
04:30 PM
Glad you got it working!
... View more
03-13-2017
04:25 PM
1 Kudo
This is a known issue when there are multiple processors with different principals, the JIRA is here and just got merged to master: https://issues.apache.org/jira/browse/NIFI-3520
... View more
03-13-2017
01:50 PM
2 Kudos
I believe that statement was referring specifically to the listing operation performed by ListFile, and not the overall state of the primary node. For example, if ListFile ran and listed files 1 and 2, and then the primary node changes and file 3 is available, it won't list files 1 and 2 again, it will start with 3, but if files 1 and 2 were in progress in the rest of the flow on the original primary node when it went down, they are stuck there until that node comes back up. The state tracking is done through the state management API which uses Zookeeper when clustered. I believe in this case it is using timestamps track the last time the processor ran and the timestamp of the most recent file it saw, and then looking for files newer than that on the next execution. Also keep in mind, this whole scenario only makes sense when listing a remote directory that all nodes in the NiFi cluster have access to, otherwise it doesn't make sense when listing a local directory that is only on one node.
... View more
03-10-2017
05:37 PM
I believe this is an issue between the truststore used by the plugin and certificates used by Solr... basically the SSL handshake believes that the certificate Solr is using is not trusted by whatever is in the truststore that the plugin is using. How did you enable SSL on Solr? Did you generate your own certificate and do this manually? The Ranger plugin that runs inside the NiFi JVM process (which is what sends the audits to Solr) will use the values of xasecure.policymgr.clientssl.truststore, xasecure.policymgr.clientssl.truststore.password, and xasecure.policymgr.clientssl.truststore.credential.file which come from ranger-nifi-policymgr-ssl.xml So the truststore specified there needs to trust the certificate authority that created the cert that Solr is using. Also, this issue could be related, but not sure: https://issues.apache.org/jira/browse/RANGER-1216 Looks like it was fixed for Ranger 0.7, but I believe HDF is using 0.6.x.
... View more
03-09-2017
09:40 PM
http://bryanbende.com/development/2016/08/30/apache-nifi-1.0.0-secure-site-to-site
... View more
03-09-2017
04:23 PM
1 Kudo
Correct, setting INFO on a package means all classes in the package will log at the INFO level, unless there is a more specific logger for a given class at a different level. For example: <logger name="org.apache.nifi.processors" level="INFO"/>
<logger name="org.apache.nifi.processors.MyProcessor" level="DEBUG"/> MyProcessor would log at DEBUG level and all other processors at INFO level. The type of information is really dependent on the processor. Generally you should use DEBUG for things you wouldn't expect to be logging all the time, but that might be helpful if someone was trying to troubleshoot something.
... View more
03-06-2017
09:10 PM
1 Kudo
The same question was already answered twice: https://community.hortonworks.com/questions/87146/how-can-we-pop-up-log-in-window-asking-user-id-and.html https://community.hortonworks.com/questions/87122/how-can-we-pop-up-the-log-in-window-while-accessin.html To clarify you can not login over "http", only via "https". It would not be secure to send your LDAP credentials from the browser to the NiFi server over unencrypted http.
... View more
03-06-2017
08:56 PM
2 Kudos
Can you verify that after the UpdateAttribute processor the "filename" attribute is set correctly? You could do this by either sticking a LogAttribute processor between UpdateAttribute and PutHDFS, or by stopping PutHDFS and listing the queue before it, and looking at the attributes of one of the flow files. I think that maybe the expression was somehow entered incorrectly in UpdateAttribute (possibly copied/pasted) because in the error message its showing your closing bracket as %7D. I would try going into UpdateAttribute and editing the filename property and deleting and retyping the closing bracket at the end of the expression to see what happens.
... View more
03-06-2017
06:57 PM
2 Kudos
The TLS/SSL properties in nifi.properties are only used for setting up authentication to NiFi. When talking to other systems, NiFi lets you specify a SSLContextService because a lot of times the certificate and truststore you want to use for communicating with other systems is going to be different. If they are the same then you can definitely create a StandardSSLContextService and point it at the same keystore and truststore that nifi.properties is pointing at.
... View more