Member since
09-29-2015
871
Posts
723
Kudos Received
255
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 4254 | 12-03-2018 02:26 PM | |
| 3194 | 10-16-2018 01:37 PM | |
| 4304 | 10-03-2018 06:34 PM | |
| 3162 | 09-05-2018 07:44 PM | |
| 2419 | 09-05-2018 07:31 PM |
09-26-2016
03:26 PM
Clicking the lock icon should bring up the policies window: What browser are you using? You may want to look for Javascript errors to see if something is going wrong client side?
... View more
09-24-2016
01:47 PM
1 Kudo
I believe you should be able to use /etc/hadoop/conf/core-site.xml and /etc/hadoop/conf/hdfs-site.xml, and if you are running NiFi/HDF directly on the sandbox there shouldn't be any connectivity problems. Do you have the relationships of PutHDFS going anywhere, or are they auto-terminated? It would be helpful to route them somewhere to see if the processor thinks it is a success or failure, you could create two LogAttribute processors, one for success and one for failure and see which it goes to.
... View more
09-23-2016
04:51 PM
users.xml and authorizations.xml are auto generated based on what you put in authorizers.xml, and authorizers.xml should be the same on all three nodes, authorizers.xml needs the Initial Admin and a Node Identity entry for all three nodes of your cluster. In your screenshot it looks you haven't setup the Node Identities correctly, because two of them have dn01 and one of them have dn02. If theres three nodes they should have dn01, dn02, and dn03. An example of authorizers.xml for your case would be: <authorizer>
<identifier>file-provider</identifier>
<class>org.apache.nifi.authorization.FileAuthorizer</class>
<property name="Authorizations File">./conf/authorizations.xml</property>
<property name="Users File">./conf/users.xml</property>
<property name="Initial Admin Identity">your admin user</property>
<property name="Legacy Authorized Users File"></property>
<property name="Node Identity 1">DN of node1</property>
<property name="Node Identity 2">DN of node2</property>
<property name="Node Identity 2">DN of node3</property>
</authorizer>
and that should be the same on all three nodes.
... View more
09-22-2016
04:27 PM
Ok that is what I was looking for, so when joining the cluster it checks the nodes flow.xml.gz and also the contents of users.xml and authorizations.xml against what the cluster has. Double check the two nodes that are not starting and see what they have in users.xml and authorizations.xml compared to the node that is starting. If they are different, you could copy over from the first node to get them in sync.
... View more
09-22-2016
03:24 PM
Can you also try to get the logs from hsuswstgdn01 ? It looks like this is a different node that was the coordinator and was logging that error that it received from hsuswstgdn01. I'm hoping that one hsuswstgdn01 there is a more detailed stacktrace for the uninheritable flow ERROR. If you are able to, could you post all three log files instead of screenshots?
... View more
09-22-2016
02:38 PM
Ok, I have only seen this problem when the flows are actually different, and then deleting the different flow.xml.gz and restarting that node corrected the problem. One thing you could try is to stop everything, delete the flow.xml.gz on each node again, then start only the first node and let it fully start and verify you can get to the UI for that node, then start the other two and see what happens.
... View more
09-22-2016
02:16 PM
Are you using the 1.0.0 BETA or the official 1.0.0 release? My blog post was written when the BETA came out, but the official 1.0.0 release is out now which fixed problems found in the BETA. https://nifi.apache.org/download.html
... View more
09-21-2016
09:07 PM
3 Kudos
In many parts of your flow you have multiple relationships routed to the next processor when you probably want only one, some examples... Between SplitText and ExtractText you have original and splits connected, but you probably only want splits here. Between ExtractText and ReplaceText you have matched and unmatched, but you probably only want matched. Between ReplaceText and PutSQL you have success and failure, but you probably only want success. On PutSQL you have route failure, success, and retry back to itself, and you probably only want retry (you definitely don't want success routed back to itself). You would most likely auto-terminate these other relationships (first tab when configured a processor).
... View more
09-21-2016
05:27 PM
There are several different aspects of HA, and a cluster does provide some of them... For example the primary node and cluster coordinator can automatically failover to another node on the cluster, so that is HA. Also, in the case where each node in the cluster had a processor listening like ListenHTTP, you can put a load balancer in front of them, so that is HA. The part that it doesn't provide yet is HA of data, meaning replicating data from node1 to node2, that is something the community is working towards. Clarifying what you said about GetHTTP... GetHTTP is pulling data, not having it pushed from somewhere. ListenHTTP would be what you described with web1 sending data to node1, web2 sending to node2, etc. This article might help give you an idea of how things are working in a cluster: https://community.hortonworks.com/articles/16120/how-do-i-distribute-data-across-a-nifi-cluster.html
... View more
09-21-2016
04:32 PM
So you deleted the flow.xml.gz from all three nodes and then tried starting again and it still wouldn't start?
... View more