Member since
10-02-2017
5
Posts
0
Kudos Received
0
Solutions
10-26-2017
03:58 AM
Hi @Abdelkrim Hadjidj These links were very informative. After reading these and doing some testing I think I have found what I need but it is not available in the version that I have. The option I am looking for is totalSizeCap. If I am reading the notes correctly is available in logback 1.1.8, and I am running Nifi 1.1.1 witch uses logback 1.1.3. Would you happen to know what version of logback is used by the newest version of Nifi?
... View more
10-23-2017
09:44 PM
This past weekend I had an issue and my nifi-app logs filed up my drive. I have address the issue but I am trying to find out how I can configure nifi to rotate the nifi-app files so it dose not file up my disk space again. Thank you
... View more
Labels:
- Labels:
-
Apache NiFi
10-04-2017
01:26 AM
Than you Matt. I will be making changes to my environment will hopefully improve with every new chalage.
... View more
10-03-2017
04:13 AM
Matt, Thank you for your
reply. It shows me that I still have a lot to learn about Nifi. 1. This explanation helped me. You mention the Cluster UI.
Is that different than http://Node1:8181/nifi? I ask because when I have only
one node running I only get a screen telling me "An unexpected error has
occurred". 2. For my testing I have it set to run every 5 min. I have
created a simple flow. Consisting of ExecuteScript->ExecuteStreamCommand->ExecuteScript->
ExecuteScript->InvokeHTTP. I have tried setting just the first one to “Primary
node only”, I have also set them all to “Primary node only”. In these two tests
after waiting 30 min. and not getting any data, I simply switch them to “All
nodes”, and I start to get data within minutes. I think I should first get my
cluster set up properly then come back to this issue. As they say baby steps. I see that you are recommending against using the embedded
zookeeper. Do you know of a good link explaining where to get the zookeeper and
how to configure it with Nifi? Do you need to have one nifi for each zookeeper? Thank you for all your help. Andre
... View more
10-02-2017
05:26 PM
I am new to Nifi and trying to learn how to use it. I have set up two servers with Nifi and configured them to be a cluster. I am currently having two issues that I think are related. 1 - If I take one server off line I can not access Nifi. 2 - If I set a work process to execute on "Primary node", it dose not run. If I set it to "All nodes" then I get two copes of the work. Any help would be appreciated. I have included what I think is the pertinent data from both servers for /conf/nifi.properties, /conf/zookeep.properties, and /state/zookeeper/myid. Thank you. For Node1.com My /conf/nifi.properties #################### # State Management # #################### nifi.state.management.configuration.file=./conf/state-management.xml # The ID of the local state provider nifi.state.management.provider.local=local-provider # The ID of the cluster-wide state provider. This will be ignored if NiFi is not clustered but must be populated if running in a cluster. nifi.state.management.provider.cluster=zk-provider # Specifies whether or not this instance of NiFi should run an embedded ZooKeeper server nifi.state.management.embedded.zookeeper.start=true # Properties file that provides the ZooKeeper properties to use if <nifi.state.management.embedded.zookeeper.start> is set to true nifi.state.management.embedded.zookeeper.properties=./conf/zookeeper.properties # web properties # nifi.web.war.directory=./lib nifi.web.http.host=Node1.com nifi.web.http.port=8181 nifi.web.https.host= nifi.web.https.port= nifi.web.jetty.working.directory=./work/jetty nifi.web.jetty.threads=200 # cluster node properties (only configure for cluster nodes) # nifi.cluster.is.node=true nifi.cluster.node.address=Node1.com nifi.cluster.node.protocol.port=12000 nifi.cluster.node.protocol.threads=10 nifi.cluster.node.event.history.size=25 nifi.cluster.node.connection.timeout=5 sec nifi.cluster.node.read.timeout=5 sec nifi.cluster.firewall.file= nifi.cluster.flow.election.max.wait.time=5 mins nifi.cluster.flow.election.max.candidates=2 # zookeeper properties, used for cluster management # nifi.zookeeper.connect.string=Node1.com:2181,Node2.com:2182 nifi.zookeeper.connect.timeout=3 secs nifi.zookeeper.session.timeout=3 secs nifi.zookeeper.root.node=/nifi My /conf/zookeep.properties clientPort=2181 initLimit=10 autopurge.purgeInterval=24 syncLimit=5 tickTime=2000 dataDir=./state/zookeeper autopurge.snapRetainCount=30 server.1=Node1.com:2888:3888 server.2=Node2.com:2888:3888 My /state/zookeeper/myid 1 For Node2.com My /conf/nifi.properties #################### # State Management # #################### nifi.state.management.configuration.file=./conf/state-management.xml # The ID of the local state provider nifi.state.management.provider.local=local-provider # The ID of the cluster-wide state provider. This will be ignored if NiFi is not clustered but must be populated if running in a cluster. nifi.state.management.provider.cluster=zk-provider # Specifies whether or not this instance of NiFi should run an embedded ZooKeeper server nifi.state.management.embedded.zookeeper.start=true # Properties file that provides the ZooKeeper properties to use if <nifi.state.management.embedded.zookeeper.start> is set to true nifi.state.management.embedded.zookeeper.properties=./conf/zookeeper.properties # web properties # nifi.web.war.directory=./lib nifi.web.http.host=Node2.com nifi.web.http.port=8181 nifi.web.https.host= nifi.web.https.port= nifi.web.jetty.working.directory=./work/jetty nifi.web.jetty.threads=200 # cluster node properties (only configure for cluster nodes) # nifi.cluster.is.node=true nifi.cluster.node.address=Node2.com nifi.cluster.node.protocol.port=12000 nifi.cluster.node.protocol.threads=10 nifi.cluster.node.event.history.size=25 nifi.cluster.node.connection.timeout=5 sec nifi.cluster.node.read.timeout=5 sec nifi.cluster.firewall.file= nifi.cluster.flow.election.max.wait.time=5 mins nifi.cluster.flow.election.max.candidates=2 # zookeeper properties, used for cluster management # nifi.zookeeper.connect.string=Node1.com:2181,Node2.com:2182 nifi.zookeeper.connect.timeout=3 secs nifi.zookeeper.session.timeout=3 secs nifi.zookeeper.root.node=/nifi My /conf/zookeep.properties clientPort=2181 initLimit=10 autopurge.purgeInterval=24 syncLimit=5 tickTime=2000 dataDir=./state/zookeeper autopurge.snapRetainCount=30 server.1=Node1.com:2888:3888 server.2=Node2.com:2888:3888 My /state/zookeeper/myid 2
... View more
Labels:
- Labels:
-
Apache NiFi
-
Apache Zookeeper