Cloudera Community

Announcements
Celebrating as our community reaches 100,000 members! Thank you!
alabbe70
user rank
Explorer

Re: How to prevent nifi-app logs from filling my d...

by Explorer in Support Questions
‎10-26-2017 03:58 AM
‎10-26-2017 03:58 AM
Hi @Abdelkrim Hadjidj These links were very informative. After reading these and doing some testing I think I have found what I need but it is not available in the version that I have. The option I am looking for is totalSizeCap. If I am reading the notes correctly is available in logback 1.1.8, and I am running Nifi 1.1.1 witch uses logback 1.1.3. Would you happen to know what version of logback is used by the newest version of Nifi? ... View more

How to prevent nifi-app logs from filling my drive...

by Explorer in Support Questions
‎10-23-2017 09:44 PM
‎10-23-2017 09:44 PM
This past weekend I had an issue and my nifi-app logs filed up my drive. I have address the issue but I am trying to find out how I can configure nifi to rotate the nifi-app files so it dose not file up my disk space again. Thank you ... View more
Labels:

Re: Issue with Nifi 1.1.1 cluster and Process set ...

by Explorer in Support Questions
‎10-04-2017 01:26 AM
‎10-04-2017 01:26 AM
Than you Matt. I will be making changes to my environment will hopefully improve with every new chalage. ... View more

Re: Issue with Nifi 1.1.1 cluster and Process set ...

by Explorer in Support Questions
‎10-03-2017 04:13 AM
‎10-03-2017 04:13 AM
Matt, Thank you for your reply. It shows me that I still have a lot to learn about Nifi. 1. This explanation helped me. You mention the Cluster UI. Is that different than http://Node1:8181/nifi? I ask because when I have only one node running I only get a screen telling me "An unexpected error has occurred". 2. For my testing I have it set to run every 5 min. I have created a simple flow. Consisting of ExecuteScript->ExecuteStreamCommand->ExecuteScript-> ExecuteScript->InvokeHTTP. I have tried setting just the first one to “Primary node only”, I have also set them all to “Primary node only”. In these two tests after waiting 30 min. and not getting any data, I simply switch them to “All nodes”, and I start to get data within minutes. I think I should first get my cluster set up properly then come back to this issue. As they say baby steps. I see that you are recommending against using the embedded zookeeper. Do you know of a good link explaining where to get the zookeeper and how to configure it with Nifi? Do you need to have one nifi for each zookeeper? Thank you for all your help. Andre ... View more

Issue with Nifi 1.1.1 cluster and Process set to "...

by Explorer in Support Questions
‎10-02-2017 05:26 PM
‎10-02-2017 05:26 PM
I am new to Nifi and trying to learn how to use it. I have set up two servers with Nifi and configured them to be a cluster. I am currently having two issues that I think are related. 1 - If I take one server off line I can not access Nifi. 2 - If I set a work process to execute on "Primary node", it dose not run. If I set it to "All nodes" then I get two copes of the work. Any help would be appreciated. I have included what I think is the pertinent data from both servers for /conf/nifi.properties, /conf/zookeep.properties, and /state/zookeeper/myid. Thank you. For Node1.com My /conf/nifi.properties #################### # State Management # #################### nifi.state.management.configuration.file=./conf/state-management.xml # The ID of the local state provider nifi.state.management.provider.local=local-provider # The ID of the cluster-wide state provider. This will be ignored if NiFi is not clustered but must be populated if running in a cluster. nifi.state.management.provider.cluster=zk-provider # Specifies whether or not this instance of NiFi should run an embedded ZooKeeper server nifi.state.management.embedded.zookeeper.start=true # Properties file that provides the ZooKeeper properties to use if <nifi.state.management.embedded.zookeeper.start> is set to true nifi.state.management.embedded.zookeeper.properties=./conf/zookeeper.properties # web properties # nifi.web.war.directory=./lib nifi.web.http.host=Node1.com nifi.web.http.port=8181 nifi.web.https.host= nifi.web.https.port= nifi.web.jetty.working.directory=./work/jetty nifi.web.jetty.threads=200 # cluster node properties (only configure for cluster nodes) # nifi.cluster.is.node=true nifi.cluster.node.address=Node1.com nifi.cluster.node.protocol.port=12000 nifi.cluster.node.protocol.threads=10 nifi.cluster.node.event.history.size=25 nifi.cluster.node.connection.timeout=5 sec nifi.cluster.node.read.timeout=5 sec nifi.cluster.firewall.file= nifi.cluster.flow.election.max.wait.time=5 mins nifi.cluster.flow.election.max.candidates=2 # zookeeper properties, used for cluster management # nifi.zookeeper.connect.string=Node1.com:2181,Node2.com:2182 nifi.zookeeper.connect.timeout=3 secs nifi.zookeeper.session.timeout=3 secs nifi.zookeeper.root.node=/nifi My /conf/zookeep.properties clientPort=2181 initLimit=10 autopurge.purgeInterval=24 syncLimit=5 tickTime=2000 dataDir=./state/zookeeper autopurge.snapRetainCount=30 server.1=Node1.com:2888:3888 server.2=Node2.com:2888:3888 My /state/zookeeper/myid 1 For Node2.com My /conf/nifi.properties #################### # State Management # #################### nifi.state.management.configuration.file=./conf/state-management.xml # The ID of the local state provider nifi.state.management.provider.local=local-provider # The ID of the cluster-wide state provider. This will be ignored if NiFi is not clustered but must be populated if running in a cluster. nifi.state.management.provider.cluster=zk-provider # Specifies whether or not this instance of NiFi should run an embedded ZooKeeper server nifi.state.management.embedded.zookeeper.start=true # Properties file that provides the ZooKeeper properties to use if <nifi.state.management.embedded.zookeeper.start> is set to true nifi.state.management.embedded.zookeeper.properties=./conf/zookeeper.properties # web properties # nifi.web.war.directory=./lib nifi.web.http.host=Node2.com nifi.web.http.port=8181 nifi.web.https.host= nifi.web.https.port= nifi.web.jetty.working.directory=./work/jetty nifi.web.jetty.threads=200 # cluster node properties (only configure for cluster nodes) # nifi.cluster.is.node=true nifi.cluster.node.address=Node2.com nifi.cluster.node.protocol.port=12000 nifi.cluster.node.protocol.threads=10 nifi.cluster.node.event.history.size=25 nifi.cluster.node.connection.timeout=5 sec nifi.cluster.node.read.timeout=5 sec nifi.cluster.firewall.file= nifi.cluster.flow.election.max.wait.time=5 mins nifi.cluster.flow.election.max.candidates=2 # zookeeper properties, used for cluster management # nifi.zookeeper.connect.string=Node1.com:2181,Node2.com:2182 nifi.zookeeper.connect.timeout=3 secs nifi.zookeeper.session.timeout=3 secs nifi.zookeeper.root.node=/nifi My /conf/zookeep.properties clientPort=2181 initLimit=10 autopurge.purgeInterval=24 syncLimit=5 tickTime=2000 dataDir=./state/zookeeper autopurge.snapRetainCount=30 server.1=Node1.com:2888:3888 server.2=Node2.com:2888:3888 My /state/zookeeper/myid 2 ... View more
Labels:
Contact Me
Online
Offline
Last Visited
‎10-26-2017 03:58 AM
Community Statistics
Member Since ‎10-02-2017 04:25 AM
Last Visited ‎10-26-2017 03:58 AM
Posts 5