Member since
07-30-2019
3432
Posts
1632
Kudos Received
1012
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 101 | 01-27-2026 12:46 PM | |
| 510 | 01-13-2026 11:14 AM | |
| 1113 | 01-09-2026 06:58 AM | |
| 944 | 12-17-2025 05:55 AM | |
| 449 | 12-17-2025 05:34 AM |
05-15-2018
01:19 PM
2 Kudos
@Sonny Chee It is likely the flow is becoming out of sync as a result of one of the following scenarios: - 1. A change replication request was made to all nodes in the cluster. One or more nodes failed to process that request in the configured allowable nifi.cluster.node.connection.timeout and/or nifi.cluster.node.read.timeout. By default these values are set very low. Recommend setting these at a minimum of 30 secs. Additional reasons why a replication request may fail can range from network issues (high latency or packet loss) to the size of the replication request (snippets) (pasting/copying large selection of canvas components or instantiating a large template. Each node must complete adding every component, connection, controller service, etc in that snippet before responding to the replication request) - 2. A node was in a disconnected state. Every cluster has a cluster coordinator which every node sends heartbeats to. If a node is disconnected from the cluster (through user manual action, or because request failure above), that node's UI cab still be accessed. It will display the work "disconnected" on status bar, but will still allow a user who is connected directly to that node's UI to make changes. So if an attempt is made later (after making some change) to reconnect this node to the cluster, it will fail because flows do not match between cluster and this node anymore. User's who are connected to anyone of the other nodes still in the cluster will see a missing node in the status bar (for example: 1/2). From these node's UIs changes will not be allowed because node is missing. - It is best to look at nifi-app.log or user the cluster UI in NiFi to see why a node was disconnected in the first place. - Users should be educated on how NiFi works as described above and should avoid making changes to canvas when "disconnected" is displayed on NiFi UI status bar above canvas. - NiFi allows changes to be made on disconnected nodes by design. The intent here is to allow users to disconnect a node to perform troubleshooting of a misbehaving node or to perform some temporary one off testing that they do not want to perform across entire cluster. IN such cases, those changes must be backed out before being able to rejoin cluster. Another option is to rename/remove flow.xml.gz on that disconnected node. On restart, in the absence of a flow.xml.gz file, the connecting node will inherit the flow.xml.gz from the cluster. - Thanks, Matt
... View more
05-14-2018
06:38 PM
@Tarek Elgamal Assuming you are referring to settings for "Max Timer Driven Thread count"? That setting controls the max number of threads that can execute at one time. Does not guarantee any order to the execution of threads. NiFi's controller in the back ground does not operate under this thread pool. Both processors will be scheduled to run based on their configured run schedule. Those concurrent tasks then get stacked in a request queue waiting on one of the threads from that pool to service them. This way, every processor is eventually going to get a chance to run thier code. Also keep in mind that some processors work on batches of FlowFiles while others process one FlowFile per task. Also hard to say that each processed FlowFile will take same amount of time to complete an operation. Really depends on processor and what it is designed to do. Thanks, Matt
... View more
05-10-2018
05:29 PM
@Veerendra Nath Jasthi Glad to hear we got this all worked out... 🙂 Please take a moment to login and click the "accept" link for this answer to close out the thread. - Thank you, Matt
... View more
05-10-2018
05:16 PM
@Veerendra Nath Jasthi The screenshot you just attached shows a non secure NiFi running over HTTP on port 9090. There is no user authentication that will occur for a unsecured NiFi. User authentication and authorization will only occur for a secured NiFi running over https. Thanks, Matt
... View more
05-10-2018
04:57 PM
@Veerendra Nath Jasthi *** Forum Tip: Please try to avoid responding to an existing Answer by starting a new Answer. This makes following multiple response threads difficult. Instead use "add comment" to respond to a posted answer. - The attached image shows that the user who is currently logged in as ""CN=niifiadmin, OU=HORTONWORKS". Your comment says you expect to see this and that is what is being shown, so I am little confused. Sorry if I am missing something here. - Are you saying that you expected the UI to instead show ""CN=niifiadmin, OU=CLOUD.HORTONWORKS.COM"? - The UI is only going to display what is presented in the certificate being used for authentication. Try performing a verbose listing on your client certificate to see what the actual DN is. - Sharing your nifi-users.log, authorizers.xml, users.xml, and authorizations.xml files may shed some light on what is going on here. also sharing the verbose output of your client cert would help. - Thanks, Matt
... View more
05-10-2018
02:34 PM
1 Kudo
@Veerendra Nath Jasthi
- Not sure I am following your question. The UI displays the user which is currently authenticated in to the UI. - Are you not expecting to see "CN=niifiadmin, OU=HORTONWORKS"? What are you expecting to see? - What forms of user authentication are you configured to use? User/client certificate? <-- User name displayed in upper right comes from DN in user/client certificate LDAP authentication? <-- User name can come from user DN in LDAP (USE_DN) or username entered in login window (USE_USERNAME) Kerberos authentication? <-- User name comes from user entered principal. <user>@<domain> (so pretty sure you are not using kerberos for authentication.) - Thanks, Matt
... View more
05-10-2018
01:07 PM
@Mustafa Ali Qizilbash *** Forum Tip: please try to avoid responding to an existing answer by starting a new answer. Instead use "add comment" to respond an answer. - The Firefox version you are trying to use is to old. https://wiki.mozilla.org/Release_Management/Calendar - Firefox 17.0.10 was released back on October 29, 2013. The developer tools are very likely going to show some JS issues here. You need to upgrade to a newer version of Firefox or switch to a new version of another browser like Chrome. - Thank you, Matt
... View more
05-09-2018
07:35 PM
@Mustafa Ali Qizilbash Do you experience same issue with a different browser? What version of Firefox and NiFi are your using? - Have tried using the developer tools (Network) available in the Firefox browser to check for any Java Script errors? From Firefox menu in upper right corner select "web developer" and then "network". A new window will open. Then try reloading NiFi page. You should be able to see where it stalls on loading the page. - Thanks, Matt
... View more
05-09-2018
07:28 PM
@Prakhar Agrawal Then your issue is going to be the backend validation that is going on. How much validation that is occurring depends on what you are doing. Shu's recommendation for disabling processors that are not running should help a great deal. - There is no such thing as "Max Cron Driven Thread Count". I am going to assume you meant "Max Event Driven Thread count." There is nothing you will add to your canvas that will use the "Event Driven" scheduling strategy. You physically need to change the configuration on a component to use "Event Driven". Event Driven is consider experimental and deprecated in favor of advance made with the Timer driven Scheduling strategy. By setting "Max Event Driven Thread count" to 50 you are creating a pool of 50 reserved threads. I recommend not using "Event Driven" Scheduling strategy anywhere and reducing the max Event Driven thread count setting to "1". Reducing the "Max Event driven thread count" setting will require a NiFi restart.
... View more
05-09-2018
05:26 PM
2 Kudos
@David Miller Sorry for leaving out some of the details. The ListSFTP and FetchSFTP processors where originally developed and added to NiFi back in Apache NiFi 0.4.0. The 0.x versions of NiFi did not use zookeeper for cluster or state. ZK was introduced with the major redesign work that went into Apache NIFi 1.x. - To avoid issues for users upgrading from 0.x to 1.x+ versions of NiFi, the distributedMapCache properties remained in all the state keeping processors created prior to 1.x release. NiFi 1.x and newer version will automatically store state in ZK. Having the DistributeMapCache configured gives users who had state previously stored in a cache server to be read and NiFi would then write that state to ZK moving forward. - So in newer versions (NiFi 1.x +), there is no need to use the distributeMapCache property. - Thanks, Matt
... View more