@Muhammad Umar When NiFi starts and has not been configured with a specific hostname or IP in the (nifi.web.http.host=) in the nifi.properties file, it looks to bind to the IP address registered to every NIC card present on the host system. If you try to specify a hostname or IP that does not resolve or match the IP registered to any of your NIC cards, NiFi will fail to start. NiFi can not bind to a port that belongs to an IP it does not own. You can run the "ifconfig" command on the host running NiFi to see all NICs and the IP registered to them. You should see the 172.17.x.x address and not the 192.168.x.x address shown. It definitely sounds like there is some network address translation going on here. The fact that you can reach NiFi over http://192.68.x.x:8078// confirms this. It is simply routing all traffic from the 192.169.x.x address to the internal 172.17.x.x address. We confirmed already your browser cannot resolve a path directly to 172.17.x.x, because if you could, NiFi's UI would have opened. NiFi is in fact bound to 172.17.x.x and not 192.168.x.x. NiFi cannot control how traffic is being routed to this endpoint by the network. Thanks, Matt ... View more

@Gaurav Jain If the NiFi node suddenly goes down, how is it going to notify the other nodes of this? If the node goes down, the result of the job is neither a failure or success as NiFi defines it. The FlowFile that triggers your SparkJobExecutor should remain tied to the incoming connections until the job successfully completes or reports a failure. At that time the FlowFile would be moved to one of the corresponding relationship. If the NIFi node goes down, when it comes back up FlowFiles are restored to the last known connection. This means this FlowFile will trigger your SparkJob executor to run again. Are you looking for a way to notify another node that the last spark job did not complete only or are you also looking for a way for that other node to then run the job? That becomes even more difficult since you must also tell the node that went down not to run again next time it starts back up. As far as the notification goes, you might be able to build a flow using the new wait and notify processor just released in Apache NiFi 1.2.0. You could send a copy of your FlowFile to another one of your nodes before executing the Spark job and then send another FlowFile after job completes. On the other node it should receive the FlowFile and send it to a wait processor. The wait processor can be configured for a time limit. Should that time expire the FlowFile gets routed to expired relationship which can use run the job again on that node or simply send out a email alert. If the job completes before expiration time occurs, a FlowFile could be send to same node to notify successful completion which will cause wait processor to rout FlowFile to success relationship which you may choose to just terminate. Here is the doc for these new processors: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.2.0/org.apache.nifi.processors.standard.Wait/index.html https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.2.0/org.apache.nifi.processors.standard.Notify/index.html Bottom line, NiFi has no behind the scenes monitoring capability to accomplish what you are trying to do here, so a programmatic dataflow design must be used to meet this need. Now if you are talking about a node simply becoming disconnected from the cluster, that is a different story. Just because a node disconnects does not mean it shuts down or stops running its dataflows. It will continue to run as normal and constantly attempt to reconnect. Thanks, Matt ... View more

Both your NiFi clusters can use teh same zookeeper, but you need to make sure each cluster is configured to use a different ZK root node. The root node is set in the nifi.properties file and the state-management.xml file. ... View more

@frank chen The CN for your certificate should match the FQDN of the server where you installed NiFi. Using localhost in a certificate is never a good idea from a security standpoint. Alternatively you can create a certificate that uses SAN names. These SAN names should be DNS resolvable hostnames (With a SAN entry that matches the FQDN of the server the CN does not then need to contain the FQDN. While it is possible to add a security exception in your browser for this bad server cert, you will not be able to do this should you standup a NiFi cluster where the nodes talk securely to one another. I suggest using the toolkit to generate a certificate that uses the servers FQDN as both its CN and as a Subject Alternative Name (SAN) DNS entry. That aside, where did you get your user certificate that is being used to authenticate you as a user to access NiFi? You can use the tls-tookit to create a user certificate as well which you will need to load in your browser. Alternatively, you could configure NiFi to use an external LDAP server or kerberos for user authentication. When you access a secured NiFi instance/cluster URL, the server looks for a valid user certificate it can trust in the request. If no client cert is presented to authenticate with, NiFi will check to see fi any other authentication method has been configured in NiFi. If none have been configured the connection is closed. So your issue is one of the following: 1. Your browser is not have a client (user) certificate loaded to pass to NIFi for authentication. 2. You have a user certificate but it is not trusted by your NiFi instance/cluster. The entries in the NiFi truststore.jks are used to trust the client certificates presented. The keystore typically includes a bunch Certificate authority (CA) trustedCertEntries. It may also contain the public keys of self-signed certificates as trustedCertEntries. If you found this answer addressed your question, please don't forget to mark it "accepted". Thank you, Matt ... View more

@Muhammad Umar Both ports 8078 and 8079 are likely not being forwarded by your HDP sandbox and will need to be added. https://hortonworks.com/hadoop-tutorial/sandbox-port-forwarding-guide/ Thanks, Matt ... View more

@bhumi limbu Is this error followed by a stack trace in the nifi-app.log? ... View more

@Muhammad Umar If the following command shows NiFi is still running: ./nifi.sh status and the following command shows NiFi listening on port 8079: netstat -ant|grep LISTEN Then the issue is not with NiFi. There is something external to NiFi blocking connections to port 8079 from your host where your browser is running. What OS version is running on the server/VM where NiFi is running? Thanks, Matt ... View more