Skip to main content
luv4diamonds
luv4diamonds
Silver ⭐️⭐️
April 21, 2023
Question

nifi with external zookeeper errors

I have installed nifi on 3 hosts and zookeeper on one, for now.  In the nifi-app.log, I keep seeing these errors.

org.apache.nifi.priority.Rul;esManager Exception occurred when trying to update rules from teh stateManageProvider.  java.io.  Failed to obtain value from zookeeper for component with ID RulesManagerID with the exception code CONNECTIONLOSS.  I'm running zookeeper-3.4.13 until I can get a later release.  

 

In addition, in my zookeeper.out log, These messages keep rolling in the log:

Accepted socket connection from /IP/port

Exception causing close of sessions 0x0 Len error 369296128

Closed socket connection for client /IP/port (no session established for client)

Any help would be greatly appreciated.

Cathy

 

 

8 replies

DianaTorres
Community Manager
DianaTorres
Community Manager
April 22, 2023

@luv4diamonds Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our NiFi experts @SAMSAL @cotopaul  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

Translate
Regards, Diana Torres, Senior Community Moderator
luv4diamonds
luv4diamondsAuthor
Silver ⭐️⭐️
April 26, 2023

Question was not answered.

Translate
DianaTorres
Community Manager
DianaTorres
Community Manager
April 26, 2023

@MattWho Any insights here by any chance? Thanks!

Translate
Regards, Diana Torres, Senior Community Moderator
cotopaul
cotopaul
Platinum ⭐️
April 27, 2023

@luv4diamonds,

I do not have much experience with zookeeper but if you are running a 3 node nifi cluster and you have installed zookeeper on one of those nodes, why don't you use the embedded zookeeper directly? It would make sense to have your external zookeeper made totally external as right now you are assigning resources to your zookeeper instance as well. For example if you will have a problem with the node on which you have your zookeeper client, the entire cluster will be affected, whereas if you had your zookeeper on a different machine, the other nodes would not be affected. This is just an opinion 🙂

Also, please share some light on what NiFi Version you are using and how you configured your properties? I am talking here about nifi.properties, zookeeper.properties and state-management.xml.

Translate
MattWho
Titanium ⭐️⭐️⭐️
MattWho
Titanium ⭐️⭐️⭐️
April 27, 2023

@luv4diamonds 

NiFi will NOT form a cluster with a single instance Zookeeper.  Zookeeper must have quorum otherwise a cluster coordinator and primary node will not get elected for NiFi.  For Quorum you should have an odd number of ZK nodes with a minimum of 3.

If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.

Thank you,

Matt



Translate
    luv4diamonds
    luv4diamondsAuthor
    Silver ⭐️⭐️
    April 27, 2023

    I'm running nifi-11.14.0.  I switched to embedded on my cluster and I'm still getting connection refused to port 2181 for only one of the nifi's.  In addition, I'm seeing this error.  KeeperErrorCode - ConnectionLoss for /nifi/components/RulesManagerID.  Any other ideas?

    This messages did go away:  Exception causing close of sessions 0x0 Len error 369296128. 

    Also, I can't share my files on here.

    My state-management.xml

    Connect String:  >zoo.mydomain:2181,zoo2.mydomain:2181,zoo3.mydomain:2181<

    zookeepers.properties:

    I have ssl configured and
    server.1-zoo.mydomain:2888:3888;2181

    server.1-zoo2.mydomain:2888:3888;2181

    server.1-zoo3.mydomain:2888:3888;2181

     

    nifi.properties embedded zookeeper: true

    nifi.zookeeper.connect.string=zoo.mydomain:2181,zoo2.mydomain:2181,zoo3.mydomain:2181

    Translate
    cotopaul
    cotopaul
    Platinum ⭐️
    April 28, 2023

    @luv4diamonds 
    On my instances where I am running the embedded zookeeper, I am using port 2182 and not 2181. Maybe you can try and change it like that and test.
    In addition, I assume that you have the file myid generated on every node, right?(https://docs.cloudera.com/HDPDocuments/HDF3/HDF-3.5.1/nifi-state-management/content/embedded_zookeeper.html)

    Translate
    Terms & ConditionsAccessibility statement