CFM

[#1]

Hi,

I am using CFM 1.0.1.7, when I am using the remote process group to transfer file between two cluster. There is an error message "Failed to persist link of Peer due to java.io.FileNotFoundException:./conf/state/xxxxxxxxxxx_HTTP.peer (No such file or directory); If restarted and peer's NCM is down, may be unable to transfer data unitil communuication with NCM is restored.

How shall I proceed with this error. The NIFI version is 1.9.0 and according to the online documentation, NCM is only avaliable for NIFI 0.X, what am I seeing this error?

[#2 · Accepted solution]

You might be hitting a known issue where NiFi process is looking for "/conf/state/" directory under NiFi Process directory (/var/run/cloudera-scm-agent/process/*nifi/) created by Cloudera Manager Agent. But the "/conf/state/" directory is not created by default causing "No such file or directory" exception.

This exception should not actually cause any issue with the data flow.

To get rid of the exception please add 'nifi.persistent.state.directory' property to nifi.properties and point it directory that is writeable by nifi

Regards,

Mohd Nadeem Ul Islam

Was your question answered? Make sure to mark the answer as the accepted solution

[#3]

@paygb 

CFM 1.0 is based off apache NiFi 1.9 and was released more than 2 years ago.  I would recommend upgrading to a much newer release.

I am confused about what in the error message leads you to believe it is specific or related to the NiFi Cluster Manager (NCM) which you are correct only existed in the Apache NiFi 0.x versions.  

With Apache NiFI 1.x, NiFi had some major changes which included the introduction of zero master clustering which did away with the NCM.  Zookeeper became a requirement for NiFi clusters so that any node could be elected the role of Cluster Coordinator (CC) at any time.  That CC was responsible for the functions previously handled by the NCM.

Now the issue you are seeing is specific to the deployment setup done specifically by CM for the NiFi service which restricts access to some directory path thus preventing NiFi from being able to create the directory being used by the S2S protocol used by the Remote Process Groups (RPGs).  The following NiFi  improvement was done to address this as of the Apache NiFi 1.10 release:
https://issues.apache.org/jira/browse/NIFI-6598

CFM 1.1 is based off Apache NiFi 1.11.4 along with some additional fixes (https://docs.cloudera.com/cfm/1.1.0/release-notes/topics/cfm-nifi-patches.html) and would include this improvement.

If you found any of the answers provided assisted with your query, please take a moment to login and click on the "Accept as Solution" button below each answer.

Thank you,

Matt

[#4]

Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.