Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Nifi Processor PutHDFS not working to write files on HDFS Filesystem

Nifi Processor PutHDFS not working to write files on HDFS Filesystem

New Contributor

Hi,

I am struggling with an error. I am trying to write a file on HDFS from NiFi but its failing with error below. I am using Hortonworks Sandbox(HDP) and NiFi(HDF) both hosted on ESXi server.

Caused by: org.apache.hadoop.ipc.RemoteException: File /user/nifi/.temp1.txt could only be replicated to 0 nodes instead of minReplication (=1). There are 1 datanode(s) running and 1 node(s) are excluded in this operation.

Checklist:

1. Lot of space available in DFS, means HDFS health is in good state. I am able to execute hadoop fs -put <filename> /tmp and write considerable file of good size onto HDFS.

2. Property set to true <property> <name>dfs.client.use.datanode.hostname</name> <value>true</value> </property>

3. hdfs-site.xml,core-site.xml copied to HDF

4. HDP hostname entry copied to /etc/hosts on NiFi Machine

1 REPLY 1

Re: Nifi Processor PutHDFS not working to write files on HDFS Filesystem

New Contributor

Anyone who has encountered a similar problem. I noticed that Nifi is also not able to read data also(through GetHDFS). The problem which I am anticipating is Nifi Client is able to connect to NameNode but its not able to connect to Data Node since NameNode sends it a invalid private address (I am using using Hortonworks Sandbox) and that's why NiFi client is unable to read/write data from HDP

Don't have an account?
Coming from Hortonworks? Activate your account here