Skip to main content
YasBHK
YasBHK
Silver ⭐️⭐️
April 8, 2023
Question

Exception while Loading Data to HDFS from Talend

When I try to put files to HDFS with Talend, it creates the file but the file is empty I get this error :

File /user/hdfs/data/file.xlsx could only be written to 0 of the 1 minReplication nodes. There are 1 datanode(s) running and 1 node(s) are excluded in this operation.

 

I'm really struggling 

hope someone can help 

4 replies

Shelton
Shelton
Gold ⭐️⭐️⭐️
April 9, 2023

@YasBHK 

Please ensure both data nodes (2) are running.
You definitely have an issue with one of the data nodes and because of your replication factor which I guess is 2 from the output the file /user/hdfs/data/file.xlsx can't be persisted if it can't meet the min replication of 2.
Firstly understand why the second data node has been excluded by YARN either  its space related issue or it just isn't started.
Please check the dfs.hosts.exclude location usually in HDP /etc/hadoop/conf/dfs.exclude remove the host in the file and run the below 

hdfs dfsadmin -refreshNodes or from Ambari Ui just run the refresh nodes

refreshnode.PNG

 


That should resolve the issue.
Restart the faulty datanode and your HDFS put command will succeed 

Translate
YasBHK
YasBHKAuthor
Silver ⭐️⭐️
April 9, 2023

first of all, thanks for answering 

I actually Have 1 data node and it is running not 2.

 

I tried to check the file dfs.exclude but it's empty 

Translate
Shelton
Shelton
Gold ⭐️⭐️⭐️
April 9, 2023

@YasBHK 
File /user/hdfs/data/file.xlsx could only be written to 0 of the 1 minReplication nodes. There are 1 datanode(s) running and 1 node(s) are excluded in this operation.
So that means your data node is down can you restart the HDFS service and retry?

Translate
Terms & ConditionsAccessibility statement