question Re: Exception while Loading Data to HDFS from Talend in Support Questions

Exception while Loading Data to HDFS from Talend

YasBHK — Sat, 08 Apr 2023 21:02:09 GMT

When I try to put files to HDFS with Talend, it creates the file but the file is empty I get this error :

File /user/hdfs/data/file.xlsx could only be written to 0 of the 1 minReplication nodes. There are 1 datanode(s) running and 1 node(s) are excluded in this operation.

I'm really struggling

hope someone can help

Re: Exception while Loading Data to HDFS from Talend

Shelton — Sun, 09 Apr 2023 10:26:15 GMT

@YasBHK

Please ensure both data nodes (2) are running.
You definitely have an issue with one of the data nodes and because of your replication factor which I guess is 2 from the output the file /user/hdfs/data/file.xlsx can't be persisted if it can't meet the min replication of 2.
Firstly understand why the second data node has been excluded by YARN either its space related issue or it just isn't started.
Please check the dfs.hosts.exclude location usually in HDP /etc/hadoop/conf/dfs.exclude remove the host in the file and run the below

hdfs dfsadmin -refreshNodes or from Ambari Ui just run the refresh nodes

That should resolve the issue.
Restart the faulty datanode and your HDFS put command will succeed

Re: Exception while Loading Data to HDFS from Talend

YasBHK — Sun, 09 Apr 2023 10:54:06 GMT

first of all, thanks for answering

I actually Have 1 data node and it is running not 2.

I tried to check the file dfs.exclude but it's empty

Re: Exception while Loading Data to HDFS from Talend

YasBHK — Sun, 09 Apr 2023 10:55:12 GMT

first of all, thanks for answering

I actually Have 1 data node and it is running not 2.

I tried to check the file dfs.exclude but it's empty

Re: Exception while Loading Data to HDFS from Talend

Shelton — Sun, 09 Apr 2023 16:06:52 GMT

@YasBHK
File /user/hdfs/data/file.xlsx could only be written to 0 of the 1 minReplication nodes. There are 1 datanode(s) running and 1 node(s) are excluded in this operation.
So that means your data node is down can you restart the HDFS service and retry?