I used Microsoft SSIS to create a file in HDFS using WebHDFS but it failed.
To get better error message, I tested with pyhdfs. I was able to create and list a directory but I can't create a file inside the directory. Following is my sandbox version and test. Could anyone help me?
[root@sandbox-hdp ~]# sandbox-version Sandbox information: Created on: 01_02_2018_10_47_41 Hadoop stack version: Hadoop 184.108.40.206.6.4.0-91 Ambari Version: 220.127.116.11-143 Ambari build: Release : 143 Java version: 1.8.0_161 OS Version: CentOS release 6.9 (Final)
Test and error:
>>> fs = pyhdfs.HdfsClient(hosts='sandbox.hortonworks.com:50070', user_name='root')
requests.exceptions.ConnectionError: HTTPConnectionPool(host='sandbox-hdp.hortonworks.com', port=50075): Max retries exceeded with url: /webhdfs/v1/mytestdir/a.txt?op=CREATE&user.name=root&namenoderpcaddress=sandbox-hdp.hortonworks.com:8020&createflag=&createparent=true&overwrite=false (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x000001FF2A116470>: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond',))
It was my fault.
I had to add sandbox-hdp.hortonworks.com for 127.0.0.1 in hosts file because in case of creating files, it uses sandbox-hdp.hortonworks.com. Actually it was in the manual of Sandbox on a VM but I missed it.