How to upload a file to HDFS using Webhdfs rest api

[#1]

I'm looking for Wehdfs Rest api example to upload a file to HDFS. I tried with below Api but could not upload a file to hdfs

curl -i -X PUT "http://<namenode host>:50070/webhdfs/v1/tmp/testa/a.txt?user.name=livy&op=CREATE"

curl -i -T /tmp/a.txt "http://<namenode host>:50070/webhdfs/v1/tmp/testa/a.txt?op=CREATE&overwrite=false"

[#2 · Accepted solution]

Hi @yvora

The above API looks fine. Is the API returning 201 or any other status code. Did you make sure that /tmp/testa path exists and livy has write permission on it.

Thanks,

Aditya

[#3]

@Aditya Sirna, /tmp/testa dir is present. an livy user has permission to write to it.

I received below output while trying to run webhdfs rest api.

[root@xx user]# curl -i -X PUT "http://<namenode host>:50070/webhdfs/v1/tmp/testa/a.txt?user.name=livy&op=CREATE"
HTTP/1.1 307 TEMPORARY_REDIRECT
Cache-Control: no-cache
Expires: Tue, 26 Sep 2017 17:33:17 GMT
Date: Tue, 26 Sep 2017 17:33:17 GMT
Pragma: no-cache
Expires: Tue, 26 Sep 2017 17:33:17 GMT
Date: Tue, 26 Sep 2017 17:33:17 GMT
Pragma: no-cache
X-FRAME-OPTIONS: SAMEORIGIN
Set-Cookie: hadoop.auth="u=livy&p=livy&t=simple&e=1506483197716&s=dRvADKPG0lrenLje4fmEEdgChFw="; Path=/; HttpOnly
Location: http://xxx:50075/webhdfs/v1/tmp/testa/a.txt?op=CREATE&user.name=livy&namenoderpcaddress=xxx:8020&cre...
Content-Type: application/octet-stream
Content-Length: 0
Server: Jetty(6.1.26.hwx)
[root@xxx user]# curl -i -T /tmp/a.txt "http://<namenode host>:50070/webhdfs/v1/tmp/testa/a.txt?op=CREATE&overwrite=false"
HTTP/1.1 100 Continue
HTTP/1.1 307 TEMPORARY_REDIRECT
Cache-Control: no-cache
Expires: Tue, 26 Sep 2017 17:33:49 GMT
Date: Tue, 26 Sep 2017 17:33:49 GMT
Pragma: no-cache
Expires: Tue, 26 Sep 2017 17:33:49 GMT
Date: Tue, 26 Sep 2017 17:33:49 GMT
xRAME-OPTIONS: SAMEORIGIN

Location: http://xxx:50075/webhdfs/v1/tmp/testa/a.txt?op=CREATE&namenoderpcaddress=xx:8020&createflag=&createp...
Content-Type: application/octet-stream
Content-Length: 0
Server: Jetty(6.1.26.hwx)

[#4]

@yvora,

Make the 2nd curl call to the Location which was returned in the first call.

ie., http://xxx:50075/webhdfs/v1/tmp/testa/a.txt?op=CREATE&user.name=livy&namenoderpcaddress=xxx:8020&cre...

[#5]

It worked. Thanks.