Support Questions

Find answers, ask questions, and share your expertise
Celebrating as our community reaches 100,000 members! Thank you!

Curl command- Connected httpfs/webhdfs but not getting output


Ran below curl commands, able to connected but not getting results. Just its throwong letter P along with special characters.
Env is Cloudera 6.1.2.
Want to use httpfs to put the files in HDFS.
Please suggest.


curl -i -X PUT -T /home/CORP/isuy/Ganesh.txt "http://aalhauap2u01:14000/hdfs:////user/isuy/testfiles?op=CREATE"

curl -i "http://aalhauap2g01:14000/webhdfs/v1/user/"

curl -i "http://aalhauap2u02:14000/webhdfs/v1/user/" -v
* About to connect() to aalhauap2u02 port 14000 (#0)
* Trying
* Connected to aalhauap2u02 ( port 14000 (#0)
> GET /webhdfs/v1/user/ HTTP/1.1
> User-Agent: curl/7.29.0
> Host: aalhauap2u02:14000
> Accept: */*
* Connection #0 to host aalhauap2u02 left intact




Hi Sree,


I'm not entirely sure of the syntax you're using for your curl request.

The HttpFS role still uses the same API syntax as the WebHDFS API.


If you haven't already, I would suggest reviewing the API method for creating and writing to a file via the WebHDFS API[1]. 

Your command, when executed correctly, should look similar to the following:

curl -i -k -X PUT ""


The HDFS service should then spit back a URL to follow through to actually upload your file to. In the above example, that returned looked like this:

HTTP/1.1 307 Temporary Redirect
Date: Thu, 18 Apr 2019 22:23:31 GMT
Cache-Control: no-cache
Expires: Thu, 18 Apr 2019 22:23:31 GMT
Date: Thu, 18 Apr 2019 22:23:31 GMT
Pragma: no-cache
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
WWW-Authenticate: Negotiate YGwGCSqGSIb3EgECAgIAb10wW6ADAgEFoQMCAQ+iTzBNoAMCARCiRgRE/UEMlOPP/tGaYJaWJMk3AyJMlMu9cHklguw/oEKRQwULVYvOiRhJAaTHo8FL9x3Lqhn/F4XpCscNa8IWg/LCflyYci8=
Set-Cookie: hadoop.auth="u=hdfs&p=hdfs@YOUR.DOMAIN.COM&t=kerberos-dt&e=1555662211059&s=TWQoO1FJ/fNNEHxLTNOyIivzsEieJ46M1oUv+DzB7OY="; Path=/; Secure; HttpOnly
Location:             <======*THIS IS THE URL WE WRITE TO*
Content-Type: application/json;charset=utf-8
Content-Length: 0


At that point, we actually submit the payload against the returned path from HDFS:

curl -i -k -H "Content-Type:application/octet-stream" -X PUT -T file.txt ""

And the data shows up:

hdfs dfs -cat /tmp/file.txt
>>this is a file





Thank you, it helps