Ran below curl commands, able to connected but not getting results. Just its throwong letter P along with special characters.
Env is Cloudera 6.1.2.
Want to use httpfs to put the files in HDFS.
curl -i -X PUT -T /home/CORP/isuy/Ganesh.txt "http://aalhauap2u01:14000/hdfs:////user/isuy/testfiles?op=CREATE"
curl -i "http://aalhauap2g01:14000/webhdfs/v1/user/user.name=cloudera&op=GETFILESTATUS"
curl -i "http://aalhauap2u02:14000/webhdfs/v1/user/user.name=cloudera&op=GETFILESTATUS" -v
* About to connect() to aalhauap2u02 port 14000 (#0)
* Trying 10.91.23.152...
* Connected to aalhauap2u02 (10.91.23.152) port 14000 (#0)
> GET /webhdfs/v1/user/user.name=cloudera&op=GETFILESTATUS HTTP/1.1
> User-Agent: curl/7.29.0
> Host: aalhauap2u02:14000
> Accept: */*
* Connection #0 to host aalhauap2u02 left intact
I'm not entirely sure of the syntax you're using for your curl request.
The HttpFS role still uses the same API syntax as the WebHDFS API.
If you haven't already, I would suggest reviewing the API method for creating and writing to a file via the WebHDFS API.
Your command, when executed correctly, should look similar to the following:
curl -i -k -X PUT "https://your.host.com:14000/webhdfs/v1/tmp/file.txt?op=CREATE"
The HDFS service should then spit back a URL to follow through to actually upload your file to. In the above example, that returned looked like this:
HTTP/1.1 307 Temporary Redirect Date: Thu, 18 Apr 2019 22:23:31 GMT Cache-Control: no-cache Expires: Thu, 18 Apr 2019 22:23:31 GMT Date: Thu, 18 Apr 2019 22:23:31 GMT Pragma: no-cache X-Content-Type-Options: nosniff X-XSS-Protection: 1; mode=block WWW-Authenticate: Negotiate YGwGCSqGSIb3EgECAgIAb10wW6ADAgEFoQMCAQ+iTzBNoAMCARCiRgRE/UEMlOPP/tGaYJaWJMk3AyJMlMu9cHklguw/oEKRQwULVYvOiRhJAaTHo8FL9x3Lqhn/F4XpCscNa8IWg/LCflyYci8= Set-Cookie: hadoop.auth="u=hdfs&p=hdfs@YOUR.DOMAIN.COM&t=kerberos-dt&e=1555662211059&s=TWQoO1FJ/fNNEHxLTNOyIivzsEieJ46M1oUv+DzB7OY="; Path=/; Secure; HttpOnly Location: https://your.host.com:14000/webhdfs/v1/tmp/file.txt?op=CREATE&data=true <======*THIS IS THE URL WE WRITE TO* Content-Type: application/json;charset=utf-8 Content-Length: 0
At that point, we actually submit the payload against the returned path from HDFS:
curl -i -k -H "Content-Type:application/octet-stream" -X PUT -T file.txt "https://your.host.com:14000/webhdfs/v1/tmp/file.txt?op=CREATE&data=true"
And the data shows up:
hdfs dfs -cat /tmp/file.txt >>this is a file