Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created on
09-27-2019
07:21 AM
- last edited on
09-28-2019
02:13 AM
by
ask_bill_brooks
Hi,
We are using CDH 5.14. Webhdfs is enabled by default on cloudera manger and i am able to list HDFS data through webhdfs protocol.
[root@hadoop-slave-1 ~]# curl -i -L "http://172.16.0.10:50070/webhdfs/v1/?op=LISTSTATUS"
HTTP/1.1 200 OK
Cache-Control: no-cache
Expires: Fri, 27 Sep 2019 13:52:56 GMT
Date: Fri, 27 Sep 2019 13:52:56 GMT
Pragma: no-cache
Expires: Fri, 27 Sep 2019 13:52:56 GMT
Date: Fri, 27 Sep 2019 13:52:56 GMT
Pragma: no-cache
Content-Type: application/json
X-FRAME-OPTIONS: SAMEORIGIN
Transfer-Encoding: chunked
{"FileStatuses":{"FileStatus":[
{"accessTime":0,"blockSize":0,"childrenNum":2,"fileId":33381,"group":"supergroup","length":0,"modificationTime":1569326622576,"owner":"hdfs","pathSuffix":"cli","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"},
{"accessTime":0,"blockSize":0,"childrenNum":10,"fileId":17365,"group":"supergroup","length":0,"modificationTime":1569589022824,"owner":"hdfs","pathSuffix":"hadoop","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"},
{"accessTime":0,"blockSize":0,"childrenNum":5,"fileId":33483,"group":"supergroup","length":0,"modificationTime":1569331210359,"owner":"hdfs","pathSuffix":"restorefromcli","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"},
{"accessTime":0,"blockSize":0,"childrenNum":3,"fileId":33489,"group":"supergroup","length":0,"modificationTime":1569331240068,"owner":"hdfs","pathSuffix":"restorefromcli1","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"},
{"accessTime":0,"blockSize":0,"childrenNum":4,"fileId":33645,"group":"supergroup","length":0,"modificationTime":1569331744422,"owner":"hdfs","pathSuffix":"restorefromcli_1","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"},
{"accessTime":0,"blockSize":0,"childrenNum":10,"fileId":33346,"group":"supergroup","length":0,"modificationTime":1569589022883,"owner":"hdfs","pathSuffix":"restorehdfs","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"},
{"accessTime":0,"blockSize":0,"childrenNum":3,"fileId":16386,"group":"supergroup","length":0,"modificationTime":1568894436274,"owner":"hdfs","pathSuffix":"tmp","permission":"1777","replication":0,"storagePolicy":0,"type":"DIRECTORY"},
{"accessTime":0,"blockSize":0,"childrenNum":5,"fileId":16387,"group":"supergroup","length":0,"modificationTime":1569324647144,"owner":"hdfs","pathSuffix":"user","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"}
But i am not able to see any config related to webhdfs in hdfs-site.xml .My concern is that, if this value(dfs.webhdfs.enabled) is enabled in cloudera, then it should also reflect in hdfs-site.xml file.
do anybody have any idea?
Thanks
Devendra
Created on 09-27-2019 07:32 AM - edited 09-27-2019 07:34 AM
As per Hadoop Doc:
https://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml
The default configuration for "dfs.webhdfs.enabled" is "true". Which means if you do not see it in hdfs-site.xml means it is by default enabled. You can override it by editing the "hdfs-site.xml" based on your requirement.
| dfs.webhdfs.enabled | true | Enable WebHDFS (REST API) in Namenodes and Datanodes. |
Created on 09-27-2019 07:32 AM - edited 09-27-2019 07:34 AM
As per Hadoop Doc:
https://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml
The default configuration for "dfs.webhdfs.enabled" is "true". Which means if you do not see it in hdfs-site.xml means it is by default enabled. You can override it by editing the "hdfs-site.xml" based on your requirement.
| dfs.webhdfs.enabled | true | Enable WebHDFS (REST API) in Namenodes and Datanodes. |
Created 09-28-2019 04:02 AM
Thanks.. Jsen.