Hi, there are many log files in /var/log/hadoop/hdfs, and file name like hdfs-audit.log.2018-06-01, and some file name like hadoop-hdfs-namenode-hostname are very large. may i delete it all? (this host running namenode, ranger, datanode flume and node manager.) and if any configure can setting those log as auto cycling or auto delete? Thanks ... View more

Hi, i added 2 new data node to hdfs cluster and in the same rack, will data auto move to new data node? my old data node space are 10T per node, new data node space are 20T per node, when old data node space full, new data node only 50% usage right? or will hdfs put more data to large data node? or i need manually excute hdfs balancer when old data node full? Thanks. ... View more

Hi everyone, i want decommision datanode and keep other service (prevent disk space not enough), in the server have Hive metastore/hiveserver2/flume/node manager service, i want keep these services. how to do? may i also remove node manager? Thanks. ... View more

Hi All: when curl via knox i only can use admin (-u admin:admin-password) to access and can't use other account or will reply : HTTP/1.1 401 Unauthorized Date: Fri, 13 Oct 2017 05:45:38 GMT Set-Cookie: rememberMe=deleteMe; Path=/gateway/default; Max-Age=0; Expires=Thu, 12-Oct-2017 05:45:38 GMT WWW-Authenticate: BASIC realm="application" Content-Length: 0 Server: Jetty(9.2.15.v20160210) my command: curl -i -k -u user1:Hadoop -X PUT 'https://knoxHost:8443/gateway/default/webhdfs/v1/user1/senfile1?op=CREATE' folder permission: drwxr-xr-x - user1 hdfs 0 2017-10-05 11:08 /user1 Knox users-ldif: # entry for user1 dn: uid=user1,ou=people,dc=hadoop,dc=apache,dc=org objectclass:top objectclass:person objectclass:organizationalPerson objectclass:inetOrgPerson cn: user1 sn: user1 uid: user1 userPassword:Hadoop Ranger (Sync Source is Unix) HDFS config: add user1 to default all-path policy Ranger knox config: add user1 to default all-topology, service policy if any wrong in my config? ... View more

Hi All, to fix knox rest api 404 not found error, i modify WEBHDFS url from {{webhdfs_service_urls}} to http://My_NameNode:50070/webhdfs, it's worked. then i using Ranger to control HDFS access permission, and configure Ranger HDFS and Knox plug-in... i run curl command again, will display 307 Temporary Redirect! i research about 307 message says maybe have name node HA and need add other name node url to Knox WEBHDFS setting, but this is not match my ambari system because i have only one name node (no HA), if any setting missing in Ranger or Knox? ... View more

Hi All, i want use curl command to access HDFS via Knox, but always response HTTP/1.1 404 Not Found error. command example: curl -u admin:admin-password -i -v -k "https://KnoxHostName:8443/gateway/default/webhdfs/v1/user/hdfs&op=GETFILESTATUS" and i know how to use curl to create/modify folder or file when url host is Name node with 50070 port, just don't know how to via knox, if any other configs need to setting? My Knox version is 0.12 Thanks. ... View more

Hi All: i'm using ambari 2.52 and HDP 2.6 by repo install (not sandbox), i created a new lab only start HDFS/YARN/MapReduce for knox test, and add new host only install knox, now, i can access ambari manager web via knox (i have add AMBARIUI config in advanced topology), but when click file view (or other view) will get 404 not found error, try to use curl command to access hdfs folder also get 404 not found error, if any other configure need setting? ... View more