Not sure what your plan is? If you decommission a data node avoiding that the rebalancing takes places could lead to data loss. For sure it will leave some file chunks without any redundant storage. So either you are able to delete some data on your HDFS to allow the rebalancing to suceed, or you add some capacity (i.e. with a new temporary node) to hdfs before decommissioning the data node. ... View more

To delete /spark2-history: hdfs dfs -rm -r /spark2-history/* ... View more

in Ambari, go to the host details, and there you can click on the button right to the 'Datanode HDFS' service line .screenshot-decomission.png You should turn on maintenance mode before to avoid alerts. ... View more

for the trash dir, try also to execute the command without the / at the end. ... View more

One question: have you performed an upgrade of HDFS? You may also want to check with: hdfs fsck / -includeSnapshots ... View more

the message could be caused by a process still or already accessing the file. Try to check if this is the case by: lsof | grep /opt/app/data11/hadoop/hdfs/data/current/BP-441779837-135.208.32.109-1458040734038 The first three columns are: command process id user If there is a process locking the file, this should help you to identify it. ... View more

It allows you to run brokers of different versions in one cluster, to avoid a downtime of the cluster during the upgrade. Before you upgrade any broker, you set the inter.broker.protocol.version to the existing version on all brokers. Then you start upgrading broker by broker, the newer broker will still use the 'old' protocol to communicate with the other brokers. This keeps the cluster functional during the time when just some brokers are updated. Once all brokers are upgraded, you change the inter.broker.protocol.version to the new version and restart them one by one. More details here: https://kafka.apache.org/documentation/#upgrade ... View more

@Jay Kumar SenSharma : Thanks for your answer, I simply wasn't aware that the process will change directory permissions, the only reason i used root to start it was that I tried to make sure that any issue I experience wasn't due to lacking permissions. In the meantime the service has stopped itself: [root@cgihdp4 ~]# ambari-metrics-collector status AMS is not running. [root@cgihdp4 ~]# su - ams [ams@cgihdp4 ~]$ ambari-metrics-collector status AMS is not running. [ams@cgihdp4 ~]$ ambari-metrics-collector start tee: /var/log/ambari-metrics-collector/ambari-metrics-collector-startup.out: Permission denied Sun Feb 4 14:31:19 CET 2018 Starting HBase. tee: /var/log/ambari-metrics-collector/ambari-metrics-collector-startup.out: Permission denied master is running as process 23182. Continuing master running as process 23182. Stop it first. tee: /var/log/ambari-metrics-collector/ambari-metrics-collector-startup.out: Permission denied Verifying ambari-metrics-collector process status... Sun Feb 4 14:31:21 CET 2018 Collector successfully started. Sun Feb 4 14:31:21 CET 2018 Initializing Ambari Metrics data model ... [ams@cgihdp4 ~]$ ambari-metrics-collector status AMS is running as process 22414. I guess the permission denied is caused by what you just pointed out, so I will change this again, but I am confused about 'master is running as process 23182', which is the Hbase Master, running with user 'ams', but does it indicate an issue now? Otherwise nothing changed now, still no process listening to port 6188 ... View more