@doron zukerman You can remove "--privileged" if you don't intend to use Kerberos. By default, Docker containers are “unprivileged” and cannot, for example, run a Docker daemon inside a Docker container. This is because by default a container is not allowed to access any devices, but a “privileged” container is given access to all device. Please see: https://docs.docker.com/engine/reference/run/#/runtime-privilege-and-linux-capabilities ... View more

@Volodymyr Ostapiv You may want to refer to the following article which talks about a new feature added to ambari 2.4 in order to have dynamic auto-recovery, which allows auto-start properties to be configured without needing an ambari-agent / ambari-server restart. https://community.hortonworks.com/content/kbentry/71748/how-do-i-enable-automatic-restart-recovery-of-serv.html Example: {"recovery_enabled":"true"} curl -u admin:<password> -H "X-Requested-By: ambari" -X PUT 'http://localhost:8080/api/v1/clusters/<cluster_name>/components?ServiceComponentInfo/component_name.in(HBASE_REGIONSERVER)' -d '{"ServiceComponentInfo" : {"recovery_enabled":"true"}}' In the above curl call you can define your own service component. Above will help in setting auto start in case of host reboot. Also for your custom service you may customize your scripts like "package/scripts" to have your own desired start feature. - You can take a look at the "metainfo.xml" file of AMS something like following 'recovery_enabled': <component> <name>METRICS_COLLECTOR</name> <displayName>Metrics Collector</displayName> <category>MASTER</category> <recovery_enabled>true</recovery_enabled> - More info regarding this feature for ambari 2.4.x (and prior versions can be found at: https://cwiki.apache.org/confluence/display/AMBARI/Recovery%3A+auto+start+components ) . ... View more

@Punit kumar Please run the following command to verify the permissions on HDFS directory "/" # su - hdfs # hdfs dfs -stat "%u %g" / hdfs hdfs It should be owned by "hdfs:hdfs" if it is not then you can fix it by running the following command: # hdfs dfs -chwon hdfs:hdfs / . - It is Hue code that is explicitly checking for the default permission : https://github.com/cloudera/hue/blob/9916b2b6389323ad8139d709e248156a7cf943f5/desktop/libs/hadoop/src/hadoop/fs/webhdfs.py#L914-L916 . And the default superuser is defined in "hue.ini" as following: default_hdfs_superuser=hdfs https://github.com/cloudera/hue/blob/ef8812cb2033b628d90f5bffacc4ce7c27374e45/desktop/conf.dist/hue.ini#L62-L63 . ... View more

@Punit kumar Regarding your latest error: java.io.IOException: Incompatible clusterIDs in /mnt/disk1/hadoop/hdfs/data: namenode clusterID = CID-297a140f-7cd6-4c73-afc8-bd0a7d01c0ee; datanode clusterID = CID-7591e6bd-ce9b-4b14-910c-c9603892a0f1 at Looks like your VERSION file has different cluster IDs present in NameNode and DataNode that need to be correct. So please check. cat <dfs.namenode.name.dir>/current/VERSION cat <dfs.datanode.data.dir>/current/VERSION Hence Copy the clusterID from nematode and put it in the VERSION file of datanode and then try again. Please refer to: http://www.dedunu.info/2015/05/how-to-fix-incompatible-clusterids-in.html . ... View more