Under "NameNode High Availability" of the Apache Ambari Operations document, one of the steps was to adjust the ZooKeeper Failover Controller retries setting for your environment. We were to "Set ha.failover-controller.active-standbyelector. zk.op.retries=120." (Actually this was already done automatically when we "Enable NameNode HA"). My customer is curious to know the unit for the default ha.failover-controller.active-standbyelector.zk.op.retries setting? Is it 120 retries or 120 sec(or ms) between retries? ... View more

I have recently completed a HDP 2.6.4 setup using Ambari Cluster Setup Wizard with various services distributed to various nodes like: WebHCat Server HiveServer2 Hive Metastore Infra Solr Instance Grafana Metrics Collector Activity Explorer Activity Analyzer HST Server ZooKeeper Server ResourceManager History Server Spark2 History Server App Timeline Server Livy for Spark2 Server Spark2 Thrift Server The installation wizard completed successfully with warning. I was able to clear most of the warnings by restarting or starting services except for Hive Metastore. (For errors, please see attachment). I would like to know if there are documents on tests to verify the successful installation of each of the above services. How do we assure the customer that the HDP is successfully setup? For the warning errors for Hive Metastore, do I need to perform the installation again since the log says something about "Failed to download file from....". I suspect the error is caused by the customer changing the hostname of the management node prior to the Cluster setup using Ambari setup wizard. ... View more

Excluding OS disks(RAID1), I have 3 other disks in management, NameNode and Secondary NameNode and 6 other disks in DataNode 1-3. They are mounted as shown in the table below: Management Node NameNode Secondary NameNode XFS XFS XFS Mount Point Mount Point Mount Point /grid/1 /grid/1 /grid/1 /grid/2 /grid/2 /grid/2 /grid/3 /grid/3 /grid/3 DataNode 1 DataNode 2 DataNode 3 XFS XFS XFS Mount Point Mount Point Mount Point /grid/1 /grid/1 /grid/1 /grid/2 /grid/2 /grid/2 /grid/3 /grid/3 /grid/3 /grid/4 /grid/4 /grid/4 /grid/5 /grid/5 /grid/5 /grid/6 /grid/6 /grid/6 Question1 Do I specify the following under "Customize Services" section for HDFS: NameNode Directories: /grid/1/hadoop/hdfs/namenode, /grid/2/hadoop/hdfs/namenode, /grid/3/hadoop/hdfs/namenode DataNode Directories: /grid/1/hadoop/hdfs/data, /grid/2/hadoop/hdfs/data, /grid/3/hadoop/hdfs/data, /grid/4/hadoop/hdfs/data, /grid/5/hadoop/hdfs/data, /grid/6/hadoop/hdfs/data Question2 Will the Ambari wizard configure the same namenode directories for the secondary namenode? Question3 Is it OK for me to specify the following directories under YARN Section for all DataNodes: For yarn.nodemanager.localdirs: /grid/1/hadoop/yarn/local, /grid/2/hadoop/yarn/local, /grid/3/hadoop/yarn/local, /grid/4/hadoop/yarn/local, /grid/5/hadoop/yarn/local, /grid/6/hadoop/yarn/local For yarn.nodemanager.logdirs: /grid/1/hadoop/yarn/log, /grid/2/hadoop/yarn/log, /grid/3/hadoop/yarn/log, /grid/4/hadoop/yarn/log, /grid/5/hadoop/yarn/log, /grid/6/hadoop/yarn/log Question4 If I have assigned the following services to the management node: WebHCat Server HiveServer2 Infra Solr Instance* Grafana Metrics Collector* Activity Explorer* Activity Analyzer* HST Server NFS Gateway (Assign Slaves and Clients Section) Client (Assign Slaves and Clients Section) Any recommendations on how I should best utilize the 3 disks(i.e: /grid/1, /grid/2, /grid/3) in the management node? Question5 I have the following services under namenode and secondary namenode: NameNode SNameNode ResourceManager ZooKeeper Server* History Server ZooKeeper Server* Spark2 History Server Hive Metastore App Timeline Server Will the secondary namenode be assigned all the services for primary namenode besides Zookeeper? Any concerns with the above services assigned to namenode? ... View more