Hello, I have upgraded HDP from 2.5.3 to 2.6.2.14 Getting benefits of Hive 2.1.0 was one of the motivation points in my upgrade plan. However, even i kept myself optimistic every step i did not see it Hive 2 was not there as upgrade finalised eventually. Furthermore, it is not there as I want to add new services. What am i missing? Info: Centos 7.3.1611 Cluster of 4 VMs Ambari was upgraded from 2.4.2 to 2.5.2 Services able to add: Ranger 0.7.0, Ranger KMS 0.7.0, Druid 0.10.1, Storm 1.1.0 Services that i expected to see but missing: Apache Calcite 1.2.0, Apache DataFu 1.3.0, Apache Hive 2.1.0, Apache Phoenix 4.7.0, Cascading 3.0.0, Hue 2.6.1 ... View more

Hello, I am checking JMX metrics with a period of time to monitor the cluster health. When I try to check my monitoring platform I saw that it is too late to update. The case is a dead datanode. I stop one of the datanode services on Ambari and expect to see below data to change from 0 to 1: http://namenodeaddress:50070/jmx?qry=Hadoop:service=NameNode,name=FSNamesystemState { "name" : "Hadoop:service=NameNode,name=FSNamesystemState", "modelerType" : "org.apache.hadoop.hdfs.server.namenode.FSNamesystem", ... "NumDeadDataNodes" : 0, ... ... } It was updated 6 minutes later. It is a very long time to take an action. However when I start the service again, it is updated from 1 to 0 as soon as service was started. Can someone check it for me if this is the normal update time? PS: I know Ambari is faster to detect. Probably it uses another method to detect dead nodes. I need to check this to continue parsing other metrics. Thanks in advance. ... View more

Hi, To clarify the question I will illustrate the case; Lets name datanodes like; [dnode1, dnode2, dnode3, dnode4, dnode5, dnode6, dnode7, dnode8, dnode9] I don't want one block to be replicated among dnode1, dnode2 and dnode3 because I have to turn off these 3 at once in case of maintenance. Is there any such replication setting in hdfs so as to specify replication targets instead of random nodes? Like, replication group definition? ... View more