SMART SENSE IMPLEMENTATION

[#1]

Hi Team,

Iam Implementing Smart Sense on my Cluster. I need your help before i start

I have to change the below parameters before i implement Smart sense. Can someone please explain the dependency property also require for the parent property

Value needs to change for the Property

dfs.namenode.safemode.threshold-pct

Dependency Property which is not in my cluster dfs.namenode.replication.min. should this property is must ?

[#2 · Accepted solution]

As far as I know there is no need to change anything in your cluster in ordert to install SmartSense. The properties you mentioned have completely acceptable defaults:

dfs.namenode.safemode.threshold-pct=0.999f
dfs.namenode.replication.min=1

I usually set threshold-pct to 0.99f but 0.999 is fine as well. One of the purposes of SmartSense is to give you advice on your setup, so you can just go ahead, install it and try it.

[#3]

Thanks a lot. So i need to update above parameter on hdfs-site.xml. so do i need to stop, HIVE, HBASE & other components in cluster.. or just stopping HDFS and YARN , updating the property and restarting will work. Please advice

[#4]

No, they are defaults, they are valid even if you haven't specified them in your hdfs-site file. No need to change anything, just go ahead and install SmartSense.

[#5]

Correct, But i have to change the value to 0.999f, currently i have value 1 for

1. dfs.namenode.safemode.threshold-pct=1 And i don't have this value on my cluster dfs.namenode.replication.min=1, so i have to add that as well.. that is why i have raised that question.

[#6]

Shall i update the below configuration in ambari hdfs-site.xml file ? which will update on HA NN and all DN's correct

1. dfs.namenode.safemode.threshold-pct=0.999f
2. dfs.namenode.replication.min=1

any suggetions. I don't chef or puppet.

[#7]

Okay, then set "dfs.namenode.safemode.threshold-pct=0.999f" in Ambari, that's all you need to do.

[#8]

H Predrag, I also need to add dfs.namenode.replication.min=1 property right (which i dont have on my clueter )

[#9]

I need to update the below parameters in my cluster. Hence updating hdfs-site.xml and yarn-site.xml on Ambari server is enough ? Will the below change will update on all the NN's and DN's after service restart ?

NOTE: DO i need to restart the entire cluster components or just HDFS and Yarn is enough

dfs.namenode.safemode.threshold-pct = 0.999f

ha.zookeeper.acl = sasl:nn:rwcda

dfs.datanode.max.transfer.threads = 16384

yarn.nodemanager.vmem-pmem-ratio = 2.1

yarn.resourcemanager.recovery.enabled = true

yarn.resourcemanager.work-preserving-recovery.enabled = true

Please advice

[#10]

After you restart HDFS and Yarn, Ambari will show you which other services to restart, like MapRed, Hive and Oozie.