question Re: SMART SENSE IMPLEMENTATION in Archives of Support Questions (Read Only)

SMART SENSE IMPLEMENTATION

suresh_b_k — Tue, 10 May 2016 20:49:11 GMT

Hi Team,

Iam Implementing Smart Sense on my Cluster. I need your help before i start

I have to change the below parameters before i implement Smart sense. Can someone please explain the dependency property also require for the parent property

Value needs to change for the Property

dfs.namenode.safemode.threshold-pct

Dependency Property which is not in my cluster dfs.namenode.replication.min. should this property is must ?

Re: SMART SENSE IMPLEMENTATION

pminovic — Tue, 10 May 2016 21:10:59 GMT

As far as I know there is no need to change anything in your cluster in ordert to install SmartSense. The properties you mentioned have completely acceptable defaults:

dfs.namenode.safemode.threshold-pct=0.999f
dfs.namenode.replication.min=1

I usually set threshold-pct to 0.99f but 0.999 is fine as well. One of the purposes of SmartSense is to give you advice on your setup, so you can just go ahead, install it and try it.

Re: SMART SENSE IMPLEMENTATION

suresh_b_k — Tue, 10 May 2016 21:11:43 GMT

Thanks a lot. So i need to update above parameter on hdfs-site.xml. so do i need to stop, HIVE, HBASE & other components in cluster.. or just stopping HDFS and YARN , updating the property and restarting will work. Please advice

Re: SMART SENSE IMPLEMENTATION

pminovic — Tue, 10 May 2016 21:19:46 GMT

No, they are defaults, they are valid even if you haven't specified them in your hdfs-site file. No need to change anything, just go ahead and install SmartSense.

Re: SMART SENSE IMPLEMENTATION

suresh_b_k — Tue, 10 May 2016 21:23:50 GMT

Correct, But i have to change the value to 0.999f, currently i have value 1 for

dfs.namenode.safemode.threshold-pct=1 And i don't have this value on my cluster dfs.namenode.replication.min=1, so i have to add that as well.. that is why i have raised that question.

Re: SMART SENSE IMPLEMENTATION

suresh_b_k — Tue, 10 May 2016 21:50:42 GMT

Shall i update the below configuration in ambari hdfs-site.xml file ? which will update on HA NN and all DN's correct

dfs.namenode.safemode.threshold-pct=0.999f
dfs.namenode.replication.min=1

any suggetions. I don't chef or puppet.

Re: SMART SENSE IMPLEMENTATION

pminovic — Tue, 10 May 2016 22:26:27 GMT

Okay, then set "dfs.namenode.safemode.threshold-pct=0.999f" in Ambari, that's all you need to do.

Re: SMART SENSE IMPLEMENTATION

suresh_b_k — Wed, 11 May 2016 15:59:34 GMT

H Predrag, I also need to add dfs.namenode.replication.min=1 property right (which i dont have on my clueter )

Re: SMART SENSE IMPLEMENTATION

suresh_b_k — Wed, 11 May 2016 19:20:53 GMT

I need to update the below parameters in my cluster. Hence updating hdfs-site.xml and yarn-site.xml on Ambari server is enough ? Will the below change will update on all the NN's and DN's after service restart ?

NOTE: DO i need to restart the entire cluster components or just HDFS and Yarn is enough

dfs.namenode.safemode.threshold-pct = 0.999f

ha.zookeeper.acl = sasl:nn:rwcda

dfs.datanode.max.transfer.threads = 16384

yarn.nodemanager.vmem-pmem-ratio = 2.1

yarn.resourcemanager.recovery.enabled = true

yarn.resourcemanager.work-preserving-recovery.enabled = true

Please advice

Re: SMART SENSE IMPLEMENTATION

pminovic — Wed, 11 May 2016 19:25:28 GMT

After you restart HDFS and Yarn, Ambari will show you which other services to restart, like MapRed, Hive and Oozie.

Re: SMART SENSE IMPLEMENTATION

suresh_b_k — Wed, 11 May 2016 20:07:24 GMT

One last question, I don't have this parameter on my cluster. dfs.namenode.replication.min=1 . Is it mandatory to have it on the cluster..

Re: SMART SENSE IMPLEMENTATION

pminovic — Wed, 11 May 2016 21:03:04 GMT

To repeat, 1 is the default, see defaults here, but just to be sure you can go ahead and set it directly in hdfs-site, and restart HDFS and dependent services.

Re: SMART SENSE IMPLEMENTATION

suresh_b_k — Thu, 12 May 2016 14:10:44 GMT

I want to add the below parameter in hdfs-site.xml."""" Property name: ha.zookeeper.acl Value = sasl:nn:rwcda """ since its belong to zookeer do i need to restart the zookeerper service ?

Re: SMART SENSE IMPLEMENTATION

suresh_b_k — Fri, 13 May 2016 16:02:55 GMT

Thanks for the information. as per your above reply 0.99f but 0.999 is the value. just want to make sure. 0.99 is also a correct value. ?