Sure Vidya , Thank you  ... View more

second curl command which is for sessions may not work as there is no property called className in sessions REST API ... View more

It seems they are removed ... View more

Check the heap assigned for the node manager   in the hadoop/etc/hadoop/yarn-env.sh define the heap for node manager   JAVA=$JAVA_HOME/bin/java JAVA_HEAP_MAX=-Xmx256m # For setting YARN specific HEAP sizes please use this # Parameter and set appropriately YARN_HEAPSIZE=256 After that, you can configure the resources in yarn-site.xml <configuration> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> <property> <name>yarn.scheduler.minimum-allocation-mb</name> <value>2000</value> </property> <property> <name>yarn.scheduler.minimum-allocation-vcores</name> <value>1</value> </property> <property> <name>yarn.nodemanager.resource.memory-mb</name> <value>2000</value> </property> <property> <name>yarn.nodemanager.resource.cpu-vcores</name> <value>1</value> </property> </configuration>   please like and confirm if it helps   Thanks Ashish ... View more

@Jona  Great article...may i ask can you use "custom spark2-metrics-properties" in spark config instead of advanced spark2-metrics-properties? ... View more

Hi, I have a question regarding Cluster version. There are separate nodes for primary and secondary machines, so the cluster version will be same or unique in this scenario? ... View more

Hello ,  I have question , i have cloudera cluster how to configure this kafka-handler within the cluster in order to integrate with hiv e , spark , and the other services in the cluster? is there special configuration from cloudera manager ?? ... View more

Hi,   We understand that logs are not getting deleted even though you had enabled spark.history.fs properties. Did you found any errors in SHS logs with regarding to this?   Thanks AKR ... View more

What service did you delete? ... View more

0: jdbc:mysql://hdp1.demo.lab/hive> set mapred.job.queue.name=root.admin; Error: Unknown system variable 'job' (state=HY000,code=1193) 0: jdbc:mysql://hdp1.demo.lab/hive> set tez.queue.name=root.admin; Error: Unknown system variable 'queue' (state=HY000,code=1193) 0: jdbc:mysql://hdp1.demo.lab/hive> SET tez.queue.name=root.admin; Error: Unknown system variable 'queue' (state=HY000,code=1193) 0: jdbc:mysql://hdp1.demo.lab/hive> SET TEZ.QUEUE.NAME=ROOT.ADMIN 0: jdbc:mysql://hdp1.demo.lab/hive> SET TEZ.QUEUE.NAME=ROOT.ADMIN; Error: Unknown system variable 'QUEUE' (state=HY000,code=1193) 0: jdbc:mysql://hdp1.demo.lab/hive> set TEZ.queue.name=root.admin; Error: Unknown system variable 'queue' (state=HY000,code=1193) 0: jdbc:mysql://hdp1.demo.lab/hive> SET tez.queue.name=ROOT.ADMIN; Error: Unknown system variable 'queue' (state=HY000,code=1193) 0: jdbc:mysql://hdp1.demo.lab/hive> SET tez.queue.name=admin; Error: Unknown system variable 'queue' (state=HY000,code=1193) 0: jdbc:mysql://hdp1.demo.lab/hive> SET tez.job.queue.name=admin; Error: Unknown system variable 'job' (state=HY000,code=1193) 0: jdbc:mysql://hdp1.demo.lab/hive> ... View more