https://cwiki.apache.org/confluence/display/AMBARI... When the REST API is used to trigger the decommission of the datanode a request ID is returned since its an async operation (see https://github.com/apache/ambari/blob/trunk/ambari...) After polling on the status of the returned request ID for sometime, the request status shows "COMPLETED". However the decommission is actually still progressing (incomplete!) Is this a bug ? Shouldn't the Ambari asynchronous request be marked COMPLETED only after the decommission has finished completely ? ... View more

HDP-2.3.8.0-74 Spark 1.5.2.2.3 The start of "Spark history server" is successful in Ambari but very quickly it stops/fails. In the logs I see the following 16/01/01 16:42:36 INFO HistoryServer: History provider class: org.apache.spark.deploy.history.yarn.server.YarnHistoryProvider Exception in thread "main" java.lang.ClassNotFoundException: org.apache.spark.deploy.history.yarn.server.YarnHistoryProvider at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:191) at org.apache.spark.deploy.history.HistoryServer$.main(HistoryServer.scala:224) at org.apache.spark.deploy.history.HistoryServer.main(HistoryServer.scala) 16/01/01 16:42:36 INFO Utils: Shutdown hook called The spark-defaults.conf is as follows # Generated by Apache Ambari. Fri Jan 1 16:42:31 2016 spark.history.kerberos.keytab none spark.history.kerberos.principal none spark.history.provider org.apache.spark.deploy.history.yarn.server.YarnHistoryProvider spark.history.ui.port 18080 spark.yarn.containerLauncherMaxThreads 25 spark.yarn.driver.memoryOverhead 384 spark.yarn.executor.memoryOverhead 384 spark.yarn.historyServer.address host10.demo.com:18080 spark.yarn.max.executor.failures 3 spark.yarn.preserve.staging.files false spark.yarn.queue default spark.yarn.scheduler.heartbeat.interval-ms 5000 spark.yarn.services org.apache.spark.deploy.yarn.history.YarnHistoryServicespark.yarn.submit.file.replication 3 ... View more

Does PQS have any dependency on data (non-OS) disks of worker node? If PQS is installed on slave/worker node (note - JBOD disk organization), is there any impact to the service when replacing a data disk? Are there any pros/cons of installing PQS on master node vs worker node ? Ambari docs seem to indicate that both are supported ... View more