Ambari services can be configured to start automatically on system boot. Each service can be configured to start all components, masters and workers, or selectively. so how to enable all services in ambari cluster to start automatically on system boot by API ? ... View more

when I run/start the thrift server su spark ./sbin/start-thriftserver.sh or ./sbin/start-thriftserver.sh --master yarn-client --executor-memory 512m --hiveconf hive.server2.thrift.port=100015 I get from the log: /var/log/spark2/spark-spark-org.apache.spark.sql.hive.thriftserver.HiveThriftServer2-1-master.out Spark Command: /usr/jdk64/jdk1.8.0_112/bin/java -Dhdp.version=2.6.0.3-8 -cp /usr/hdp/2.6.0.3-8/spark2/conf/:/usr/hdp/2.6.0.3-8/spark2/jars/*:/usr/hdp/current/hadoop-client/conf/ -Xmx10000m org.apache.spark.deploy.SparkSubmit --master yarn-client --class org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 --name Thrift JDBC/ODBC Server --executor-mem ory 512m spark-internal --hiveconf hive.server2.thrift.port=100015 ======================================== Warning: Master yarn-client is deprecated since 2.0. Please use master "yarn" with specified deploy mode instead. 17/09/01 07:02:05 WARN AbstractLifeCycle: FAILED ServerConnector@1fc793c2{HTTP/1.1}{0.0.0.0:4040}: java.net.BindException: Address already in use java.net.BindException: Address already in use how to solve it? ... View more

I need to update the following parameters in my old ambari cluster to be most updated ( each parameter have config type) recovery_enabled fs.trash.interval dfs.datanode.data.dir dfs.namenode.accesstime.precision delete.topic.enable log.retention.bytes spark.history.fs.cleaner.enabled spark.history.fs.cleaner.interval spark.history.fs.cleaner.maxAge spark_daemon_memory spark_thrift_cmd_opts spark.broadcast.blockSize spark.driver.maxResultSize spark.dynamicAllocation.executorIdleTimeout spark.dynamicAllocation.initialExecutors spark.dynamicAllocation.maxExecutors spark.dynamicAllocation.schedulerBacklogTimeout spark.executor.memory spark.files.maxPartitionBytes spark.files.openCostInBytes spark.kryoserializer.buffer.max spark.memory.offHeap.enabled spark.memory.offHeap.size spark.sql.autoBroadcastJoinThreshold spark.sql.shuffle.partitions spark.storage.memoryMapThreshold tez.runtime.io.sort.mb tez.runtime.unordered.output.buffer.size-mb tez.task.resource.memory.mb initLimit syncLimit hive.auto.convert.join.noconditionaltask.size hive.tez.container.size mapreduce.map.java.opts mapreduce.map.memory.mb mapreduce.reduce.java.opts mapreduce.reduce.memory.mb mapreduce.task.io.sort.mb yarn.app.mapreduce.am.command-opts yarn.app.mapreduce.am.resource.mb spark_thrift_cmd_opts resourcemanager_heapsize yarn.nodemanager.resource.cpu-vcores yarn.nodemanager.resource.memory-mb yarn.resourcemanager.am.max-attempts yarn.scheduler.maximum-allocation-mb yarn.scheduler.maximum-allocation-vcores syncLimit after we set all parameters and with their updated values ( with config.sh script ) , and perform restart on all required services / components some of the services/components cannot started what is wrong with my procedure? list of the config type for the parameters : cluster-env core-site hdfs-site hive-interactive-site hive-site kafka-broker mapred-site spark2-defaults spark2-env spark2-thrift-sparkconf tez-interactive-site tez-site yarn-env yarn-site zoo.cfg remark - all the updated parameters already set on other new ambari cluster ... View more