I need to update the following parameters in my old ambari cluster to be most updated ( each parameter have config type) recovery_enabled fs.trash.interval dfs.datanode.data.dir dfs.namenode.accesstime.precision delete.topic.enable log.retention.bytes spark.history.fs.cleaner.enabled spark.history.fs.cleaner.interval spark.history.fs.cleaner.maxAge spark_daemon_memory spark_thrift_cmd_opts spark.broadcast.blockSize spark.driver.maxResultSize spark.dynamicAllocation.executorIdleTimeout spark.dynamicAllocation.initialExecutors spark.dynamicAllocation.maxExecutors spark.dynamicAllocation.schedulerBacklogTimeout spark.executor.memory spark.files.maxPartitionBytes spark.files.openCostInBytes spark.kryoserializer.buffer.max spark.memory.offHeap.enabled spark.memory.offHeap.size spark.sql.autoBroadcastJoinThreshold spark.sql.shuffle.partitions spark.storage.memoryMapThreshold tez.runtime.io.sort.mb tez.runtime.unordered.output.buffer.size-mb tez.task.resource.memory.mb initLimit syncLimit hive.auto.convert.join.noconditionaltask.size hive.tez.container.size mapreduce.map.java.opts mapreduce.map.memory.mb mapreduce.reduce.java.opts mapreduce.reduce.memory.mb mapreduce.task.io.sort.mb yarn.app.mapreduce.am.command-opts yarn.app.mapreduce.am.resource.mb spark_thrift_cmd_opts resourcemanager_heapsize yarn.nodemanager.resource.cpu-vcores yarn.nodemanager.resource.memory-mb yarn.resourcemanager.am.max-attempts yarn.scheduler.maximum-allocation-mb yarn.scheduler.maximum-allocation-vcores syncLimit after we set all parameters and with their updated values ( with config.sh script ) , and perform restart on all required services / components some of the services/components cannot started what is wrong with my procedure? list of the config type for the parameters : cluster-env core-site hdfs-site hive-interactive-site hive-site kafka-broker mapred-site spark2-defaults spark2-env spark2-thrift-sparkconf tez-interactive-site tez-site yarn-env yarn-site zoo.cfg remark - all the updated parameters already set on other new ambari cluster ... View more

how to perform restart (on services/components) after we change around 50 parameters (blueprint parameters) in ambari cluster we change around 50 values in ambari cluster ( with config.sh script ) after that we restart by API all relevant services/components ( by API down) , but after half hour we get fail on some services/components so maybe the right thing is to perform restart after each value change ? ( in place to restart after all values are set ) ? or maybe need to restart the services/components according to some priority/rules ? example how we perform restart curl -iv -u admin:admin -H "X-Requested-By: ambari" -X POST -d '{"RequestInfo":{"command":"RESTART","context":"Restart all required services","operation_level":"host_component"},"Requests/resource_filters":[{"hosts_predicate":"HostRoles/stale_configs=true"}]}' http://$HOSTANAME:8080/api/v1/clusters/$CLUSTER_NAME/requests ... View more

how to capture the last request ID from API , in order to trace the status ( as IN_PROCESS ) for example we run some API to restart a service , then I want to run API that will capture the last request ID example of request "id" { "href" : "http://192.23.45.2.ur.com:8080/api/v1/clusters/HDP01/requests/17", "Requests" : { "id" : 17, "status" : "Accepted" } ... View more

the following API give only the hosts from the amabri server curl -sH "X-Requested-By: ambari" -u admin:admin -i http://localhost:8080/api/v1/hosts is it possible to change the API in way that we can get also the IP address for each host? the following example print only the hosts : curl -sH "X-Requested-By: ambari" -u admin:admin -i http://localhost:8080/api/v1/hosts | grep host_name | sed s'/"//g' | awk '{print $NF}' master01 master02 master03 worker01 worker02 expected results master01 192.9.200.12 master02 192.9.200.13 master03 192.9.200.14 worker01 192.9.200.15 worker02 192.9.200.16 ... View more

the following API will restart all services that restart action is required ( after paramter changes ) but how to know if any service in ambari cluster need to restart by API ( I mean just to check if restart is needed without to perform restart ? ) # curl -iv -u admin:admin -H "X-Requested-By: ambari" -X POST -d '{"RequestInfo":{"command":"RESTART","context":"Restart all required services","operation_level":"host_component"},"Requests/resource_filters":[{"hosts_predicate":"HostRoles/stale_configs=true"}]}' http://$AMBARI_SERVER:8080/api/v1/clusters/$CLUSTER_NAME/reques ... View more

hi all I need advice regarding the following problem my target is to guarantees that the leadership load across the brokers in a cluster is evenly balanced. sh /usr/hdp/2.6.0.3-8/kafka/bin/kafka-preferred-replica-election.sh --zookeeper zklist Failed to start preferred replica election org.I0Itec.zkclient.exception.ZkException: Unable to connect to zklist at org.I0Itec.zkclient.ZkConnection.connect(ZkConnection.java:71) at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1227) at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:156) at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:130) at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:97) at kafka.utils.ZkUtils$.createZkClient(ZkUtils.scala:70) at kafka.admin.PreferredReplicaLeaderElectionCommand$.main(PreferredReplicaLeaderElectionCommand.scala:57) at kafka.admin.PreferredReplicaLeaderElectionCommand.main(PreferredReplicaLeaderElectionCommand.scala) Caused by: java.net.UnknownHostException: zklist: Temporary failure in name resolution at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928) at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323) at java.net.InetAddress.getAllByName0(InetAddress.java:1276) at java.net.InetAddress.getAllByName(InetAddress.java:1192) at java.net.InetAddress.getAllByName(InetAddress.java:1126) at org.apache.zookeeper.client.StaticHostProvider.<init>(StaticHostProvider.java:61) at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:446) at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:380) at org.I0Itec.zkclient.ZkConnection.connect(ZkConnection.java:69) ... View more