our ambary cluster with HDFS 100% I try to capture the trash size with - hadoop dfsadmin -report but from the output I not see the trash size , so how to know the size of trash ? and what are the CLI to get all used partition on HDFS include trash ? second in order to solve the HDFS 100% we think to add disks on the worker machine , is it usful in our case ? if yes how to re balance the data on the new disks ? ... View more

hi all sometimes after we finished to install ambari cluster by the scripts ( ambari cluster include 3 masters machine and workers machines between 2 to 24 machines ) some components are down , so we want to give a second try and to start only the components / services that are down what is the API for that ... View more

when we do safemode get we get the following: su hdfs -c 'hdfs dfsadmin -safemode get' Safe mode is ON in master01.sys23dns.com/107.64.25.61:8020 Safe mode is ON in master03.sys23dns.com/107.64.25.63:8020 its tell us that master01 and master02 are in safe mode can we get more specific details from "safemode get" , about who are the services/components that are in safe mode? I ask because I guess one service or more are in safe mode , and I want to get exactly the details who are in safe mode from the services , ... View more

we have ambari cluster with 3 worker machines ( each worker have 8G memory ) when we start the spark thrift server on master01/03 machines we get the following errors Warning: Master yarn-client is deprecated since 2.0. Please use master "yarn" with specified deploy mode instead. 18/02/05 18:12:52 WARN Utils: spark.executor.instances less than spark.dynamicAllocation.minExecutors is invalid, ignoring its setting, please update your configs. 18/02/05 18:12:53 ERROR SparkContext: Error initializing SparkContext. java.lang.IllegalArgumentException: Required executor memory (10240+1024 MB) is above the max threshold (6144 MB) of this cluster! Please check the values of 'yarn.scheduler.maximum-allocat ion-mb' and/or 'yarn.nodemanager.resource.memory-mb'. please advice - what these errors means ? about - Required executor memory (10240+1024 MB) , what this memory values and how to set the parameters in spark or other in order to solve these isshue ? ... View more

how to configure by API the HIVE parameters: log4j.appender.DRFA.MaxBackupIndex=20 log4j.appender.DRFA.MaxFileSize=1200MB from ambari : Hive -> advanced -> Advanced hive-log4j # 30-day backup log4j.appender.DRFA.MaxBackupIndex=20 log4j.appender.DRFA.MaxFileSize=1200MB log4j.appender.DRFA.layout=org.apache.log4j.PatternLayout ... View more

the following API , print all components curl -u admin:admin -H "X-Requested-By: ambari" -X GET http://localhost:8080/api/v1/clusters/HDP/components/ | grep component_name in out case we need list of component per master machine we have 3 master machines in the cluster so what is the API that will capture the component on specific master machine ? ... View more