hi all   we are examining the option to installed new Hadoop cluster based on CDH cloudera packages   I will appreciate to get feedback's / notes / remarks / else ,  from someone that have the experienced of installing the CDH packages , and how this Hadoop looks after installing the  Cloudera open source ( CDH )         ... View more

hi all   we have HDP 2.6.4 cluster with 365 data-nodes ( workers machines )    we intend to add to this cluster 67 new data-nodes ,   but for now these new data-nodes should not be active , in spite the Linux OS is up and machine is up   so is it possible to disable HDFS and YARN on these new machines?    what is the preferred way?    for example - decommission ? , or just maintenance mode? or maybe both them?    the target is to avoid from spark jobs to run on the new data-nodes machines and disable the HDFS Temporarily     ... View more

Hi all     I am running the following ambari API in order to find the CPU count on data node machine     curl -H "X-Requested-By: ambari" -u admin:admin -X GET http://master:8080/api/v1/clusters/hdp_test/hosts?fields=Hosts/host_name,Hosts/public_host_name,Hosts/cpu_count&host_compnents/HostRoles/component_name.in(DATANODE)   we get the error about:     {   "status" : 400,   "message" : "The properties [host_compnents/HostRoles/component_name] specified in the request or predicate are not supported for the resource type Host."     The target is to find the number of cpu on data node machine as we can do it on datanode linux machine   nproc 16 ... View more

As all know when is need to print the kafka broker id’s we can use the following cli zookeeper-shell.sh zoo_server1:2181 <<< "ls /brokers/ids" this cli print the following   WatchedEvent state:SyncConnected type:None path:null [1018, 1017, 1016]   Its means that we have kafka with id’s 1018 1017 1016 But our kafka names are   Kafka_confluent01 Kafka_confluent02 Kafka_confluent03 So how to know which kafka broker id ( 1018 , 1017 , 1016 ) is belong to the real host ( Kafka_confluent01 / Kafka_confluent02 / Kafka_confluent03 )   ... View more

  hi all   the following ambari API will stop only the data-nodes on workers machines   # curl -s -u admin:admin -H "X-Requested-By: ambari" -X PUT -d '{"RequestInfo":{"context":"Stop DataNodes","operation_level":{"level":"SERVICE","cluster_name":"$CLUSTER_NAME"},"query":"HostRoles/component_name=DATANODE&HostRoles/host_name.in(dn1.example.com,dn2.example.com,dn3.example.com)&HostRoles/maintenance_state=OFF"},"Body":{"HostRoles":{"state":"INSTALLED"}}}' "h ttp://$AMBARI_FQDN:8080/api/v1/clusters/$CLUSTER_NAME/host_components"   now we want to replace the list - n1.example.com,dn2.example.com,dn3.example.com   with variable - DATANODES=n1.example.com,dn2.example.com,dn3.example.com   so $DATANODES will replce the list in the API as the following     # curl -s -u admin:admin -H "X-Requested-By: ambari" -X PUT -d '{"RequestInfo":{"context":"Stop DataNodes","operation_level":{"level":"SERVICE","cluster_name":"$CLUSTER_NAME"},"query":"HostRoles/component_name=DATANODE&HostRoles/host_name.in($DATANODES)&HostRoles/maintenance_state=OFF"},"Body":{"HostRoles":{"state":"INSTALLED"}}}' "h ttp://$AMBARI_FQDN:8080/api/v1/clusters/$CLUSTER_NAME/host_components"     but this isn't working , any advice how to set it correctly ? ... View more

hi all     we have two master machines    master01 master02     on both machines yarn resource manager is running as active/standby )   how to verify which machine is with active resource manager by ambari rest api ?  ... View more