from ambari we can capture all version , by click on `Admin` button and the click on `stack and version` finally click on `Versions` , then we get the following details     we want to know how to capture all these version by using ambari rest api? we try curl -u admin:admin -H 'X-Requested-By:admin' 'http://localhost:8080/api/v1/clusters/HDP/configuratons/service_config_versions' but its not return any info   ... View more

    I have a remote server and servers authenticated Hadoop environment. I want to copy file from Remote server to Hadoop machine to HDFS Please advise efficient approach/HDFS command to copy files from remote server to HDFS. Any example will be helpful. as ordinary way to copy file from remote server to server itself is scp -rp file remote_server:/tmp but this approach not support copy directly to hdfs ... View more

we have Hadoop cluster with only 2 data nodes machines   in HDFS configuration we defined the Block replication to 3   so Block replication=3 is it OK? to defined Block replication=3 , when we have only two data nodes in the cluster?   from my understanding when we defined Block replication=3 while we have 2 data nodes machines in HDFS cluster   its means that one machine should have 2 replica .  and the other machine one replica , am I correct here? ... View more

hi all   we are examining the option to installed new Hadoop cluster based on CDH cloudera packages   I will appreciate to get feedback's / notes / remarks / else ,  from someone that have the experienced of installing the CDH packages , and how this Hadoop looks after installing the  Cloudera open source ( CDH )         ... View more

hi all   we have HDP 2.6.4 cluster with 365 data-nodes ( workers machines )    we intend to add to this cluster 67 new data-nodes ,   but for now these new data-nodes should not be active , in spite the Linux OS is up and machine is up   so is it possible to disable HDFS and YARN on these new machines?    what is the preferred way?    for example - decommission ? , or just maintenance mode? or maybe both them?    the target is to avoid from spark jobs to run on the new data-nodes machines and disable the HDFS Temporarily     ... View more

Hi all     I am running the following ambari API in order to find the CPU count on data node machine     curl -H "X-Requested-By: ambari" -u admin:admin -X GET http://master:8080/api/v1/clusters/hdp_test/hosts?fields=Hosts/host_name,Hosts/public_host_name,Hosts/cpu_count&host_compnents/HostRoles/component_name.in(DATANODE)   we get the error about:     {   "status" : 400,   "message" : "The properties [host_compnents/HostRoles/component_name] specified in the request or predicate are not supported for the resource type Host."     The target is to find the number of cpu on data node machine as we can do it on datanode linux machine   nproc 16 ... View more

As all know when is need to print the kafka broker id’s we can use the following cli zookeeper-shell.sh zoo_server1:2181 <<< "ls /brokers/ids" this cli print the following   WatchedEvent state:SyncConnected type:None path:null [1018, 1017, 1016]   Its means that we have kafka with id’s 1018 1017 1016 But our kafka names are   Kafka_confluent01 Kafka_confluent02 Kafka_confluent03 So how to know which kafka broker id ( 1018 , 1017 , 1016 ) is belong to the real host ( Kafka_confluent01 / Kafka_confluent02 / Kafka_confluent03 )   ... View more