in Ambari we have the following feature that is disable , under yarn Configs what is the relevant ambari rest api in order to change the state of CPU Scheduling from disable to enable ? ... View more

hi all     we are trying to download the ambari version 2.6.1 but without success ( according to https://docs.hortonworks.com.s3.amazonaws.com/HDPDocuments/HDF3/HDF-3.1.1/bk_installing-hdf-on-hdp-ppc/content/ambari_repositories.html )    wget http://public-repo-1.hortonworks.com/ambari/centos7-ppc/2.x/updates/2.6.1.0 --2021-02-16 14:44:08-- http://public-repo-1.hortonworks.com/ambari/centos7-ppc/2.x/updates/2.6.1.0 Resolving public-repo-1.hortonworks.com (public-repo-1.hortonworks.com)... 13.225.255.100, 13.225.255.128, 13.225.255.124, ... Connecting to public-repo-1.hortonworks.com (public-repo-1.hortonworks.com)|13.225.255.100|:80... connected. HTTP request sent, awaiting response... 403 Forbidden 2021-02-16 14:44:09 ERROR 403: Forbidden.   any idea why this version cant be download?   but we can download from other site as   wget http://archive.apache.org/dist/ambari/ambari-2.6.1/apache-ambari-2.6.1-src.tar.gz ... View more

We have ambari cluster , HDP version 2.6.5   Cluster include management of two name-node ( one is active and the secondary is standby ) And 65 datanode machines   We have problem with the standby name-node that not started and from the namenode logs we Can see the following   2021-01-01 15:19:43,269 ERROR namenode.NameNode (NameNode.java:main(1783)) - Failed to start namenode. java.io.IOException: There appears to be a gap in the edit log. We expected txid 90247527115, but got txid 90247903412. from ambari we can see    For now the active namenode is up but the standby name node is down , and the root cause for This issue is because **namenode matadata is damaged/corrupted.** So we have two solution - A or B   A)   run the following recover on standby namenode su hadoop namenode -recover   B)   Put Active NN in safemode su hdfs hdfs dfsadmin -safemode enter Do a savenamespace operation on Active NN su hdfs hdfs dfsadmin -saveNamespace Leave Safemode su hdfs hdfs dfsadmin -safemode leave Login to Standby NN Run below command on Standby namenode to get latest fsimage that we saved in above steps. su hdfs hdfs namenode -bootstrapStandby -force what is the preferred solution ( solution A or Solution B ) for our problem?  ... View more

we have Hadoop cluster with 2 name-nodes ( active standby ) and 12 data nodes   all 12 data-nodes machines have disks for HDFS   we are before the action of `hadoop namenode -recover` , and that because we suspect about corrupted files as fsimage_0000000000001253918 or edits_0000000000001203337-0000000000001214475 etc   so to recover the hdfs meta data we can do the following           $ hadoop namenode -recover DEPRECATED: Use of this script to execute hdfs command is deprecated. Instead use the hdfs command for it. 21/01/19 17:56:35 INFO namenode.NameNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting NameNode STARTUP_MSG: user = hdfs STARTUP_MSG: host = master1.sys67.com/17.2.12.78 STARTUP_MSG: args = [-recover] STARTUP_MSG: version = 2.7.3.2.6.5.0-292 21/01/19 17:56:35 INFO namenode.NameNode: createNameNode [-recover] You have selected Metadata Recovery mode. This mode is intended to recover lost metadata on a corrupt filesystem. Metadata recovery mode often permanently deletes data from your HDFS filesystem. Please back up your edit log and fsimage before trying this! Are you ready to proceed? (Y/N) (Y or N) y           the question is: dose this action could also affected the data itself on the data-nodes machines ?   or only the meta data on namenode machines? ... View more

we have HDP cluster version `2.6.5` with `8` data nodes , all machines are installed on rhel 7.6 version   HDP cluster is based amabri platform version - `2.6.1`   each data-node ( worker machine ) include two disks and each disk size is 1.8T   when we access the data-node machines we can see differences between the size of the disks   for example on the first data-node the size is : ( by `df -h` )   /dev/sdb 1.8T 839G 996G 46% /grid/sdc /dev/sda 1.8T 1014G 821G 56% /grid/sdb   on the second data-node the size is: /dev/sdb 1.8T 1.5T 390G 79% /grid/sdc /dev/sda 1.8T 1.5T 400G 79% /grid/sdb   on the third data-node th size is: /dev/sdb 1.8T 1.7T 170G 91% /grid/sdc /dev/sda 1.8T 1.7T 169G 91% /grid/sdb   and so on   the big question is why HDFS not perform the re-balance on the HDFS disks?   *for example expected results on all disks should be with the same size on all datanodes machines* why is the used size differences between `datanode1` to `datanode2` to `datanode3` etc ?   any advice about the tune parameters in HDFS that can help us?   *because its very critical when one disk is reached `100%` size and the other are more small as `50%`* ... View more