Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Question regarding client installation/deletions on datanodes

Solved Go to solution

Question regarding client installation/deletions on datanodes

Expert Contributor

Hello All,

We have a cluster with 160+ nodes. We are planning for an rolling upgrade from HDP 2.5 to 2.6 using ambari. We have edgenodes where all the clients has been installed. In addition, we also have clients installed on all the datanodes (150+ nodes) like HDFS client, Zookeeper client etc.

If you review the below link, we could see that ambari does validate the services multiple times by running service checks during upgrade process.

https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.0.0/bk_ambari-upgrade/content/upgrading_HDP_per...

This will consume a lot of time during the upgrade. We can minimize the time by limiting the clients service checks.

Do we really need all the clients to be installed on the datanodes? If we remove the clients from datanodes, will there be any impact on the cluster in terms of functionality. If there is no impact then what all clients can be removed .

Appreciate your input!!

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Question regarding client installation/deletions on datanodes

@Shyam Shaw,

Note: I haven't tried it myself but hope this should work. Please be cautious if you trying it on production cluster

1) Find the upgrade xml file on ambari server host.

/var/lib/ambari-server/resources/stacks/HDP/2.5/upgrades/upgrade-2.6.xml

2) Comment out the following piece of code in the upgrade XML file and save it. This piece of code will occur thrice . SERVICE_CHECK_1, SERVICE_CHECK_2, SERVICE_CHECK_3. Comment all the 3.

<group name="SERVICE_CHECK_1" title="All Service Checks" xsi:type="service-check">
      <direction>UPGRADE</direction>
      <skippable>true</skippable>
      <priority>
        <service>ZOOKEEPER</service>
        <service>RANGER</service>
        <service>RANGER_KMS</service>
        <service>HDFS</service>
        <service>KAFKA</service>
        <service>YARN</service>
        <service>MAPREDUCE2</service>
        <service>HBASE</service>
      </priority>
    </group>

3): skip service check failures. Change it to true from false

<skip-service-check-failures>true</skip-service-check-failures> 

4) Restart Ambari Server for it to pick up the changes.

ambari-server restart

5) Start rolling upgrade

Thanks,

Aditya

View solution in original post

5 REPLIES 5
Highlighted

Re: Question regarding client installation/deletions on datanodes

Expert Contributor

We are now sure that we can't remove the clients from Datanodes as our applications will start failing.

So, is there any way we can skip the service checks during the rolling upgrade process.

Highlighted

Re: Question regarding client installation/deletions on datanodes

@Shyam Shaw,

Note: I haven't tried it myself but hope this should work. Please be cautious if you trying it on production cluster

1) Find the upgrade xml file on ambari server host.

/var/lib/ambari-server/resources/stacks/HDP/2.5/upgrades/upgrade-2.6.xml

2) Comment out the following piece of code in the upgrade XML file and save it. This piece of code will occur thrice . SERVICE_CHECK_1, SERVICE_CHECK_2, SERVICE_CHECK_3. Comment all the 3.

<group name="SERVICE_CHECK_1" title="All Service Checks" xsi:type="service-check">
      <direction>UPGRADE</direction>
      <skippable>true</skippable>
      <priority>
        <service>ZOOKEEPER</service>
        <service>RANGER</service>
        <service>RANGER_KMS</service>
        <service>HDFS</service>
        <service>KAFKA</service>
        <service>YARN</service>
        <service>MAPREDUCE2</service>
        <service>HBASE</service>
      </priority>
    </group>

3): skip service check failures. Change it to true from false

<skip-service-check-failures>true</skip-service-check-failures> 

4) Restart Ambari Server for it to pick up the changes.

ambari-server restart

5) Start rolling upgrade

Thanks,

Aditya

View solution in original post

Highlighted

Re: Question regarding client installation/deletions on datanodes

Expert Contributor

@Aditya Sirna

Thanks for the response. Could there be any downside of skipping the service check during rolling upgrade.

Highlighted

Re: Question regarding client installation/deletions on datanodes

@Shyam Shaw,

There should not be any issue because of this. Please check Error Handling in the doc

https://community.hortonworks.com/articles/2473/rolling-upgrade-express-upgrade-in-ambari.html

Highlighted

Re: Question regarding client installation/deletions on datanodes

Expert Contributor

@Aditya Sirna

Thanks for the information.

Don't have an account?
Coming from Hortonworks? Activate your account here