What services not needed? and can be removed

[#1]

i have a large cluster installed currently it is used only for spark on yarn jobs

what services on the master i can remove\uninstall to free up some ram?

one server to accept spark jobs currently have the following services:

1. Timeline Service V1.5 / YARN Master
2. History Server / MapReduce2 Master
3. Hive Metastore / Hive Master
4. HiveServer2 / Hive Master
5. Metrics Collector / Ambari Metrics Master
6. Grafana / Ambari Metrics Master
7. NameNode / HDFS Master
8. ResourceManager / YARN Master
9. SNameNode / HDFS Master
10. Spark2 History Server / Spark2 Master
11. Timeline Service V2.0 Reader / YARN Master
12. YARN Registry DNS / YARN Master
13. ZooKeeper Server / ZooKeeper Master
14. DataNode / HDFS Slave
15. Metrics Monitor / Ambari Metrics Slave
16. HDFS Client / HDFS Client
17. Hive Client / Hive Client
18. MapReduce2 Client / MapReduce2 Client
19. Spark2 Client / Spark2 Client
20. Tez Client / Tez Client
21. YARN Client / YARN Client
22. ZooKeeper Client / ZooKeeper Client

and multiple clients with:

1. Metrics Monitor / Ambari Metrics Slave
2. NodeManager / YARN Slave

[#2]

@Ilia K

If you are not using Hive Service (HIve Metastore / HiveServer2 / Hive Client) then you can remove it.

Also AMS collector which internally starts an HBase instance to store the cluster & services related metrics data ... So if you are not much interested in Metrics Data then you can also Delete the AMS service (AMS Collector / Grafana / Metrics Monitors).

[#3]

Thanks for the quick response,

Can i delete all hive?

1. Hive Metastore
2. HiveServer2
3. Hive Client

Metrics are useful. i can see some bottlenecks and improve the performance,

Currently still struggling the improve the time it take to allocate the containers each job ( all the time they are the same and it take 30-60 seconds every job to allocate them )

Btw why it is a requirements (in the install stage it was selected cause i needed spark\yarn)

[#4 · Accepted solution]

@Ilia K

Bit correction here.

I just checkecd that Spark2 has a dependency on HIve Service.

# grep -A1 -B1 'HIVE' /var/lib/ambari-server/resources/common-services/SPARK2/2.0.0/metainfo.xml
            <dependency>
              <name>HIVE/HIVE_METASTORE</name>
              <scope>cluster</scope>

Hence you should not delete the HIVE service .. but if you are not using Hive then just Stop the "Hive Service" components that way it will not use RAM and put the Hive Service in Maintenance Mode.

[#5]

ok, just stopped 3 hive services, still it abit bother me that there are red dots in the ui :),

Btw why there are a dependency on hive?

[#6]

@Ilia K

Spark can be used to interact with Hive.

When you install Spark using Ambari, the hive-site.xml file is automatically populated with the Hive metastore location.

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.5/bk_spark-component-guide/content/spark-conf...