installation of spark2-client

[#1]

i want to install spark2 clients at all nodes in my cluster . but i want to this from machine . in other sense I want to do this form machine rather than ambari or with the help of rest api . are there some steps to achieve the same .

Thanks

[#2]

@Anurag Mishra

Can you try something like this:

Step1). Post the SPARK2_CLIENT

# curl -u admin:admin -H "X-Requested-By: ambari" -X POST  http://hdfcluster1.example.com:8080/api/v1/clusters/TestCluster/hosts/hdfcluster2.example.com/host_c...

Step2). Change the client State from INIT to INSTALLED.

 curl -u admin:admin -H "X-Requested-By: ambari" -X PUT -d '{"HostRoles": {"state" : "INSTALLED"}}' http://hdfcluster1.example.com:8080/api/v1/clusters/TestCluster/hosts/hdfcluster2.example.com/host_c...

.

[#3]

@Jay Kumar SenSharma

Hi Jay , I wanted to do without help of rest api , is there any other way .

[#4]

@Anurag Mishra

From Ambari UI

Ambari UI --> Hosts (Tab) --> Click on your desired hostname link --> Install Clients

.

However it does not give to many options to selectively installed any particular client.

[#5]

Jay Kumar SenSharma

thanks for your response jay , but My requirement is to install spark2-client in machine without use of ambari and rest api , rest api I also doing same if I am doing it from ambari . i wanted to manually install spark2-client . I am looking for that option

[#6]

@Anurag Mishra

# yum install spark2

And then copy and paste the Spark2 config from the Spark2 host.

.

[#7]

@Anurag Mishra

If you are not installing Spark2 Clients via Ambari Or Ambari UI then it will not be managed by Ambari at all. So you will have to do yum install of spark binaries and then you will have to manually get the required spark configurations from the Spark Service host everytime you make any config changes to make it in sync with client.

In that case please refer to:

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.4/bk_command-line-installation/content/ch_ins...

[#8]

@Jay Kumar SenSharma

Hi jay I went to the link but little confuse over spark2-client installation process :

yum search spark2

spark_<version>_<build>-master.noarch : Server for Spark 2 master
spark_<version>_<build>-python.noarch : Python client for Spark 2
spark_<version>_<build>-worker.noarch : Server for Spark 2 worker
spark_<version>_<build>.noarch : Lightning-Fast Cluster Computing

yum install spark2_<version>-master spark_<version>-python

can you please help me for only client installation which package has to only install ?

and when i do yum search spark2 it lists so many spark versions but on yum install there is only spark2_version-master and spark_version-python ?

could you please make it more understandable for me ?

thanks

[#9]

@Anurag Mishra

This is what i tested.

I took a host where Spark2 client was not installed. I installed the SPARK2_CLIENT from ambari server UI to see exactly which all packages it is installing on that host.

Based on the "/var/log/yum.log" file i could see the following packages were installed.

Mar 20 09:06:02 Installed: shc_2_6_4_0_91-1.1.0.2.6.4.0-91.noarch
Mar 20 09:06:09 Installed: spark_llap_2_6_4_0_91-1.0.0.2.6.4.0-91.noarch
Mar 20 09:06:58 Installed: spark2_2_6_4_0_91-2.2.0.2.6.4.0-91.noarch
Mar 20 09:06:58 Installed: spark2-2.2.0.2.6.4.0-91.noarch

.