Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Configure Spark2 with Ambari

avatar
New Contributor

Hi everyone.

We have HDP cluster with several Spark2 clients. I want to perform a remote connection, the manual says that I need to set the SPARK_MASTER_HOST

Is there a way to make such configuration with Ambari UI or I have to edit spark-env.sh? If second, which one I have to choose (namenode, datanode etc)?

1 ACCEPTED SOLUTION

avatar
Super Collaborator
@Chingiz Akmatov

spark-env.sh is ambari managed file and expose on UI. So you can make changes to it from Amabri UI.

To do so Go to Spark2->Configs->Advanced spark2-env section->content property and add the env variable

34709-spark-env.png

View solution in original post

4 REPLIES 4

avatar
Super Collaborator
@Chingiz Akmatov

spark-env.sh is ambari managed file and expose on UI. So you can make changes to it from Amabri UI.

To do so Go to Spark2->Configs->Advanced spark2-env section->content property and add the env variable

34709-spark-env.png

avatar
New Contributor

@jaimin

I set SPARK_MASTER_HOST from the UI as you said, spark-env.sh updated with the line I put, but there is no listening port 7077. What am I missing? And I forgot to mention that we have zookeeper and spark along with spark2 if it makes a sense

avatar
New Contributor

My mistake, forgot to run

start-master.sh

after configuring spark-env.sh

avatar
New Contributor

Hi,

I can not edit my spark config finals, am i missing something ?