Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Set SPARK_HOME for spark2

avatar
Contributor

Spark2 2.3.0 on cdh.5.13 deployed with CSD and parcels.

 

by default it uses /etc/spark2/conf,

but it seems it should use /etc/spark2/conf.cloudera.spark2_on_yarn/

 

is there a way to set it properly?

i could set it in spark2-env.sh but it seems not the appropriate place.

1 ACCEPTED SOLUTION

avatar
Super Guru
@sbn,

/etc/spark2/conf should be a symlink to /etc/spark2/conf.cloudera.spark2_on_yarn, can you confirm by running:

ls -al /etc/spark2

Cheers
Eric

View solution in original post

2 REPLIES 2

avatar
Super Guru
@sbn,

/etc/spark2/conf should be a symlink to /etc/spark2/conf.cloudera.spark2_on_yarn, can you confirm by running:

ls -al /etc/spark2

Cheers
Eric

avatar
Contributor

thanks, i suppose one could just move that to wherever you want the home to be.

this was part of a hack deployment, so ultimately not needed.