Set SPARK_HOME for spark2

Spark2 2.3.0 on cdh.5.13 deployed with CSD and parcels.

by default it uses /etc/spark2/conf,

but it seems it should use /etc/spark2/conf.cloudera.spark2_on_yarn/

is there a way to set it properly?

i could set it in spark2-env.sh but it seems not the appropriate place.

EricLAnswer

Platinum ⭐️⭐️

@sbn,

/etc/spark2/conf should be a symlink to /etc/spark2/conf.cloudera.spark2_on_yarn, can you confirm by running:

ls -al /etc/spark2

Cheers
Eric

EricLAnswer

Platinum ⭐️⭐️

@sbn,

/etc/spark2/conf should be a symlink to /etc/spark2/conf.cloudera.spark2_on_yarn, can you confirm by running:

ls -al /etc/spark2

Cheers
Eric

Translate

sbnAuthor

Silver ⭐️⭐️

thanks, i suppose one could just move that to wherever you want the home to be.

this was part of a hack deployment, so ultimately not needed.

Translate

Sign up