We are using Zeppelin to run spark job that require hive databases access. In the first time it didn't worked because of the missing hive-site.xml in spark2 conf directory.
Now it works fine except that each time we redeploy the config via cloudera manager the spark2 conf folder is crushed with the new one and we lost the hive-site.xml.
Any suggestion on how avoid this ?
@saranvisa Yes, I did a symlink.
Note that this could also be a Zeppelin's issue since I can list Hive DB for spark2-shell without the hive-site symlink in spark2/conf dir.
whitout hive-site :
I have some news from Zeppelin side, the hive-site.xml is required in the spark folder to support multi-version as dsicribed here : https://issues.apache.org/jira/browse/ZEPPELIN-2244
For now I have to create the symlink to hive-site.xml again and again after each config deploy. Any workaround to solve that ?
We are feeling the same pain here.
In cloudera manager there is usually a "safety valve" for relevant configuration files where you get the opportunity to tweak the configuration for each role. In the Spark2 section in cloudera manager there is no safety valve for hive-site.xml.