I use HDP 2.6.3, two users get following error when they try Zeppelin with Spark2 interpreter,
org.apache.zeppelin.interpreter.InterpreterException: Host key verification failed.
Spark2 interpreter configs are below.
@Erkan ŞİRİN We don't support Spark2/Spark interpreter impersonation - Please change the following global configuration for the interpreter so that is instantiated Globally in shared process:
then save and restart Zeppelin.
For impersonation use Livy interpreter.
*** If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer.
Hi @Felix Albani thank you. I tried your solution. It worked with some side effects. When two users use same spark interpreter both use the same Spark Context, Spark Application. Therefore they cant write seperate, isolated application. For example user A creates:
val a = 100
User B prints a:
res2: Int = 100
In this solution as a Cluster Admin I have to create many spark interpreters per user. Indeed we have less than 10 Spark users but what if we had 100s. So, I consider this is a workaround solution, not an ideal one.
@Erkan ŞİRİN as I mentioned before you should consider using Livy if your requirement is isolating the users or doing impersonation. Also Livy is more scalable because it lets you run in cluster mode vs client mode which is the only supported deployment mode for spark interprepter.