On HDP 2.6, I have configured Spark Thrift Server for Spark 1.6.x based on community wiki with which I see the queries are executed as the end user, User when connects to Spark Thrift Server for Spark-1, using beeline, I see a new instance of the process getting listed under Resource Manager with end user ( YARN Process runs as end user)
Now I am trying to configure Spark2 Thrift Server, following the official documentation .
In my understanding Now,
Above all are not respected in Spark2 Thrift Server ( But with impersonation support in Spark-1 Thrift Server, above all three are working as expected).
Attaching the screenshots for anomalies.
I am not sure if I missed something here.
1 Queries still run as Hive.
2. STS is not listed under Resource Manager
3. Spark2 Thrift Server Still Runs As Hive User
Thanks in advance.
Still not working with up to Version HDP 2.6.4.... 😞 Any updates about this? Unfortunatelly there are many Spark Jira's open related to this topics. For example this one: