Support Questions
Find answers, ask questions, and share your expertise

Zeppelin failed to find Kerberos tgt for some functions (e.g. sc.textFile)

Zeppelin failed to find Kerberos tgt for some functions (e.g. sc.textFile)

New Contributor

We are actually using Microsoft HDI with ESP but I am wondering if anyone has had similar experience.

 

From our zeppelin notebook, when I read data to a Dataset (spark.read.text) it works but when I try to read it to an RDD (sc.textFile), I get an authentication exception for failure to find Kerberos tgt

notebook_screen.png

 

Note that while sc.textFile failed in zeppelin, it works well from spark-shell. Moreover, as a workaround for Zeppelin, we can first read the data to Dataset and convert it to RDD by "spark.read.text(path).rdd".

 

I looked at some answers here in the community. I added livy.spark.yarn.security.credentials.hiveserver2.enabled but it doesn't make any difference. Some mentioned that we need to enable 'user impersonate' but I got other errors from that and I am not sure if this is the right direction since only certain functions don't work.