Hi, thanks a lot for this blog. However I didn't manage to make it work, and I have a couple of questions: 1. I'm not using the Livy interpreter, but the Spark one. We have everything set up with it, so I'm wondering if this method can also be used there. For instance, I tried running the Livy interpreter, but I can't make it work (this cluster was originally set up by other people and nobody ever used the Livy interpreter, I think) 2. If it's equivalent with the spark interpreter, I just can't make it work. I'm not sure if I got it properly but this is what I did: -- a) create the jaas file similar to what you show. Copy it to all nodes with right permissions. I even used the /tmp fodler as you do, just in case. -- b) copy it also to hdfs in the 'same' path with right permissions. Is this correct? -- c) copy the keytabs to all the nodes, in the same path,  with right permissions. -- d) do not copy the keytab to hdfs -- c) call System.setProperty before the sc is created in the notebook.   Does this make sense? It just doesn't work for me. I get this type of error and unfortunately I can't see all of it.   Py4JJavaError: An error occurred while calling o118.load. : org.apache.kafka.common.KafkaException: Failed to construct kafka consumer at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:799) at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:615) at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:596) at org.apache.spark.sql.kafka010.SubscribeStrategy.createConsumer(ConsumerStrategy.scala:62) at org.apache.spark.sql.kafka010.KafkaOffsetReader.createConsumer(KafkaOffsetReader.scala:314) at org.apache.spark.sql.kafka010.KafkaOffsetReader.<init>(KafkaOffsetReader.scala:78) at org.apache.spark.sql.kafka010.KafkaSourceProvider.createContinuousReader(KafkaSourceProvider.scala:130) at org.apache.spark.sql.kafka010.KafkaSourceProvider.createContinuousReader(KafkaSourceProvider.scala:43) at org.apache.spark.sql.streaming.DataStreamReader.load(DataStreamReader.scala:185) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) at py4j.Gateway.invoke(Gateway.java:282) at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) at py4j.commands.CallCommand.execute(CallCommand.java:79) at py4j.GatewayConnection.run(GatewayConnection.java:238) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.SecurityException: java.io.IOException: Configuration Error: No such file or directory at sun.security.provider.ConfigFile$Spi.<init>(ConfigFile.java:137) at sun.security.provider.ConfigFile.<init>(ConfigFile.java:102) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at javax.security.auth.login.Configuration$2.run(Configuration.java:255) at javax.security.auth.login.Configuration$2.run(Configuration.java:247) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.login.Configuration.getConfiguration(Configuration.java:246) at org.apache.kafka.common.security.JaasContext.defaultContext(JaasContext.java:172) at org.apache.kafka.common.security.JaasContext.load(JaasContext.java:156) at org.apache.kafka.common.security.JaasContext.loadClientContext(JaasContext.java:142) at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:119) at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:65) at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:88) at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:713) ... 19 more Caused by: java.io.IOException: Configuration Error: No such file or directory at sun.security.provider.ConfigFile$Spi.init(ConfigFile.java:335) at sun.security.provider.ConfigFile$Spi.init(ConfigFile.java:271) at sun.security.provider.ConfigFile$Spi.<init>(ConfigFile.java:135) ... 36 more   Thanks and best regards, Valerio ... View more