Hi @RangaReddy , is something similar applicable also for structured streaming? Thanks. ... View more

Thanks for sharing this workaround @rsanchez . In the docs we found that is not recommended to use experimental flags, but at this moment seems no other solution is available.    I am wondering why there isn't possibility to use JAAS configuration to authenticate against Kudu with Spark, as it is with Java client for example.     ... View more

Hello, we have exactly the same issue with our structured streaming jobs. We are trying to consume data from Kafka and write it to the Kudu, but when the job exceed 7 days we are getting same errors.  Did someone find solution for this issue?   ... View more

Hi @haridjh  Thanks for reply. Procedure you described in your reply is utilizing HDFS as a store for JAR files used by Spark job. We don't have problem to utilize HDFS in Spark job, problem is when trying to access Ozone FS, e.g. (ofs) when job is submitted via LIVY. 1. Access files on Ozone in spark job e.g.:   df = spark.read.parquet("ofs://ozone-service/volume/bucket/parquet")   2. Python job submitted via Livy:   kinit user   curl --negotiate -k -v -u : -X POST \ -H "Content-Type: application/json" \ --data '{ "file": "ozone_access.py"}' \ https://livy:28998/batches      3. Job is failing with:   Caused by: java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]     When we are trying to access Ozone normally via spark-shell or spark-submit, everything works fine, e.g.:   spark-shell \ --keytab ${KEY_TAB} \ --principal ${PRINCIPAL} \ --conf spark.yarn.access.hadoopFileSystems=o3fs://bucket1.vol1.om.host.example.com:9862     Setting keytab and principal is not possible when submitting job via Livy, because we are using proxy users with Livy.  Thanks.     ... View more

Hi, Any idea how to access Ozone FS when submitting spark job through Livy?    Caused by: java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] at org.apache.hadoop.ipc.Client$Connection$1.run(Client.java:808) ~[hadoop-common-3.1.1.7.1.9.14-2.jar:?] at java.security.AccessController.doPrivileged(AccessController.java:712) ~[?:?] at javax.security.auth.Subject.doAs(Subject.java:439) ~[?:?] at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899) ~[hadoop-common-3.1.1.7.1.9.14-2.jar:?] at org.apache.hadoop.ipc.Client$Connection.handleSaslConnectionFailure(Client.java:771) ~[hadoop-common-3.1.1.7.1.9.14-2.jar:?] at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:866) ~[hadoop-common-3.1.1.7.1.9.14-2.jar:?] at org.apache.hadoop.ipc.Client$Connection.access$3800(Client.java:430) ~[hadoop-common-3.1.1.7.1.9.14-2.jar:?] at org.apache.hadoop.ipc.Client.getConnection(Client.java:1681) ~[hadoop-common-3.1.1.7.1.9.14-2.jar:?] at org.apache.hadoop.ipc.Client.call(Client.java:1506) ~[hadoop-common-3.1.1.7.1.9.14-2.jar:?] ... 48 more     ... View more