After installing CDSW 1.6 on Cloudera 7.1, in every job I start I see this message: FileNotFoundError: [Errno 2] No such file or directory: '{{SPARK_HOME}}/./bin/spark-submit': '{{SPARK_HOME}}/./bin/spark-submit' After re-installing Spark gateway and adding Hive and hdfs instances on the machine with CDSW nothing has changed. spark-shell works fine.   After some investigation, I've changed file  /var/lib/cdsw/client-config/spark-conf/ spark-defaults.conf , namely,  export SPARK_HOME= {{SPARK_HOME}} to an actual value of SPARK_HOME. It helped, and the exception was gone. But it feels like a hack or a bug, and I would like to know if there's a better way to solve this? ... View more

We have Cloudera 7.0.3 cluster, with Kerberos enabled, Ranger, and FreeIPA as KDC. OS RHEL 7.8 on all hosts in cluster. On a FreeIpa host, in krb5kdc.log, one can see TGS_REQ request like this every 30 seconds: Jun 29 18:49:32 "freeIPA_host" krb5kdc[45414](info): TGS_REQ (1 etypes {18}) "freeIPA_ip": ISSUE: authtime 1593445168, etypes {rep=18 tkt=18 ses=18}, hdfs/some.host@SOMED.DOMAIN for HTTP/some.host@SOMED.DOMAIN After reading this article: https://blog.cloudera.com/hadoop-delegation-tokens-explained/ dfs.namenode.delegation.token.renew-interval and fs.namenode.delegation.token.max-lifetime params were added to both KMS and HDFS NameNode configs, to no avail. After reading this question: How do you set the Kerberos ticket lifetime from Java? Newer version of Java were installed on all machines in the cluster. https://bugs.openjdk.java.net/browse/JDK-8044500 New jdk was also explicitly set in the Cloudera Manager as JAVA_HOME, but again, the problem is still there.   What could be the cause of this problem and how can it be fixed? ... View more