Member since
08-29-2023
2
Posts
1
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3025 | 09-05-2023 09:26 AM |
01-06-2024
09:34 PM
Hi @Faisal_555 , @DianaTorres we've also encountered this error although the Ozone service in the Base cluster is healthy. We have seen all the containers' logs but haven't found any clear problems with Ozone service in the Base cluster. Please help us, we have stuck in this problem for one week by now.
... View more
09-05-2023
09:26 AM
1 Kudo
i have also encountered this weird situation interpreter died when using livy pyspark on spark3. I have lost for 2 days debugging this errors and i've found out that we need to: 1. In the "Spark 3" service on Cloudera manager portal, set spark pyspark python and python driver executable files configuration in the section "Spark 3 Client Advanced Configuration Snippet (Safety Valve) for spark3-conf/spark-defaults.conf" as below: spark.pyspark.driver.python=/usr/bin/python3 spark.pyspark.python=/usr/bin/python3 2. Restart "livy for Spark 3" service on cloudera manager. 3. Restart the Zeppelin Livy Interpreter. => After being restarted, the zeppelin's livy interpreter on Spark 3 can execute %pyspark interpreter. => Note that, the python version to work with livy spark 3 on zeppelin must be less than or equal 3.7. otherwise it will generate error "required field "type_ignores" missing from Module" when executing pyspark script on Zeppelin.
... View more