Thank you @Nishant Bangarwa! "No module psycopg2" solved! Though I am getting another error now: Error: Fatal: no pg_hba.conf entry for host "x.x.x.x", user "x", database "x", SSL off. Plus, I am not sure about the user & password for my Postgres database ... View more

@Khouloud Landari, Did you check if there is enough space in /tmp folder in all the nodes (workers + master). Also check the permissions of the folder. . -Aditya ... View more

@Khouloud Landari Do you see it stuck after those WARN Service SparkUI could not bind to port 4041? If that is the case I think the problem maybe is not able to start an application on yarn. What happens is spark2 pyspark launches a yarn application on your cluster and I think this is what is probably failing. Try this command and let me know if this works: SPARK_MAJOR_VERSION=2 pyspark --master local --verbose Also I would advise you to check the Resource Manager logs. RM logs can be found on RM host under /var/log/hadoop-yarn This will probably show what the problem is with yarn and why your zeppelin user is not able to start applications on the hadoop cluster. HTH ... View more