Member since
04-22-2019
9
Posts
1
Kudos Received
0
Solutions
07-02-2019
12:00 AM
@sshao am submitting spark job through livy , below are my conf parameters data = {'kind': 'pyspark','driverMemory':'2G','driverCores':2,'numExecutors': 1, 'executorMemory': '1G','executorCores':1,'conf':{'spark.yarn.appMasterEnv.PYSPARK_PYTHON':'/usr/bin/python3'}} > livy.spark.master : yarn-cluster and in the above explanation you mentioned "livy.spark.deploy-mode" property , i guess it is "livy.spark.deployMode" correct me if am wrong. In addition to above what other config should i change.
... View more