Member since
04-22-2019
9
Posts
1
Kudos Received
0
Solutions
07-02-2019
12:00 AM
@sshao am submitting spark job through livy , below are my conf parameters data = {'kind': 'pyspark','driverMemory':'2G','driverCores':2,'numExecutors': 1, 'executorMemory': '1G','executorCores':1,'conf':{'spark.yarn.appMasterEnv.PYSPARK_PYTHON':'/usr/bin/python3'}} > livy.spark.master : yarn-cluster and in the above explanation you mentioned "livy.spark.deploy-mode" property , i guess it is "livy.spark.deployMode" correct me if am wrong. In addition to above what other config should i change.
... View more
04-23-2019
08:31 PM
@Jay Kumar SenSharma Thanks for the answer, am using python2.7 all files are present in /usr/lib/ambari-server. After adding above python path mentioned am getting the below error Using python /usr/bin/python Setup ambari-server ERROR: File ambari.properties not found in search path $AMBARI_CONF_DIR: /etc/ambari-server/conf Traceback (most recent call last): File "/usr/sbin/ambari-server.py", line 37, in <module> from ambari_server.checkDatabase import check_database File "/usr/lib/ambari-server/lib/ambari_server/checkDatabase.py", line 26, in <module> from ambari_server import serverConfiguration File "/usr/lib/ambari-server/lib/ambari_server/serverConfiguration.py", line 603, in <module> configDefaults = ServerConfigDefaults() File "/usr/lib/ambari-server/lib/ambari_server/serverConfiguration.py", line 512, in __init__ super(ServerConfigDefaultsLinux, self).__init__() File "/usr/lib/ambari-server/lib/ambari_server/serverConfiguration.py", line 366, in __init__ properties = get_ambari_properties() File "/usr/lib/ambari-server/lib/ambari_server/serverConfiguration.py", line 252, in get_ambari_properties conf_file = find_properties_file() File "/usr/lib/ambari-server/lib/ambari_server/serverConfiguration.py", line 245, in find_properties_file raise FatalException(1, err) ambari_commons.exceptions.FatalException: 'Fatal exception: File ambari.properties not found in search path $AMBARI_CONF_DIR: /etc/ambari-server/conf, exit code 1'
... View more
04-23-2019
08:30 PM
@Jay Kumar SenSharma Thank you for the answer, am using python2.7 after adding python path u have give am getting the bellow error, by the way am using ubunut 16.4 so i can not use yum reinstall Setup ambari-server ERROR: File ambari.properties not found in search path $AMBARI_CONF_DIR: /etc/ambari-server/conf Traceback (most recent call last): File "/usr/sbin/ambari-server.py", line 37, in <module> from ambari_server.checkDatabase import check_database File "/usr/lib/ambari-server/lib/ambari_server/checkDatabase.py", line 26, in <module> from ambari_server import serverConfiguration File "/usr/lib/ambari-server/lib/ambari_server/serverConfiguration.py", line 603, in <module> configDefaults = ServerConfigDefaults() File "/usr/lib/ambari-server/lib/ambari_server/serverConfiguration.py", line 512, in __init__ super(ServerConfigDefaultsLinux, self).__init__() File "/usr/lib/ambari-server/lib/ambari_server/serverConfiguration.py", line 366, in __init__ properties = get_ambari_properties() File "/usr/lib/ambari-server/lib/ambari_server/serverConfiguration.py", line 252, in get_ambari_properties conf_file = find_properties_file() File "/usr/lib/ambari-server/lib/ambari_server/serverConfiguration.py", line 245, in find_properties_file raise FatalException(1, err) ambari_commons.exceptions.FatalException: 'Fatal exception: File ambari.properties not found in search path $AMBARI_CONF_DIR: /etc/ambari-server/conf, exit code 1'
... View more
04-23-2019
04:33 PM
@Saumil Mayani pls help
... View more
04-23-2019
04:33 PM
1 Kudo
Not able to kill a spark job/task/stage ,I have created a spark session using livy, a spark job is running even after the session got expired, when i execute yarn "application -list -appStates RUNNING " application which is running the job doesn't get displayed.
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache Spark
-
Apache YARN
04-23-2019
11:49 AM
@Akhil S Naik pls help
... View more
04-23-2019
10:48 AM
@Farhad Heybati pls help i followed your solution given for centos here
... View more
04-23-2019
10:46 AM
Am getting the below error when i execute " ambar-server setup" followed : installation link Using python /usr/bin/python Resetting ambari-server Traceback (most recent call last): File "/usr/sbin/ambari-server.py", line 28, in <module> from ambari_commons.exceptions import FatalException, NonFatalException ImportError: No module named åambari_commons.exceptions
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Hadoop
-
Apache Spark