Member since
10-04-2017
13
Posts
0
Kudos Received
0
Solutions
01-23-2018
01:03 PM
@gnovak thx for the answer, I was able to make it work according to your explanation with one small addition. I did have to allow excessive queue usage with: yarn.scheduler.capacity.root.test1.user-limit-factor=2 I have another question, I tried to define test2 to be fair and submit two application to it, but no preemption has happened and one app simply took 100% of the queue. I tried posting from different users and everything but couldn't make it happen. Is there another configuration I need to set for it to work?
... View more
01-22-2018
02:46 PM
I try to run two Yarn queues where if only one queue is active it will consume all the resources and once a job will arrive to the second queue Yarn will preempt some of the resources of the first queue to start the second job. For that I follow the manual written YARN Preemption with Spark using a Fair Policy but for some reason only the leaf queues has the option of fair ordering policy. In fact the selection for ordering policy doesn't exist at all on non leaf queue. I am running on hdp version: 2.6.3.0-235 I have followed the instruction perfectly on the manual. I have also tried manually set the fair policy in 'Scheduler' tab of Yarn, when I do that Yarn crashes and would not start. What can I do? Screenshot are attached
... View more
Labels:
- Labels:
-
Apache Spark
-
Apache YARN
10-10-2017
12:06 PM
It was exactly the issue, only the version I had installed is 9.2. I followed the instructions for the pre-installed database and it worked perfectly. Thanks alot.
... View more
10-05-2017
07:41 AM
i am installing ambari-server with RHEL7.3 but unfortunately it fails, when I run it with verbose: `ambari-server setup -v` I am getting the following error: Traceback (most recent call last): File "/usr/sbin/ambari-server.py", line 951, in <module> mainBody() File "/usr/sbin/ambari-server.py", line 921, in mainBody main(options, args, parser) File "/usr/sbin/ambari-server.py", line 873, in main action_obj.execute() File "/usr/sbin/ambari-server.py", line 78, in execute self.fn(*self.args, **self.kwargs) File "/usr/lib/python2.6/site-packages/ambari_server/serverSetup.py", line 1144, in setup _setup_database(options) File "/usr/lib/python2.6/site-packages/ambari_server/serverSetup.py", line 960, in _setup_database dbmsAmbari.setup_database() File "/usr/lib/python2.6/site-packages/ambari_server/dbConfiguration.py", line 150, in setup_database self._setup_local_database() File "/usr/lib/python2.6/site-packages/ambari_server/dbConfiguration_linux.py", line 479, in _setup_local_database retcode, out, err = self._configure_postgres() File "/usr/lib/python2.6/site-packages/ambari_server/dbConfiguration_linux.py", line 748, in _configure_postgres PGConfig._configure_pg_hba_postgres_user() File "/usr/lib/python2.6/site-packages/ambari_server/dbConfiguration_linux.py", line 727, in _configure_pg_hba_postgres_user pg_hba_conf_file_content_in = sudo.read_file(PGConfig.PG_HBA_CONF_FILE) File "/usr/lib/python2.6/site-packages/resource_management/core/sudo.py", line 144, in read_file with open(filename, "rb") as fp: IOError: [Errno 2] No such file or directory: '/var/lib/pgsql/data/pg_hba.conf' As you can see Ambari searches the conf file in '/var/lib/pgsql/data/pg_hba.conf' where its actual location is: '/var/lib/pgsql/9.2/data/pg_hba.conf' I couldn't find any Ambari config for specifying that path, any help would be really appreciated.
... View more
Labels:
- Labels:
-
Apache Ambari
10-04-2017
09:15 PM
Thank you for the quick and accurate response! this solved the issue.
... View more
10-04-2017
07:00 PM
I try to install Ambari-server under RedHat 7.3, python 2.7.5 I got the following Ambari and HDP repositories and ran `yum clean all` just to be sure: http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.5.2.0/ambari.repo http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.2.0/hdp.repo After I run yum `install ambari-server` i try to setup it using `ambari-server setup` and I am getting the following issue: Traceback (most recent call last): File "/usr/sbin/ambari-server.py", line 40, in <module> from ambari_server.serverUpgrade import upgrade, upgrade_stack, set_current File "/usr/lib/python2.6/site-packages/ambari_server/serverUpgrade.py", line 51, in <module> from ambari_server.setupMpacks import replay_mpack_logs File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 30, in <module> from ambari_commons.inet_utils import download_file, download_file_anyway ImportError: cannot import name download_file_anyway It is weird to me that the ambari-server packages are installed under python2.6 (which is not installed) since the RHEL7 version should work with python2.7. What is the issue? how can it be solved?
... View more
Labels:
- Labels:
-
Apache Ambari