question Re: Installing ApacheAirflow on Ambari using mpack in Support Questions

Installing ApacheAirflow on Ambari using mpack

math23 — Wed, 24 Jun 2020 12:34:23 GMT

Hi All,

I am new on Ambari. I use Rhel7 and i installed Ambari server and have been configured it succefully on my server.

So, i want to install Airflow on this existing ambari server. I've tried to install it with the command

"ambari-server install-mpack --mpack=/opt/airflow-service-mpack.tar.gz"

and i got this python message :

Using python /usr/bin/python Usage: {start|stop|restart|setup|setup-jce|upgrade|status|upgradestack|setup-ldap|sync-ldap|set-current|setup-security|setup-sso|refresh-stack-hash|backup|restore|update-host-names|check-database|db-cleanup|enable-stack} [options] Use --help to get details on options available. Or, simply invoke ambari-server.py --help to print the options.

Need help Please.

Thanks,

Math

Re: Installing ApacheAirflow on Ambari using mpack

stevenmatison — Wed, 24 Jun 2020 13:34:04 GMT

@math23 I suggest to try without /opt/ maybe in /tmp or /root (~/). Also make sure executing with the right user permissions (sudo if needed). You can also add --verbose into the main command to get more useful output.

What version of Ambari and HDP?

I love playing with mpacks and I have made quite a few of my own. I am going to experiment with this airflow one (link) and report back when I get it installed.

Re: Installing ApacheAirflow on Ambari using mpack

math23 — Wed, 24 Jun 2020 14:00:08 GMT

@stevenmatison I use root user for installation,

I've already tried to change my command as follow "ambari-server --install-mpack --mpack=/tmp/airflow-service-mpack.tar.gz --verbose" but always the same message.

I'm using Ambari 2.2 and HDP 2.4.

Re: Installing ApacheAirflow on Ambari using mpack

stevenmatison — Wed, 24 Jun 2020 14:30:43 GMT

@math23 I will give it a go with that version. Where did you get the mpack archive? Just to make sure I get same one. The one I used is from the GitHub I linked above.

ambari-server install-mpack --mpack=https://github.com/miho120/ambari-airflow-mpack/raw/master/airflow-service-mpack.tar.gz --verbose

ambari-server uninstall-mpack --mpack-name=airflow-ambari-mpack

I was just working in HDP3 and Ambari 2.7.4 and got mpack installed without any issues. When adding the service I just got tons of issues with the mpack being old airflow version and related dependency errors. I am going to have to upgrade the services and make a new management pack to make it work in HDP3 with airflow 1.10.10.

Re: Installing ApacheAirflow on Ambari using mpack

stevenmatison — Wed, 24 Jun 2020 16:51:24 GMT

@math23 I have confirmed your version of ambari is not able to install management packs. The error you are getting is indicating "install-mpack" is not an allowed ambari-server command. You will need to use a newer version of Ambari/HDP.

If you really have to install in old version you can also deliver a custom service to ambari without a management pack (.tar.gz). You can unpack the contents and copy the airflow service into right ambari version folder directly. I don't have the steps to do that for airflow, but some other examples of how to get the version, and what the service folder contents look, are including in these custom services:

https://github.com/EsharEditor/ambari-hue-service

https://github.com/abajwa-hw/ambari-flink-service

Re: Installing ApacheAirflow on Ambari using mpack

math23 — Thu, 25 Jun 2020 05:26:33 GMT

@stevenmatison , so there is no solution for my problem except upgrading my HDP and Ambari version.

Thanks for your reply

Math

Re: Installing ApacheAirflow on Ambari using mpack

stevenmatison — Thu, 25 Jun 2020 12:08:36 GMT

@math23 No, you can also just copy the service directly into your ambari. I suggest this way if you can't easily get a newer cluster. I tried to point you at how to install the custom service in the accepted solution. A management pack just makes that task a lil easier, as well as being applicable to different stack versions.