Support Questions
Find answers, ask questions, and share your expertise

Airflow installation including Kerberos authentication from the localexecutor to the celeryexecutor

Airflow installation including Kerberos authentication from the localexecutor to the celeryexecutor

New Contributor

I am trying to upgrade our Airflow installation including Kerberos authentication from the localexecutor to the celeryexecutor. Currently, we run the airflow installation from only one server.

The exact same sqoop job fails using the celeryexecutor due to an kerberos authentication error while the localexecutor is successful:

Client cannot authenticate via:[TOKEN, KERBEROS]; Host Details : local host is: "lsrv****.linux.****/10.251.128.148"; destination host is: "lsrv***.linux.****":8020; , while invoking ClientNamenodeProtocolTranslatorPB.getFileInfo over lsrv****.linux.****/10.251.128.104:8020 after 1 failover attempts. Trying to failover after sleeping for 1377ms.'

kerberos settings:

[kerberos]
ccache = /tmp/krb5cc_32606

# gets augmented with fqdn
principal = airflow
reinit_frequency = 3600
kinit_path = kinit
keytab = /var/lib/airhome/.certs/airflow.keytab

Is there anything that needs to change in the kerberos or celery setup to have kerberos work in combination with celery executor? Or does anything need to change in the Cloudera Hadoop settings? (e.g.: hadoop.security.token.service.use_ip?)

Does it have to do something with the additional ip adress mentioned in the error message?

Don't have an account?