@mike_bronson7  That's correct you need to give the path of the directory 🙂 ie usually in hdfs    $ hdfs dfs -chown airflow:hdfs   /path/in/hdfs/where/you/failed/to/write   As you didn't include the path I assumed you'd do that with the -chown command  ... View more

@Kou_Bou  Great ! ... View more

@mike_bronson7  The gid is numeric  value just to indicate the id but the valid groups for airflow  are [airflow and hdfs] $ hdfs dfs -chown airflow:hdfs Should do the magic, please revert Cheers   ... View more

@naveensangam  @jepe_desu  In reference to Invalid KDC administrator credentials issue raised by @naveensangam    I wrote a walkthrough of the solution that resolved the issue for other users like @jepe_desu  who had encountered exactly the same problem.  @naveensangam  can you update the thread if my solution resolved your issue or if not can you share what errors you have. Once you accept an answer it can be referenced by other members for similar issues rather than starting a new thread.    Happy  hadooping   ... View more

@mike_bronson7    That's a classic permissions issue "airflow" is trying to write to that directory but has no permissions as it's owned by alapati user  the inode=alapati. The  easiest solution is to grant the permissions as the hdfs user $ hdfs dfs -chown airflow:{$airflow_group} Most components like Spark, Hive, sqoop  need to access HDFS ... View more

@vishal6193  Can you share how you did your ADLS setup? Please have a look at this Hadoop Azure Support Azure Blob Storage  look at the jars and credentials to create in a secure environment     ... View more

@saivenkatg55  Great, can you gauge the improvement in percentage?   Can you back up your ambari.properties and make the below changes to further enhance the Ambari performance I usually script that as a post-install step. Can you then take time and Accept the solution and close the thread so other members can reference it should they encounter a simpler situation.   Purging in the Ambari database from time to time should be an Admin maintenance task    Backup the ambari server properties file # cp /etc/ambari-server/conf/ambari.properties  /etc/ambari-server/conf/ambari.properties.ORIG   Change the timeout of the ambari server # echo 'server.startup.web.timeout=120' >> /etc/ambari-server/conf/ambari.properties # echo 'server.jdbc.connection-pool.acquisition-size=5' >> /etc/ambari-server/conf/ambari.properties # echo 'server.jdbc.connection-pool.max-age=0' >> /etc/ambari-server/conf/ambari.properties # echo 'server.jdbc.connection-pool.max-idle-time=14400' >> /etc/ambari-server/conf/ambari.properties # echo 'server.jdbc.connection-pool.max-idle-time-excess=0' >> /etc/ambari-server/conf/ambari.properties # echo 'server.jdbc.connection-pool.idle-test-interval=7200' >> /etc/ambari-server/conf/ambari.properties   Hope that helps   Happy hadooping   ... View more

@Kou_Bou    I just deployed a cluster yesterday   and never encountered that error          Can you share your documented  steps in a document  if it's too big  to attach use google, I am sure  you might be skipping a step  but before that can you do the 2 steps below    vi /etc/ambari-agent/conf/ambari-agent.ini force_https_protocol=PROTOCOL_TLSv1_2 vi /etc/python/cert-verification.cfg [https] verify=disable To disable SSL/TLS then retry that should resolve your problem     ... View more

@lalprasanth  That's obvious because you have double $$  in the sqoop command you forgot to remove  the one in the command I posted please do that and revert. Thank you ... View more