@rwinters  You could be right that your issue is incompatibility? Can you check exactly what version of Postgres you upgraded to? It's usually advisable to check the Hortonworks support matrix  before launching any upgrade I only hope it's a dev environment  see screenshot    Here you see PostgreSQL 10.7 is only compatible with Ambari 2.7.4 and HDP 3.1.4  very useful tool.   Happy hadooping  ... View more

@mike_bronson7    The hadoop group encapsulates all the users including hdfs     You do run a # cat /etc/group  You should see someing like like  hadoop:x:1007:yarn-ats,hive,storm,infra-solr,zookeeper,oozie,atlas,ams,ranger,tez,zeppelin,kms,accumulo,livy,druid,spark,ambari-qa,kafka,hdfs,s qoop,yarn,mapred,hbase,knox   So running the -chown should only target the directory in the Diagnostics logs  NEVER  run the               -chown command on /  which is the root directory !!   Can you share your log please  ... View more

@mike_bronson7  That's correct you need to give the path of the directory 🙂 ie usually in hdfs    $ hdfs dfs -chown airflow:hdfs   /path/in/hdfs/where/you/failed/to/write   As you didn't include the path I assumed you'd do that with the -chown command  ... View more

@Kou_Bou  Great ! ... View more

@mike_bronson7  The gid is numeric  value just to indicate the id but the valid groups for airflow  are [airflow and hdfs] $ hdfs dfs -chown airflow:hdfs Should do the magic, please revert Cheers   ... View more

@naveensangam  @jepe_desu  In reference to Invalid KDC administrator credentials issue raised by @naveensangam    I wrote a walkthrough of the solution that resolved the issue for other users like @jepe_desu  who had encountered exactly the same problem.  @naveensangam  can you update the thread if my solution resolved your issue or if not can you share what errors you have. Once you accept an answer it can be referenced by other members for similar issues rather than starting a new thread.    Happy  hadooping   ... View more

@mike_bronson7    That's a classic permissions issue "airflow" is trying to write to that directory but has no permissions as it's owned by alapati user  the inode=alapati. The  easiest solution is to grant the permissions as the hdfs user $ hdfs dfs -chown airflow:{$airflow_group} Most components like Spark, Hive, sqoop  need to access HDFS ... View more

@vishal6193  Can you share how you did your ADLS setup? Please have a look at this Hadoop Azure Support Azure Blob Storage  look at the jars and credentials to create in a secure environment     ... View more