Any solution this ... View more

Hello @agsumeet , @direcision , @Dorown and @abinanths ,   thank you for reaching out! I have checked internally and the issue described in this thread looks like HIVE-16683 / SPARK-26932   As of now, there is no fix backported to any CDH release (at the time of writing it means CDH6.3.3 and earlier).   Should you have a Cloudera Support Subscription, please file a support case with us for further assistance!   Thank you: Ferenc ... View more

Step Context Start Time Duration Actions Checking that NameNode Data Directories on host either do not exist, or are writable and empty. Can optionally clear directories. Process host-validate-writable-empty-dirs (id=55074) on host (id=167) exited with 0 and expected 0 Jul 25, 4:13:54 AM 1.52s Checking that JournalNode Edits Directory on host either does not exist, or is writable and empty. Can optionally clear directory. Process host-validate-writable-empty-dirs (id=55075) on host (id=167) exited with 0 and expected 0 Jul 25, 4:13:56 AM 1.43s Stopping services that depend on HDFS service HDFS. All services successfully stopped. Cluster LIVE Jul 25, 4:13:58 AM 53.26s Transitioning NameNode on host to active mode and NameNode on host to standby mode. Successfully failed over. Jul 25, 4:14:51 AM 23.37s Stopping roles of HDFS service HDFS. Successfully executed command Stop on service HDFS HDFS Jul 25, 4:15:15 AM 9.46s Putting NameNode on host into safe mode. The NameNode successfully entered Safemode. Jul 25, 4:15:24 AM 7.12s Saving namespace of NameNode on host Command aborted because of exception: Command timed-out after 90 seconds ... View more

I use NiFi PutS3Object to load a file into an S3 bucket and use an Airflow S3Sensor to monitor that location and kick off an Airflow DAG. I'm trying to change my process to do that via the NiFi InvokeHTTP processor calling the Airflow experimental API ( https://airflow.apache.org/api.html ), because I want the process more tightly coupled. Does anyone know of a better way? ... View more