I have design the nifi flow like this
selectHiveQL->convertAvroToJson->SplitJson->EvaluateJsonPath->RoutOnAttribute(depends on condition the query goes on)
->presto or ->ReplaceText->putHiveQL
hive table is
query_id job_type query
1 presto create table hive.db.table as select * from mysql.db.table
2 hive ALTER table db.table set tblproperties ("orc.create.index"="true")
3 hive ALTER table db.table set tblproperties ("transactional"="true")
4 presto create table hive.db.table_stage as select * from mysql.db.table
selectHiveQL-select query on hive table contain query_id, job_type and query
convertAvroToJson->convert avro format into json
SplitJson-split the record
EvaluateJsonPath-set the query_id, job_type and query attribute
RoutOnAttribute-depending on condition of job_type that query is executed in presto or hive
presto-custom processor executed the query
ReplaceText-set the putHiveQL query
putHiveQL-execute hive query
1) How to stop the flow if any query failed from any processor.
2) I have tried to connect the presto failure to presto processor that time if query failed the next query not goes to presto processor but failed query is continuously executed in presto processor.
3) if the query is alternate and presto query failed that time hive query goes to hive processor(backpressure of hive failure also set to 1).
4) How to send email notification if the flow is successfully completed or failed?
Please provide your suggestions .
Thanks in advance .
Here is an article I wrote about handling failures in NiFi: https://community.hortonworks.com/articles/76598/nifi-error-handling-design-pattern-1.html
It describes how to retry failures X times, then send an email, then wait for administrative input. This might help you address the requirements of your solution.
You could also add a PutEmail processor on the "Success" relationship to send an email after processing succeeds.