Support Questions

Find answers, ask questions, and share your expertise

Can I use Nifi to run multiple SQL statements ?

avatar
New Contributor

I have 6-7 sql queries which needs to be executed in one after the other.

if my query fails at 4 step so my 4th sql query fails which is an insert and I want to retry it without the manual intervention can nifi do that?

 

also please guide regarding airflow if we can achieve the same in airflow by creating multiple dags to run one after the another. Please guide and help

1 REPLY 1

avatar
Master Guru

wrap your SQL in a view / procedure / function or other Database native grouping of statements is smartest.

 

With running multiple sql statements you may want to use Cloudera CDE, Cloudera Machine Learning jobs, YARN Spark Jobs or Airflow.

 

https://www.datainmotion.dev/2019/10/migrating-apache-flume-flows-to-apache_15.html

 

https://www.datainmotion.dev/2020/12/simple-change-data-capture-cdc-with-sql.html