Nifi - how to run a simple standalone INSERT SQL statement

ebeb — Thu, 10 Feb 2022 03:52:28 GMT

Hello Experts,

I want to run a simple Hive Insert SQL statement in Nifi periodically for example:

insert overwrite Table1 select * from Table2;

All SQL values are fixed hardcoded and don't need to be changed dynamically in the flow. As a newbie I was thinking I could write the flow as:

ReplaceText->success->PutHiveQL

Search Value: (?s)(^.*$)

Replacement Value:  insert overwrite Table1 select * from Table2; 
 
But I have an error in ReplaceText which probably needs an incoming flowfile which I don't have since the SQL is fixed hardcoded:
"Upstream Connections is invalid because Processor requires an upstream connection but currently has none." 
 
The other option I could try but not sure it will work is:
GenerateFlowFile -> success -> PutHiveQL
GenerateFlowFile Custom Text: insert overwrite Table1 select * from Table2;
 
 

Re: Nifi - how to run a simple standalone CTAS SQL statement

araujo — Thu, 10 Feb 2022 03:49:07 GMT

The alternative your mentioned of using GenerateFlow file is the way to go here.

question Nifi - how to run a simple standalone INSERT SQL statement in Support Questions

Nifi - how to run a simple standalone INSERT SQL statement

Re: Nifi - how to run a simple standalone CTAS SQL statement