question Re: SQL loader using PutSQL in Archives of Support Questions (Read Only)

SQL loader using PutSQL

yahya_najjar — Mon, 19 Aug 2019 11:12:36 GMT

Hi all,

Am trying to build SQL loader using NiFi, the thing which is a bit strange is that I have to split the input file (containing insert statments) to multiple files each containing only one SQL insert, is this the best practice ? why I cant send the whole file to putSQL processor ?

Thanks

Re: SQL loader using PutSQL

mburgess — Mon, 21 Nov 2016 21:31:27 GMT

PutSQL has a mechanism for batching together statements that were split by processors such as SplitText. Set the "Support Fragmented Transactions" property to true, and PutSQL will wait until all flow files with the same fragment.identifier have arrived, then it will process them all as a single batch.

There has also been talk of implementing the same improvement for PutSQL as is being done for PutHiveQL (NIFI-3031), to support multiple statements from a single flow file. Please feel free to file a Jira for this if you like.

Re: SQL loader using PutSQL

yahya_najjar — Tue, 22 Nov 2016 20:19:49 GMT

Thanks Matt, very helpful

is there anyway to use "Load into" instead of SQL insert ?

Re: SQL loader using PutSQL

mburgess — Wed, 23 Nov 2016 02:18:50 GMT

Yes, PutSQL accepts any SQL statement (except Callable statements like stored procedures) that does not return a result set, so DDL/DML commands such as LOAD INTO or CREATE TABLE, etc. are supported.

Re: SQL loader using PutSQL

yahya_najjar — Wed, 23 Nov 2016 03:49:59 GMT

big thanks really...