Support Questions
Find answers, ask questions, and share your expertise

How will i increse the performance on partition column using NiFi?

Dear Team,

I am ingesting the data from Oracle to Postgre, I have index on primary key in oracle and cre_date column has partition in postgre. I am using QueryDataBaseTale-->PutDatabaseRecord to import the data and max-column cre_date, while pushing the data into postgre it's very slow, how will i increase performance do I need to use any parameter for partition column? without partition column it's very fast..

Thanks in advance.


Super Guru

I don't think there's much you can do in NiFi, unless you can tune parameters in the JDBC URL, or use PartitionRecord to ensure each FlowFile has only one unique value for the partition column, but even then it looks like you'd need some special logic on the PostgreSQL side. I found this article about maybe this being more performant in later or upcoming versions of PostgreSQL, and this one about doing partitioning via child tables, not sure how your table(s) are set up but maybe some of this will help.