Member since
01-07-2024
1
Post
0
Kudos Received
0
Solutions
01-07-2024
08:53 PM
I am reading raw data from a Mongo db and loading it to Postgre db. I have the following workflow which seems to be working fine. I just need some help in figuring out how to truncate the table before I start with the data load. Nifi Flow: GetMongo -> EvaluateJSONPath -> ExecuteSQL This flow seems to work fine, but every time I run this, I need the table to be truncated first and then start the load. Can someone please help me with some info on how I can achieve these? So far, I've been trying to use ExecuteSQL first before GetMongo, but GetMongo seems unable to run correctly because of the predecessor, it does not load any data at all when I have the predecessor. Been checking this thread https://community.cloudera.com/t5/Support-Questions/Nifi-Truncate-and-Load-to-mysql-db-table/td-p/232065 too and use failure and success but still not loading correctly. I've also tried to use PutSQL after GetMongo, it works for small data, but if the data is large, then the flow starts queuing and messes up PutSQL, it runs the PutSQL processor before all the data loaded, gives me uneven data from source and destination. Thanks, Ridwan
... View more
Labels:
- Labels:
-
Apache NiFi