Member since
03-25-2021
3
Posts
1
Kudos Received
0
Solutions
02-10-2023
01:11 AM
1 Kudo
I have a flow which loads a CSV file, transforms it to a json format with one data row. This gives me a json array with one entry being one data row. Right now I split the flowfile so I end up with one flowfile per data row and then load these to a postgres via a PutDatabaseRecord processor. All of this works fine, but the splitting feels a bit inefficient and my flows end up with millions of flowfiles flying around. I tried giving the PutDatabaseRecord processor a json array, but that just threw an error. Is there a way to have multiple datarows in a flowfile and load these to the database?
... View more
Labels:
- Labels:
-
Apache NiFi
03-26-2021
04:39 AM
I solved this issue by creating a custom processor that removes '/u00A0' from the content of the flowfile. This works, but is probably not the ideal solution. It would be great if someone has ideas how to fix this in a more intuitive way
... View more
03-25-2021
05:05 AM
Hey all, I'm currently using the following processor chain to read a csv and load the data into a postgres table. ListFile -> FetchFile -> QueryRecord -> Split Record -> PutDatabaseRecord Some of my data points have a non-breaking space before or after the information. I tried using trim and replace in the QueryRecord processor, but it didn't remove the space. Any ideas how I could get rid of this type of space?
... View more
Labels:
- Labels:
-
Apache NiFi