Member since
07-11-2022
13
Posts
2
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2001 | 07-12-2022 09:46 PM |
07-15-2022
05:39 AM
Not sure I understand the question, which processor/property are you taking about? Basically if you want to access a flowfile attribute value in expression language you can use ${A} in case you have an attribute called A
... View more
07-14-2022
12:59 AM
Here is my current Nifi workflow. 1. Getfile bring the CSV file into the flow 2. As you can see, they are saved in separated tables named 'cattle_auction_data' and 'cattle_auction_invalid_data' after going through UpdateRecord processor to add a column with the original filename. Here is what I am trying to do. The Query Record processor performs two sql queries, 'valid' and 'invalid' I would like to log the sql queries and their time of execution, make them go through updaterecord processor to add a column with the filename and save them into the table named 'validation-log' . that the final product looks like below filename | sql_query | time_of_execution 2022.csv | select * from FLOWFILE where gradenm= '999'... | 22-07-14 07:33:26 2022.csv | select * from FLOWFILE where gradenm<> '999'.. | 22-07-14 07:33:26 I think that 'executequery.query.executiontime' attribute can deal with the time of execution part, but I don't know how to log the sqlquery and eventually save them into a table. A lot of awesome people gave me answers in previous posts but since I am a NB to Nifi I had to ask another question here. Thanks for the help in advance
... View more
Labels:
- Labels:
-
Apache NiFi
07-13-2022
11:20 PM
1 Kudo
@Lewis_King , Here's an idea. You can fork the "a" output of the QueryRecord processor and send it to a sequence of processors as shown below: The ReplaceText processor will simply replace the entire contents of the flowfile with the information you want to register in the log. For example: This will produce one row per flow file with the source type ("a") and the timestamp. You can them send these to a MergeRecord to avoid saving to many small log files and them to a PutFile to persist the log. You can process the "b" output in a similar way. Cheers, André
... View more
07-12-2022
09:46 PM
1 Kudo
Ok just in case anyone is suffering from the same problem as me. Since I am running this on docker compose, instead of localhost:5432, I changed it to <container name>:5432 and its working now
... View more
07-12-2022
07:28 PM
1 Kudo
Thank you so much for the help! this solved my problem
... View more
07-12-2022
05:41 PM
I am an absolute newbie and was following other tutorials. 😂 I thought it was necessary to convert it to JSON to store the data into Postgresql. Thanks a lot for the help !
... View more