Member since
07-11-2022
13
Posts
2
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1949 | 07-12-2022 09:46 PM |
07-14-2022
06:42 PM
How do you specify a property when using PropertyValue attribute ? for example, when we have 2 attributes, A and B how do we specify that we want to use the PropertyValue of property A ? I tried ${PropertyValue:A} but it did not work
... View more
Labels:
- Labels:
-
Apache NiFi
07-14-2022
12:59 AM
Here is my current Nifi workflow. 1. Getfile bring the CSV file into the flow 2. As you can see, they are saved in separated tables named 'cattle_auction_data' and 'cattle_auction_invalid_data' after going through UpdateRecord processor to add a column with the original filename. Here is what I am trying to do. The Query Record processor performs two sql queries, 'valid' and 'invalid' I would like to log the sql queries and their time of execution, make them go through updaterecord processor to add a column with the filename and save them into the table named 'validation-log' . that the final product looks like below filename | sql_query | time_of_execution 2022.csv | select * from FLOWFILE where gradenm= '999'... | 22-07-14 07:33:26 2022.csv | select * from FLOWFILE where gradenm<> '999'.. | 22-07-14 07:33:26 I think that 'executequery.query.executiontime' attribute can deal with the time of execution part, but I don't know how to log the sqlquery and eventually save them into a table. A lot of awesome people gave me answers in previous posts but since I am a NB to Nifi I had to ask another question here. Thanks for the help in advance
... View more
Labels:
- Labels:
-
Apache NiFi
07-13-2022
11:06 PM
Actually can you show me both methods ? Thank you so much for help !
... View more
07-13-2022
08:00 PM
Oh I guess I didnt make my question clear here. for example , when I have 2 SQL queries (a, b) set on my Queryrecord Processor, I want to find out if I can keep a log like SQL query | Timestamp (Time of the query execution) a | 20220714, 12:00 b | 20220714 , 12:00
... View more
07-13-2022
04:44 PM
Thank you for the reply !. Can you suggest a processor that will perform the task you mentioned? Would LogAttribute do ?
... View more
07-13-2022
01:06 AM
Hello! Attached below is a flow I have created to perform a simple sorting task with a given CSV file 1. Getfile processor brings the target CSV file into the flow 2. QueryRecord processor executes SQL commands and send the flowfile two ways 3. Each flow file is saved into Postgresql database with PutDatabaseRecord processors. What I am trying to do is. 1. Keep a log of all SQL commands that are executed on Queryrecord processor and keep timestamp (When they were executed) of them 2. Save the log into a Postgresql table named 'validation_log" that has 3 columns. (CSV file name, SQL command, Timestamp) How can I make this work?
... View more
Labels:
- Labels:
-
Apache NiFi
07-12-2022
09:46 PM
1 Kudo
Ok just in case anyone is suffering from the same problem as me. Since I am running this on docker compose, instead of localhost:5432, I changed it to <container name>:5432 and its working now
... View more
07-12-2022
09:24 PM
Hello. Im currently running nifi and postgresql on docker compose. I have a problem setting up the DBCPConnectionPool controller. I keep getting the error message below. Below is my configuration for DBCPConnectionPool I have checked the db name and docker compose yml file but nothing seems wrong on that end. What could be causing the problem here?
... View more
Labels:
- Labels:
-
Apache NiFi
07-12-2022
07:28 PM
1 Kudo
Thank you so much for the help! this solved my problem
... View more