Member since
11-14-2019
51
Posts
7
Kudos Received
3
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 2767 | 05-26-2023 08:40 AM | |
| 3032 | 05-22-2023 02:38 AM | |
| 2342 | 05-15-2023 11:25 AM |
05-26-2023
08:40 AM
Thank You @MattWho @SAMSAL for your wonderful suggestions. I achieved solution using JolttransformJSON processor, where i used below jolt specification [{ "operation": "default", "spec": { "*": { "error": "${putdatabaserecord.error}" } } }] Thanks again for your thought provoking answers..
... View more
05-25-2023
08:36 AM
Hi All, I am using putdatabaserecord to store a value into DB. Now i have a flow based on failure link to (this processor), that this failed record will be stored in disc for troubleshooting. I can store the failed record in disc by using putfile processor. Now i wanted it more informative during debugging, as the attribute "putdatabaserecord.error" contains the error reason, i want to append the flowfile content with this error reason. I tried to acheive this using below flow.. putsql/putdbrecord -> failure-> updateattribute (this is to give the flowfile a realistic name)-> attributesToJson (destination flowfileattribute/content -> putfile but in attributestoJson , if the destination is kept as flowfileattribute, the flowfile dont get this attribute or if it is kept as flowfile content, original flowfile content is replaced... My requirement is to append this single attribute to the existing flowfile content. Any help or suggestion is greatly appreciated.
... View more
Labels:
- Labels:
-
Apache NiFi
05-22-2023
02:38 AM
2 Kudos
a bit research helped to find out the way from Nifi toolkit documentations $ nifi set-param -pcid abc -pn <name> -pv <value> -ps <true/false for senstivity> and it automatically refreshed/bought-in the updated values in UI.
... View more
05-22-2023
01:28 AM
I am using Nifi parameter context to avoid hardcoding multiple variables. I used toolkit to create parameter context and used params inside my flow. Now I want to update some values of this existing parameter context.. 1. to update some values or 2. to add new parameters. I could not find a way using toolkit in the documentation. I see an answer here https://community.cloudera.com/t5/Support-Questions/Update-parameter-contexts-dynamically-using-REST-API/m-p/347697 But i am looking for any solution based on toolkit or other easier approach, so that it can be easily handled while deployments to next environments or while updating it in production environments. Please help me with your approach or suggestions.
... View more
Labels:
- Labels:
-
Apache NiFi
-
NiFi Registry
05-22-2023
01:16 AM
Thanks @MattWho for the answer. Does it also has an alternative with toolkit ? or any other easier process as Nifi evolved recent days?
... View more
05-15-2023
11:25 AM
2 Kudos
Just wanted to update that I used flow file attributes to solve this. putslack's Webhook Text supports expression language!!!
... View more
05-15-2023
08:48 AM
Hi All, I am trying to send slack alerts from Nifi whenever there is a failure in flow. I have some 10 different processor flows, I want to use a single putslack for all those flows but the message in slack is dynamic.. Example: record failed for processor1 record failed for processor2 meaning just a variable changes in slack message. and this variable i should send from processor flows. How can putslack processor know from which previous processor it got the request ? But I don't know how to make it possible. I kept a output port that collects all flowfiles of failures. Now i created a putslack processor that connects to this output port.. but i dont see a option on how can i send variables dynamically. Can someone please guide me with suggestions, or correct me if this ideation is wrong..
... View more
Labels:
- Labels:
-
Apache NiFi
05-10-2023
06:42 AM
Thanks for pointing out. @steven-matison Are you trying to mention that if the source DB change some column formats, add/delete new/old columns ? (I am taking care of these, in the sense, whenever source changes structure, destination is already mimicking it outside Nifi.) So ideally i was thinking this will work indefinitely and they will be in sync. and both of the DBs are mysql. If my thinking is wrong, can you please enlighten me on why i should not relay on this kind of syncing ? What kind of issues are you anticipating in long run ?
... View more
05-10-2023
06:36 AM
Hey @cotopaul Thanks for your reply. ohh disappointing from Nifi.. 😞 But I am not sure why this limitation for this processor. Out of your 2 options i am interested in 1st one. May be I can also query the API for Processors status ( "/processors/run-status-details/queries Submits a query to retrieve the run status details of all processors that are in the given list of Processor IDs") instead of bulletin board. Can you please share the Nifi flow template for option 1 ? Thanks for your hacky ways. 🙂
... View more
05-09-2023
10:23 AM
My Nifi Flow starts with capturechangeMysql processor. I am trying to enhance my flows by adding email alerts whenever a processor fails. I understand I can use PutEmail for this purpose. my flow fails most of the time at the capturechangeMysql because of Binlogs or connections or some other random issues. My thinking is to create a link out of failure relationship of this processor to PutEmail, But there is no failure relationship for this processor. (It only has success relation.) In this scenario how can i alert when there is a failure in capturechangeMysql processor ? Please share your thoughts or if i am missing something or other alternatives.
... View more
Labels:
- Labels:
-
Apache NiFi