Member since
11-14-2019
51
Posts
6
Kudos Received
3
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1178 | 05-26-2023 08:40 AM | |
1297 | 05-22-2023 02:38 AM | |
1198 | 05-15-2023 11:25 AM |
06-14-2023
02:29 AM
I currently use Nifi changecaptureMysql processor to read Binlogs from Mysql 5.7. (encryption not supported) I got to know that Mysql 8.0 supports binlogs to be encrypted. and soon I want to upgrade. Currently Nifi is able to read un-encrypted bin logs successfully, with just DB username and password. Can someone please help me understand what changes to be done to the processor configuration that it will read encrypted Binlogs ? Do we need to provide any keys ? to decrypt the logs ? Please suggest.
... View more
Labels:
- Labels:
-
Apache NiFi
06-08-2023
02:11 AM
1 Kudo
@MattWho Thanks much for clearing my confusions around on - how my jobs running automatically when i restart server.. I encountered restart couple of times but never know this feature. You are awesome!!
... View more
06-07-2023
04:11 AM
Hi Team, I am new to Nifi servers. I am currently working on a Development project where i developed a job to fetch changes from Mysql databases and apply them in a warehouse. I have 70 similar but different databases and therefore 70 Nifi jobs. The final aggregated database is around 100 GB. and daily some 1GB changes from all databases. I am currently working on development environment and using a single server. Now when i move to production with 70 Nifi jobs running all the time, do I need to think of a Nifi cluster ? or it is fine with single server ? What factors decide cluster vs stabdalone ? Also How can I handle upgrades to Nifi in future, Is it better for me to use Docker ? or having a normal instance is better ? Please help me with your valuable knowledge..
... View more
Labels:
- Labels:
-
Apache NiFi
06-07-2023
02:49 AM
Hi All, I am running Nifi on a single Development server. (NAME="Debian GNU/Linux" VERSION_ID="10") I start the service using command line - bin/nifi.sh start During the initial phases as i experiment with RAM, CPU and disk i need to restart machine sometimes. When machine restarted I use above command to start Nifi and I go to UI and start the jobs. Is there a way that 1. Nifi process starts automatically when ever system reboots or restarts ? 2. Nifi jobs also start automatically without me running from UI ? can I use startup scripts for this ? I want to get ready with production mindset as i soon will be launching. I want to understand how it is handled in realtime production servers.. Please suggest.
... View more
Labels:
- Labels:
-
Apache NiFi
05-26-2023
08:40 AM
Thank You @MattWho @SAMSAL for your wonderful suggestions. I achieved solution using JolttransformJSON processor, where i used below jolt specification [{ "operation": "default", "spec": { "*": { "error": "${putdatabaserecord.error}" } } }] Thanks again for your thought provoking answers..
... View more
05-25-2023
08:36 AM
Hi All, I am using putdatabaserecord to store a value into DB. Now i have a flow based on failure link to (this processor), that this failed record will be stored in disc for troubleshooting. I can store the failed record in disc by using putfile processor. Now i wanted it more informative during debugging, as the attribute "putdatabaserecord.error" contains the error reason, i want to append the flowfile content with this error reason. I tried to acheive this using below flow.. putsql/putdbrecord -> failure-> updateattribute (this is to give the flowfile a realistic name)-> attributesToJson (destination flowfileattribute/content -> putfile but in attributestoJson , if the destination is kept as flowfileattribute, the flowfile dont get this attribute or if it is kept as flowfile content, original flowfile content is replaced... My requirement is to append this single attribute to the existing flowfile content. Any help or suggestion is greatly appreciated.
... View more
Labels:
- Labels:
-
Apache NiFi
05-22-2023
02:38 AM
2 Kudos
a bit research helped to find out the way from Nifi toolkit documentations $ nifi set-param -pcid abc -pn <name> -pv <value> -ps <true/false for senstivity> and it automatically refreshed/bought-in the updated values in UI.
... View more
05-22-2023
01:28 AM
I am using Nifi parameter context to avoid hardcoding multiple variables. I used toolkit to create parameter context and used params inside my flow. Now I want to update some values of this existing parameter context.. 1. to update some values or 2. to add new parameters. I could not find a way using toolkit in the documentation. I see an answer here https://community.cloudera.com/t5/Support-Questions/Update-parameter-contexts-dynamically-using-REST-API/m-p/347697 But i am looking for any solution based on toolkit or other easier approach, so that it can be easily handled while deployments to next environments or while updating it in production environments. Please help me with your approach or suggestions.
... View more
Labels:
- Labels:
-
Apache NiFi
-
NiFi Registry
05-22-2023
01:16 AM
Thanks @MattWho for the answer. Does it also has an alternative with toolkit ? or any other easier process as Nifi evolved recent days?
... View more
05-15-2023
11:25 AM
2 Kudos
Just wanted to update that I used flow file attributes to solve this. putslack's Webhook Text supports expression language!!!
... View more