Member since
04-11-2023
9
Posts
8
Kudos Received
0
Solutions
11-01-2024
01:16 PM
1 Kudo
Hello, Planning to deploy Nifi cluster between two regions on AWS. Anyone currently have this in place that can provide some best practices guideline? Thanks!
... View more
Labels:
- Labels:
-
Apache NiFi
10-21-2024
05:52 AM
@jame1997 You can version control Process Groups (PG) into NiFi-Registry. If a process group were to get deleted, you could reload the last version stored in NiFi-Registry. With version controlled PGs, anytime a user makes a change to the PG, the PG reports local change exists and allows a quick option to commit new version. Not only does this version control feature allow you to restore last good stored version from NiFi-Registry, it makes it easy to back out changes to an older stored version. This would require you to version control the individual PG and not the top level PG to create a NiFi-Registry catalog of all your PGs to facilitate easy rollback, restore, and reuse capabilities. If you are talking about a scenario where you accidentally deleted a PG and noticed right away, you can simply swap in the newest archived flow.json.gz and restart your NiFi to restore. Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
03-29-2024
10:23 AM
1 Kudo
@jame1997 Not much to look at from a NiFi side. NiFi is writing to the network successfully and there is some loss then happening between NiFi and your syslog server. Resource usage affecting your NiFi would only slow down processing but not result in dataloss within NiFi. So the PutSyslog would successfully write all bytes to the network before passing the FlowFile to the "success" relationship. Using TCP of course would allow NiFi to confirm successful delivery thus allowing NiFi to appropriately retry, or route to either failure or success relationships. You could look at the data rate NiFi is writing from the putSyslog by looking at the stats on the processor. Then maybe you could experiment with: 1. netstat - nsu to check for UDP packet loss. 2. using a network monitoring tool Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
04-27-2023
07:51 AM
Hello @MattWho @SAMSAL @steven-matison @DigitalPlumber @cotopaul , after confirming the access I'm able to verified I can access the bucket from aws cli on the same system Nifi is running on and download files. The problem I'm having now is that the ListS3 pickup the list of the files from the bucket but the FetchS3 doesn't do anything. When I enabled debugging on the FetchS3 I receiving the following error message. FetchS3 Error message FetchS3 Configuration ListS3 Configuration Any suggestion to what is causing the issue?
... View more