Member since
03-10-2017
170
Posts
79
Kudos Received
32
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1012 | 08-12-2024 08:42 AM | |
1841 | 05-30-2024 04:11 AM | |
2335 | 05-29-2024 06:58 AM | |
1534 | 05-16-2024 05:05 AM | |
1224 | 04-23-2024 01:46 AM |
08-11-2025
09:05 PM
This article provides detailed steps about Encrypting and Decrypting files using the Pretty Good Privacy (PGP) algorithm in NiFi
Steps involve :
Setting up GPG private key and Public key
Creating NiFi flow
Setting up GPG private key and Public key
Set up the GPG home directory export GNUPGHOME=/tmp/gnupg
Generate a private and public key with a user ID gpg --full-generate-key
gpg (GnuPG) 2.2.20; Copyright (C) 2020 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Please select what kind of key you want:
(1) RSA and RSA (default)
(2) DSA and Elgamal
(3) DSA (sign only)
(4) RSA (sign only)
(14) Existing key from card
Your selection? 1
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 2048
Requested keysize is 2048 bits
Please specify how long the key should be valid.
0 = key does not expire
<n> = key expires in n days
<n>w = key expires in n weeks
<n>m = key expires in n months
<n>y = key expires in n years
Key is valid for? (0) 0
Key does not expire at all
Is this correct? (y/N) y
GnuPG needs to construct a user ID to identify your key.
Real name: TestXYZ
Email address: test@example.com
Comment:
You selected this USER-ID:
"TestXYZ <test@example.com>"
Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? o
There will be a prompt to enter a passphrase, which will be used later in the configuration
There will be 4 files created under /tmp/gnupg
Create a public and private keyring file in ASCII format
For the public keyring file, run the following command gpg --export --armor --output testexample.gpg.pub test@example.com
For the Private keyring file, run the following command gpg --export-secret-keys --armor --output testexample.gpg.asc test@example.com
We would have two new files created under /tmp/gnupg
Creating NiFi flow
At this point, all set to create flow design using EncryptContentPGP and DecryptContentPGP, which use controller services StandardPGPPublicKeyService and StandardPGPPrivateKeyService
EncryptContentPGP configured with StandardPGPPublicKeyService
DecryptContentPGP with StandardPGPPrivateKeyService Note: In StandardPGPPrivateKeyService passphrase needs to be entered, which was set while running the gpg --full-generate-key command
Snapshot from working flow tested with .csv and .xlsx files
... View more
Labels:
08-12-2024
08:42 AM
The details you see in UI other the actual content is loaded in memory and kept stored under flow file repository and content is in content repostoriru If I just take the example of flow file details, it has content that is located in Section 1 means Directory 1 under the content repo with an identifier number. So the question is after every restart does your content repository location persist?
... View more
08-12-2024
08:29 AM
From the logs snipped, it appears that the NiFi node does not have all the permissions, to get more clarity it would be nice if you could add the error you see in NiFi UI related to permissions. Thank you
... View more
08-12-2024
08:17 AM
1 Kudo
If I understand correctly, Service NiFi works fine but from the Cloudera Manager things do not look promising this has to do with how CM monitors the NiFI service I would start to at the logs directory under the NiFi processor directory on each host where niFi is running and look for file connectionMetricOutput.log NNNNNNNN-nifi-NIFI_NODE/logs/connectionMetricOutput.log On each Node where NiFi service is running, CM agent invokes nifi cli command get-nodes to get the health status of each node. bin/cli.sh nifi get-nodes If you find get-nodes command is not completing, you may want to further file a support case with us to look in detail. I hope this helps. Thank you
... View more
06-03-2024
04:13 AM
2 Kudos
NiFi Registry is a single-point service, Not meant to sync flow versioning with multiple NiFi Registry. Not sure how this is set up at your end you can provide an overview of your Architecture between NiFi and NiFi Registry and Git so I can understand further. Thank you
... View more
06-03-2024
02:39 AM
@Naveen_Sagar, can you provide a complete error stack to understand the cause behind the connection reset? along with processor property details
... View more
05-30-2024
04:11 AM
2 Kudos
This is not possible since Scheduling Strategy can not be set dynamically or via any attribute (flow or variable ) Thank You
... View more
05-29-2024
06:58 AM
1 Kudo
The following command I have tested and found working, make sure you are passing the right token as sometimes you may end up using extra information in the token curl -X PUT -H "Content-Type: application/json" -H "Authorization: Bearer <TOKEN>" -d '{"state": "RUNNING", "id": "processor-id", "revision": {"version": versionNumber, "clientId": "clidntIDstring"}}' -k --negotiate "https://hostname:8443/nifi-api/processors/<ID>/run-status" Also please make sure the user has permission to WRITE permission on components
... View more
05-28-2024
02:02 AM
3 Kudos
Log rotation settings have No/Driect impact on NiFi data flow processing performance. It is just that there should be enough space available on the file system to store the log files. Thank you
... View more
05-16-2024
05:05 AM
2 Kudos
You might be looking at the most recent version of the document. Include all violation is added from Apache NiFi 1.25 version https://issues.apache.org/jira/browse/NIFI-12593. I hope this helps. Thank you
... View more