Member since
07-30-2019
3466
Posts
1641
Kudos Received
1015
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 403 | 03-23-2026 05:44 AM | |
| 311 | 02-18-2026 09:59 AM | |
| 559 | 01-27-2026 12:46 PM | |
| 987 | 01-20-2026 05:42 AM | |
| 1299 | 01-13-2026 11:14 AM |
10-31-2019
06:57 AM
@Jette There is no need for the updateAttribute processor here, unless there are some missing details to this issue. The extractText processor already permanently adds any created attribute to the FlowFile it outputs. The dynamic property name used becomes the FlowFile attribute name. Matt
... View more
10-31-2019
06:40 AM
@Paul Yang Ranger is not offered in CFM, but will become part of the platform in the future. The only authorization offering within NiFi and NiFi-Registry within CFM is the local file based authorizer. NiFi user and group authorization is controlled via the NiFi UI instead of through an external authorization provider like Ranger. This same local file base authorization was also an option in HDF. https://docs.cloudera.com/cfm/1.0.1/securing-cfm/topics/cfm-enabling-tls.html You can configure NiFi to sync users and groups from LDAP also. You can then through the NiFi UI assign authorization policies to these sync'd user and groups. Thank you, Matt https://docs.cloudera.com/cfm/1.0.1/securing-cfm/topics/cfm-nifi-user-sync-ldap-properties.html
... View more
10-31-2019
05:57 AM
@Sergiete The nifi-app.log should tell you why the second node failed to start if NiFi got beyond the bootstrap process. If not, the nifi bootstrap.log will tell you why it failed to start. Matt
... View more
10-31-2019
05:54 AM
@Sergiete The sensitive.key file is created during NiFi startup and is removed once startup completes successfully. The fact that it still exists when you are trying to start NiFi, tells me that some previous startup attempt failed after sensitive.key was created, but before startup completed. You can safely remove this sensitive.key file from your NiFi nodes and start your NiFi service again. If NiFi fails to start and you see the sensitive.key was created and not removed again, look through your NiFi logs to see why it failed. It will be for a different reason since you had manually removed the sensitive.key before that startup. I have not seen this condition occur on any of my CFM installs yet, but have heard of this happening before. What I do not have is logs to determine what is happening in those cases. Matt
... View more
10-29-2019
01:34 PM
1 Kudo
@big-f The work to support bundles is part of NiFi-Registry 0.4.0 https://nifi.apache.org/docs/nifi-registry-docs/html/administration-guide.html#bundle-persistence-providers https://nifi.apache.org/docs/nifi-registry-docs/html/user-guide.html#manage-bundles https://issues.apache.org/jira/browse/NIFIREG-211
... View more
10-29-2019
11:04 AM
1 Kudo
@big-f NiFi-Registry storage of version controlled flows is only the beginning use case for NiFi-registry. 1. Additional features of NiFi-Registry in the pipe line include: - Storing of FlowFile parameters (NiFi parameters can be used by any property in any component in NiFi.) - Storing of NiFi nars (NiFi has grown to be a very large application. Rather then installing every single component that exists, users can set up local registries that store nars. Then these nars can be pulled and dynamically loaded during runtime in NiFi as needed by the flow which has been built.) 2. The only method of deploying flows other than through NiFi-Registry is the legacy templates. but even here you have an existing NiFi canvas (blank or with other existing flows). You then import and instantiate templates on to that existing canvas somewhere, Downsides to this legacy method include no version control and templates are stored in heap memory. Even if a template is not instantiated to the canvas, it is held in heap memory. More templates equals more heap usage. 3. I think this question is answered by point number two in answer 1. A NiFi FlowFile is what moves from processor to processor via a connection. Not sure what you are referring to when you say "flow file"? Are you talking about the flow.xml.gz?
... View more
10-28-2019
11:54 AM
@Dale1979 It would appear you have a multi-node NiFi cluster that is using the NiFi CA to issue the certificate for each NiFi node. Within a NiFi cluster, one node will be elected cluster coordinator. While you can access your NiFi cluster from any node in the cluster, any user requests (including displaying the UI) need to be replicated to the cluster coordinator. This replication request is performed by the node which you are authenticated in to on behalf of you as the user. Thus requiring that the node itself has been authorized to act as a proxy for your user. Within your authorizers.xml file you should find a file-user-group-provider which should have been configured a separate "Initial User Identity <num>" property for each of your NiFi nodes. The file-user-group-provider is used add the initial users in the users.xml file. The value would match the DN of the certificate found in each node's keystore.jks file. Using you output as an example, you should see a line like this: <property name="Initial User Identity 2">CN=server.domain:port, OU=NIFI</property> Above would be followed by additional similar lines with new number for each of your other NiFi nodes. Note: Having a port number in your CN name is unexpected. IMPORTANT: NiFi will only generate the users.xml and authorizations.xml files from the configuration in the authorizers.xml file the very first time. If you make edits to either the file-user-group-provider (build users.xml) or File-access-provider (builds authorizations.xml), those changes will not be reflected in the already existing users.xml and authorizations.xml files. You must delete these files so they are recreated. The expectation is that following successful securing of NiFi, all additional users and authorizers are added directly via the NiFi UI. Hope this helps, Matt
... View more
10-18-2019
06:13 AM
@maybegoodman NiFi is designed to operate on data which has been consumed by NiFi and placed in FlowFiles. That being said, it sounds like what you are trying to do here is zip all the files on the target SFTP server before consuming the zipped file by NiFi. Perhaps you could use the ExecuteStreamCommand processor to use ssh to manually zip the contents of the target SFTP server based on a configured scheduling cron. Then have your listSFTP processor only list .zip file instead. if the ExecuteStreamCommand is not adequate, there are several Script based processors perhaps you can use to write a custom script that handles step 1 fo connecting to SFTO server and zipping the files. Hope this helps, Matt
... View more
10-17-2019
06:54 AM
@ilyal What version of NiFi are you running? Is it Apache NiFi 1.8.x? There are numerous bugs with the new load balanced connections feature. The good number of these known bugs have been addressed between NiFi 1.9.0 and NiFi 1.9.2. https://issues.apache.org/jira/browse/NIFI-5745 https://issues.apache.org/jira/browse/NIFI-5919 https://issues.apache.org/jira/browse/NIFI-5663 https://issues.apache.org/jira/browse/NIFI-5771 https://issues.apache.org/jira/browse/NIFI-6017 There are still some additional bugs that are fixed in NiFi 1.10.0 https://issues.apache.org/jira/browse/NIFI-6353 https://issues.apache.org/jira/browse/NIFI-6760 https://issues.apache.org/jira/browse/NIFI-6517 https://issues.apache.org/jira/browse/NIFI-6736 https://issues.apache.org/jira/browse/NIFI-6285 https://issues.apache.org/jira/browse/NIFI-6759 I strongly recommend upgrading to Apache NiFi 1.10 as a first step upgrading (releasing soon). Hope this helps, Matt
... View more
10-15-2019
06:14 AM
1 Kudo
@Gcima009 It appears you are running CFM 1.0.0 which has a known issue with the sensitive props key. If a sensitive props key is not set in CFM NiFi configs, Cm randomly creates one for you. The bug here is that a different random sensitive props key is generated on each node. Since NiFi requires that all nodes in the flow have the exact same flow.xml.gz, the sensitive props key used to decrypt passwords within that flow.xml.gz must also be the same. This bug was addressed in CFM 1.0.1 Anytime you manually edit the files in /var/run/cloudera-scm-agent/process/<num>-nifi-NIFI_NODE and then restart that service via CM, it will detect a change and build new configs in a new folder. To get around this issue... 1. On one of the nodes cp the "config.zip" file out of the most recent <num>-nifi-NIFI_NODE to a tmpt directory (/tmp/config) 2. Extract the nifi-globals.xml file ( unzip config.zip staging/nifi-globals.xml ) 3. Extract the random NiFi sensitive props value ( grep -A 1 random.nifi.sensitive staging/nifi-globals.xml ) 4. Shutdown your NiFi cluster via CM. 5. Use the password obtained via step 3 from one node to manually set the password in CM NiFi configs 6. copy flow.xml.gz frorn same node where you obtained the random password to all other nodes in your cluster. 7. start your NiFi cluster. Now all nodes are running with same flow.xml.gz using same sensitive props key password. Thanks, Matt
... View more