About kellerj

kellerj · ‎11-02-2022

Disappointing, but what I was starting to suspect after going through CM's console. Is there general guidance on including those types of files to the NiFi nodes?

kellerj · ‎10-31-2022

Good morning, I was hoping someone would know if there is a supported method for installing additional libraries onto the NiFi nodes from Cloudera Manager. For example, I need to include the Oracle JDBC driver JAR on each server so it can be referenced in the controller services. I know I can load files onto the servers via server deployment tools, but I was hoping there was a way to deploy additional library files to the cluster via the manager tool as part of the configuration. (Especially as these files may need to be updated over time, and I would rather not run the scripts which impact the server at a lower level.) Thanks.

kellerj · ‎10-11-2022

Thanks for the info. I'm not actually sure what to do with the KeyTabCredentialsService as we are not using Kerberos on our NiFi nodes at present. So, without any configuration in /etc/krb5.conf or setting up a KDC server, is there a way to use that controller service?

kellerj · ‎09-30-2022

I am trying to utilize the KafkaRecordSink_2_6 as a destination for a couple of my reporting tasks. However, I do not see a way to pass in the Security Mechanism or the credentials as I do on Kafka processors. I'm guessing this has a lot to do with my unfamiliarity with Kafka authentication. I do see that there is support for a Kerberos configuration, however we are not using that mechanism for our other connections. I'm hoping that there are some "magic" / undocumented properties on this controller which will allow it to be used in our environment. Any recommendations? I'd really like to get some of the system events and bulletins posted into a topic so I can process them. (Without the overhead of saving to the database...then using Kafka Connect or another processor to pull them back out into a queue...it's doable, but seems clunky.)

kellerj · ‎08-10-2022

Thank you! I looked, but not in the right place. I skimmed over the bucket API, but didn't look in detail as I assumed (incorrectly) that it was not relevant to updating a flow.

kellerj · ‎08-10-2022

Is there any way to update the flow description in the NiFi Registry after the initial creation of the flow that does not require restarting version control? I checked the REST API endpoints as well and didn't see anything. I would like to be able to clean up descriptions (or even just correct a typo) without going through a cumbersome export versions, import flow, import versions, delete and re-import PGs process. Thanks.

kellerj · ‎08-01-2022

Thank you so much! I was looking around for the way to add arbitrary properties to the nifi-registry.properties and never hit on the right wording to find that one. Yep, figured I could always add the file manually if I could just do what you showed how to.

kellerj · ‎07-29-2022

Does anyone know the process for setting up URL aliases in NiFi registry when using Cloudera Manager to manage the instance? The property: nifi.registry.registry.alias.configuration.file is not available in the UI. And while there are options to ignore properties, there is nothing I can find to add arbitrary properties to the configuration. In addition to the above, you also need to add a registry-aliases.xml file to the server and I see no way to accomplish that. Ideally this would all be auto-configured as the registry connection URL information is required in this latter file. And the path to these configuration files is managed by Cloudera Manager. For reference, this is the feature of NiFi Registry I am attempting to make use of: https://nifi.apache.org/docs/nifi-registry-docs/html/administration-guide.html#url-aliasing Thanks for any advice anyone can give.

kellerj · ‎04-27-2022

I did end up finding a hacky way to do this. But something using APIs or the registry would be preferred. It was a multi-step process: Bring up a new NiFi server. Import the process group from the JSON file into the canvas. Stop the server and copy out flow.xml.gz Rebuild the server to get an empty flow.xml.gz, then stop it again. Unzip and edit both files. Copy the contents of the only <processorGroup> tag to replace the contents of the <rootGroup> Update the coordinates in the new root group to 0.0/0.0. Copy the <parameterContexts> over <parameterContexts/> in the new file. re-zip the updated file and start up the server. Re-enable broken controller services. Not pretty, but it worked to restore a server from a JSON export copy I had made with minimal fixes in NiFi itself. If there is no easier way, I hope this helps someone else.

kellerj · ‎04-27-2022

If I took an export of the main flow (NiFi Flow) from a server via the Export to JSON option on the right-click menu, how can I import that into a new server as the top-level flow? I know I can create new process groups with the export. But I want to restore the controller services and parameter contexts. I know I could import into a process group, and then move things out again. But, that would also require re-creating all the controller services back at the top level...which would create new UUIDs...and require updating 400-500 processors manually to map them to the new controller services. I've seen various posts about using NiFi registry and the toolkit. But even when I look at that, I don't see how that works with the top-level group. If there is some way to handle this that will require a minimum of reconfiguration on services, processors or process groups after the import, it would be appreciated.

Online	Offline
Last Visited	‎08-30-2024 11:55 AM

Member Since	‎04-27-2022 10:25 AM
Last Visited	‎08-30-2024 11:55 AM
Posts	20
Kudos received	3

Cloudera Community

Re: Installing Support Libraries on NiFi Nodes

Installing Support Libraries on NiFi Nodes

Re: How to pass Credentials to the KafkaRecordSink

How to pass Credentials to the KafkaRecordSink

Re: Update Flow Description in NiFi Registry

Update Flow Description in NiFi Registry

Re: Configuring NiFi Registry URL Aliases

Configuring NiFi Registry URL Aliases

Re: Importing JSON Export into top-level NiFi Flow

Importing JSON Export into top-level NiFi Flow