Member since
07-30-2019
3387
Posts
1617
Kudos Received
999
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 139 | 11-05-2025 11:01 AM | |
| 382 | 10-20-2025 06:29 AM | |
| 522 | 10-10-2025 08:03 AM | |
| 360 | 10-08-2025 10:52 AM | |
| 402 | 10-08-2025 10:36 AM |
03-07-2017
05:47 PM
@Anishkumar Valsalam No harm at all having one node service both roles. It is very common to see that. Matt
... View more
03-07-2017
05:45 PM
@vikash kumar You current flow is as follows? getSFTP ---->(success)----> putSFTP ---->(success)----> ???
After you put copies of your FlowFiles content to your target SFTP server using putSFTP, do you have any need for the content any longer? If not you could simply use the ReplaceText processor configured as follows: "Always replace" will replace the entire content with the configured "replacement value". If blank you will end up with a 0 byte file for every FlowFile that was successfully written to your putSFTP server. Is this what you are looking for? Thanks,
Matt
... View more
03-07-2017
05:38 PM
1 Kudo
@Sunile Manjee UUIDs are created when a template is add to the canvas. This allows users to instantiate the same template multiple times within a single NiFi. It also insures not conflict of UUID with other existing components already instantiated. There is no way for users to set the UUIDs of any component manually. The only way to maintain UUIDs between NiFi instances is to move the entire flow.xml.gz file from one NIFi to the next rather then using templates. Thanks, Matt
... View more
03-07-2017
05:34 PM
1 Kudo
@Sunile Manjee Even if you were to use NiFi's file based authorizer instead of Ranger the same limitation exists with maintaining authorizations when moving templates from one NiFi environment to another. Templates were never intended or designed to be the answer to the SDLC. Although they represent the closest thing to it for now. Templates are nothing more the a snippet of a NiFi components that can be reused within the same NiFi or downloaded and shared with user of other NiFi instances. They cannot be hardcoded to use specific component uuids nor would be want to because that what hinder there reusability within the same NiFi instance. We also can't include any authorizations with a template since there is no way of knowing that other NiFi instances in which the template is loaded will contain the same set of users. Nor can we set authorizations based on PG names. What if another PG is created with that same name in another process group? What is a user happens to use a PG name that has policies associated to it? The results could present a security issue. There is on going work towards a better SDLC model with NIFi. That being said, the default behavior when adding a template to a graph is that all components inherit the policies from the parent process group. So if at the root level you create several process groups with a specific set of authorizations for each, instantiating your templates in a given process group will establish a controlled set of authorizations. Not the ideal solution, but helps some until future work is done to make SDLC better. Thanks, Matt
... View more
03-07-2017
04:35 PM
Role change does not cause data loss. Every node in a cluster runs the same dataflow and works on its own set of FlowFiles. Processor components added to the canvas and configured to run on primary node only will run on the currently elected primary node. So when primary node assignment changes the primary node only configured processors are stopped on the old primary node and started on the new. So be mindful of what processors are set to run on primary node only. While this will not result in data loss, it could result in data being stalled in a dataflow.
... View more
03-07-2017
04:31 PM
2 Kudos
@Anishkumar Valsalam All Nodes will register with ZK to become the cluster coordinator when NiFi cluster is first started. Once all nodes have checked in to ZK or 5 minutes has passed a random node from those who connected will be picked as the cluster coordinator. ZK will also register one node as your primary node. Once a cluster coordinator has been elected, all nodes will start sending heartbeats directly to that node. The cluster coordinator assumes the role of disconnecting nodes from the cluster who do not send heartbeats and reconnecting nodes who heartbeat later after previously being disconnected. Nodes in a cluster also heartbeat with ZK. If either the primary node or cluster coordinator fails to heartbeat, another connected node(s) at random is elected to assume those roles. There is no ability for users to manually assign either of these roles to a specific node in a cluster. Thanks, Matt
... View more
03-07-2017
04:02 PM
@Anishkumar Valsalam Not sure exactly what you are looking for. - Verify you can access the NIFi UI from any one of your nodes URLs. - Verify the UI shows 3/3 connected nodes. - Verify you can add components to the canvas (processors and connections) - Verify you can add controller services and reporting tasks. - Go into "cluster" UI and manually disconnect the "cluster coordinator" and/or " primary node". Make sure that a new node(s) is assigned these roles. - From "Cluster" UI make sure you can reconnect a disconnected node. Since your NiFi cluster is running unsecured there isn't much more to validate from an administration standpoint. - You don't have node to node mutual authentication to worry about - You don't have user authentication or authorization to worry about. Dataflows are built in a cluster the same way they are built in a standalone NiFi installation. Just keep in mind that every node in your cluster runs its own copy of the dataflow and works on its own set of FlowFiles. Thanks, Matt
... View more
03-06-2017
10:07 PM
@Ayaskant Das Based on the output from the nifi-user.log, the following user successfully authenticated to NiFi but was not authorized view the NiFi UI. Did you configure this user as your "Initial Admin Identity"? The "Initial Admin Identity" is configured in the authorizers.xml file and is used to initially create both the users.xml and authorizations.xml files. It is case sensitive and white spaces do count. If you NiFi had previously been started with an incorrect value for "Initial Admin Identity" you will need to correct the authorizers.xml with the correct value, delete both users.xml and authorizations.xml files, and restart NiFi so that these files are created again. Thanks,
Matt
... View more
03-06-2017
09:12 PM
2 Kudos
@Ayaskant Das @Joe Petro NiFi must be secured to run over HTTPS using a server certificates (loaded into a keystore) and a truststore before you can enable some form of user authentication.
You can create your own keystore and truststore using the below procedure: https://community.hortonworks.com/articles/17293/how-to-create-user-generated-keys-for-securing-nif.html Note: If you are going to be using LDAP or Kerberos for user authentication, you can skip the last part of the above procedure about generating a user SSL certificate to load in your browser. Thanks, Matt
... View more
03-06-2017
09:02 PM
1 Kudo
@Eric Lloyd Start troubleshooting your flow by stepping a FlowFile through one processor at a time. Try right clicking on the connections and select "list queue". Clicking the "details" icon to the far left of a FlowFile in the queue list to get details of that FlowFile in that connection.
Does the filename look correct? Thanks, Matt
... View more