Member since
07-30-2019
3387
Posts
1617
Kudos Received
999
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 206 | 11-05-2025 11:01 AM | |
| 410 | 10-20-2025 06:29 AM | |
| 550 | 10-10-2025 08:03 AM | |
| 375 | 10-08-2025 10:52 AM | |
| 412 | 10-08-2025 10:36 AM |
03-29-2017
12:57 PM
@Bram Klinkenberg The "Roles" noted above are only valid for us in the older Apache NiFi 0.x baseline. They were part of the authorized-users.xml file used in that baseline. The Apache NiFi 1.x baseline added support for multi-tenancy and a granular access control via access policies. It is an entirely new authorization method and uses different files. There is no notion of Roles in NiFI 1.x. The authorizers.xml file allows you to specify a legacy authorized-usesr.xml file in place of configuring an "Initial Admin Identity" simply to make it easy for user of NiFi 0.x to port their existing users over to NiFi 1.x. Matt
... View more
03-29-2017
12:41 PM
1 Kudo
@Bram Klinkenberg The users.xml and authorizations.xml files are generated for you the first time NiFi is started after being secured. Initially they are populated using the configuration from the authorizers.xml file. In that file you specified an "Initial Admin Identity" (assuming you used CN=admin). As a result a user (CN=admin) was added to the users.xml file and the relevant "admin" related access policies were assigned to that user in the authorizations.xml file. At this point your user (CN=admin) should be able to access the NiFi UI. The admin will use the NIFi UI to add additional users and authorize them for various access policies: Users are managed and Global Policies are applied as follows: Adding "Users" within NiFi has nothing to do with user authentication. The users you add here are for authorization to NiFi resources only. User Authentication must occur first and can be accomplished using User issued certs load in browser, Kerberos, or LDAP. Global access policies include the following: Component (Processors, process groups, and other things on canvas) level access policies are assigned to users as follows: Component level access policies include: Some Component level access policies are on available to specific components. If the currently selected component does not support the policy it will be greyed out in the list. More detail on teh various access policies can be found in teh admin guide: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#config-users-access-policies Thank you, Matt
... View more
03-28-2017
06:43 PM
@Emmanouil Petsanis Was there some error or condition that occurred prior to this issue (Out of disk space issues, repo corruption, etc...)? Do you run into the same issue if you switch to a newer release version?
The latest HDF release is HDF 2.1.2 http://docs.hortonworks.com/HDPDocuments/HDF2/HDF-2.1.2/index.html Thanks, Matt
... View more
03-28-2017
06:33 PM
@Bram Klinkenberg Glad to hear it is resolved. If this answer provided what you needed to resolve your issue, please accept the answer. Thank you,
Matt
... View more
03-28-2017
04:23 PM
@Anishkumar Valsalam No problem, everyone starts somewhere. Keep in mind that in a cluster every node is running the same dataflow. Node 1 has no idea what node 2 is doing and vice versa. So by default all 3 nodes in your cluster running the above dataflow and each may perform slightly different. When looking at the Ui of any one of the nodes in your NiFi cluster, the stats shown are the cumulative stats for all node sin your cluster. You should not assume that the numbers will always evenly divide between your connected nodes. When you make a request in the UI, that request must be replicated to all nodes in your cluster. So image a request to start or stop your GenerateFlowFile processor, that Processor may get started or stopped at not the exact same moment in time on each node. Considering the rate at which it produces your small test files, i would not expect the numbers to be the same. In addition, other very small difference can affect each node differently. What other process, service, OS level thing happen to run on one node and not another. which node is the cluster coordinator (does extra work), etc... While in the big picture the impact to over performance is negligible, with this simple flow you can see some differences. You can right click on a processor and select "Status History" to open a graph that will show various stats per node. The different stats are in a pull-down menu in the upper right corner of the Status History window. Blue line shows cumulative values (same as what is shown on processors). There is a different colored line for each node. Some suggestion for using this forum: 1. Try to keep one question per post, you tend to get better responses that way. This questions is related, so you are good there. 2. If you find an answer that got you the answer you were looking for, accept that answer so it benefits others using this forum. Thank you,
Matt
... View more
03-28-2017
03:27 PM
4 Kudos
@Nikhil Chaudhary The encryption of values in the NiFi variable registry is not there yet. It is a future goal in Apache NiFi. There is an existing Apache Jira that covers the adding of this capability: https://issues.apache.org/jira/browse/NIFI-2653 Thanks, Matt
... View more
03-28-2017
03:24 PM
@Anishkumar Valsalam
... View more
03-28-2017
02:22 PM
1 Kudo
@Anishkumar Valsalam Each Node in a NiFi cluster runs its own copy of the dataflow. Nodes do not automatically share data between them. I agree that the simple flow described by mliem above is great to test dataflow performance per node, it does not test comms between nodes. 1. Try setting up a dataflow that uses S2S as well to distribute data between your nodes. GenerateFlowFile ---> RPG inputport ---> CompressContent --> updateAttribute. The above will show how network between nodes behaves. While the dataflow is running, try removing and/or adding an additional node to your NiFi cluster. 2. If you are running securely, make sure your user can successfully login to the UI of every node in your cluster and successfully make a change on the canvas. 3. Manually disconnect the node that is marked as the cluster coordinator and/or primary node. Verify that a new Cluster coordinator and/or primary node is elected. 4. Configure a processor to run on "primary node only". Verify that the node assigned "Primary" node is the only node running that processor. Shutdown current node elected as primary node and make sure whichever node is elected as new primary node starts running that processor. Thanks, Matt
... View more
03-28-2017
12:37 PM
1 Kudo
@Emmanouil Petsanis My First thought would be to try increasing the number of available file handles for the user running you NiFi in the limits.conf file on your server. I would suggest setting a value of at least: * hard nofile 50000
* hard noproc 50000
* soft noproc 50000
* soft nofile 50000 After making the above change, see if you can start NiFi. Thanks, Matt
... View more
03-28-2017
12:28 PM
2 Kudos
@Chaitanya Manthine It appears Ambari is complaining that the following property was not set in your NiFi configs: Thanks,
Matt
... View more