Created on 10-11-2016 12:44 PM
Credits to @mgilman for contributing to this article:
Since the introduction of NiFi 1.0.0 administrators have a greater ability to manage policy through the addition of Ranger Integration and a more granular authorization model. This article provides a guide for those looking to define and manage NiFi policies in Ranger. To learn more on configuring NiFi to use Ranger via Ambari please review the parent article HDF 2.0 - Integrating Secured NiFi with Secured Ranger for Authorization Management.
Behind the scenes NiFi uses a REST based API for all user interaction; therefore resource based policies are used in Ranger to define users' level of permissions when executing calls against these REST endpoints via NiFi's UI . This allows administrators to define policies by selecting a NiFi resource/endpoint, choosing whether users have Read or Write (Modify) permissions to that resource, and selecting users who will be granted the configured permission. For example the image below shows a policy in Ranger where a user is granted the ability to View Flows in NiFi’s interface. This was configured by selecting /flow as the authorized resource and granting the selected user the Read permission to that resource.
Example of Global Level Policy Definition with Kerberos User Principal
Policies can be created that will apply authorizations to features at a global level or on a specific component level in NiFi. The following describes the policies that can be defined in Ranger using a combination of the indicated NiFi Resource and Permission (Read or Write).
|View the user Interface||Allows users to view the user interface||/flow||Read|
|Access the controller||Allows users to view/modify the controller, including Reporting Tasks, Controller Services, and clustering endpoints. Explicit access to reporting tasks and controller services can be overridden||
Read (for View)
Write (for Modify)
|Query Provenance||Allows users to submit a Provenance Search and request Event Lineage. Access to actual provenance events or lineage will be based off the data policies of the components that generated the events. This simply allows the user to submit the query.||
|Access Users/User Groups||Allows users to view/modify users and user groups||
Read (View User/Groups)
Write (Modify Users/Groups)
|Retrieve Site-To-Site Details||This policy should be granted to other NiFi systems (or Site-To-Site clients) in order to retrieve the listing of available ports (and peers when using HTTP as the transport protocol). Explicit access to individual ports is still required to see and initiate Site-To-Site data transfer.||/site-to-site||
Read (Allow retrieval of data)
|View System Diagnostics||This policy should be granted in order to retrieve system diagnostic details including JVM memory usage, garbage collection, system load, and disk usage.||
|Proxy User Requests||This policy should be granted to any proxy sitting in front of NiFi or any node in the cluster that will be issuing requests on a user's behalf.||
|Write (granted on Node Users defined in Ranger)|
|Access Counters||This policy should be granted to users to retrieve and reset counters. This policy is separated from each individual component has the counters can also be rolled up according to type.||
Read (Read counter information)
Write (Reset Counters)
Note: Setting authorizations on the /policy resource is not applicable when using Ranger since NiFi’s policy UI is disabled when Ranger Authorization enabled.
Component Level policies can be set in Ranger on individual components on the flow within a process group or on the entire process group (with the root process group being the top level for all flows). Most components types (except for connections) can have a policy applied directly to it. For example the image below demonstrates a policy defined for a specific processor instance (noted by the unique identifier included in the resource url) which grants Read and Modify permissions to the selected user.
Example of Component Level Policy for Kerberos User Principal
If no policy is available on the specific component then it will look to the parent process group for policy information.
Below are the available resources for components where a specific policy can be applied to an instance of that component. Detailed information on component descriptions can be found in NiFi Documentation.
|Component Type||Resource (Rest API)||Description (from NiFi Documentation)|
|Controller Service||/controller-services||Extension Point that provides centralized access data/configuration information to other components in a flow|
|Funnel||/funnels||Combine data from several connections into one connection|
|Input Port||/input-ports||Used to receive data from other data flow components|
|Label||/labels||Documentation for flow|
|Output Port||/output-ports||Used to send data to other data flow components|
|Processor||/processor||NiFi component that pulls data from or publishes to external sources, or route, transforms or extracts information from flow files.|
|Process Group||/process-groups||An abstract context used to group multiple components (processors) to create a sub flow. Paired with input and output ports process groups and be used to simplify complex flows and use logical flows|
|Reporting Task||/reporting-tasks||Runs in the background and provides reporting data on NiFi instance|
|Template||/templates||Represents a predefined dataflow available for reuse within NiFi. Can be imported and exported|
The following table describes the types of policies that can be applied to the previously mentioned components. Note: UUID is the unique identifier of an individual component within the flow.
|Read or Update Component||This policy should be granted to users for retrieving component configuration details and modifying the component.||
|View Component Data or Allow Emptying of Queues and Replaying||This policy should be granted to users for retrieving or modifying data from a component. Retrieving data will encompass listing of downstream queues and provenance events. Modifying data will encompass emptying of downstream queues and replay of provenance events. Additionally, data specific endpoints will require every link in the request chain is authorized with this policy. Since they will be traversing each link, we need to ensure that each proxy is authorized to have the data.||
|Write Receive Data, Write Send Data||These policies should be granted to other NiFi instances and Site-To-Site clients that will be sending/receiving data from the specified port. Once a client has been added to a port specific Site-To-Site policy, that client will be able to retrieve details about this post and initiate a data transfer. Additionally, data specific endpoints will require every link in the request chain is authorized with this policy. Since the will be traversing each link, we need to ensure that each proxy is authorized to have the data.||
For more information on Authorization configuration with Ranger and NiFi please review