Community Articles

Find and share helpful community-sourced technical articles.
avatar
Expert Contributor

Credits to @mgilman for contributing to this article:

Since the introduction of NiFi 1.0.0 administrators have a greater ability to manage policy through the addition of Ranger Integration and a more granular authorization model. This article provides a guide for those looking to define and manage NiFi policies in Ranger. To learn more on configuring NiFi to use Ranger via Ambari please review the parent article HDF 2.0 - Integrating Secured NiFi with Secured Ranger for Authorization Management.

Behind the scenes NiFi uses a REST based API for all user interaction; therefore resource based policies are used in Ranger to define users' level of permissions when executing calls against these REST endpoints via NiFi's UI . This allows administrators to define policies by selecting a NiFi resource/endpoint, choosing whether users have Read or Write (Modify) permissions to that resource, and selecting users who will be granted the configured permission. For example the image below shows a policy in Ranger where a user is granted the ability to View Flows in NiFi’s interface. This was configured by selecting /flow as the authorized resource and granting the selected user the Read permission to that resource.

Example of Global Level Policy Definition with Kerberos User Principal

Policies can be created that will apply authorizations to features at a global level or on a specific component level in NiFi. The following describes the policies that can be defined in Ranger using a combination of the indicated NiFi Resource and Permission (Read or Write).

Global Policies:

Policy Privilege NiFi Resource Permission(s)
View the user Interface Allows users to view the user interface /flow Read
Access the controller Allows users to view/modify the controller, including Reporting Tasks, Controller Services, and clustering endpoints. Explicit access to reporting tasks and controller services can be overridden

/controller

Read (for View)

Write (for Modify)

Query Provenance Allows users to submit a Provenance Search and request Event Lineage. Access to actual provenance events or lineage will be based off the data policies of the components that generated the events. This simply allows the user to submit the query.

/provenance

Read
Access Users/User Groups Allows users to view/modify users and user groups

/tenants

Read (View User/Groups)

Write (Modify Users/Groups)

Retrieve Site-To-Site Details This policy should be granted to other NiFi systems (or Site-To-Site clients) in order to retrieve the listing of available ports (and peers when using HTTP as the transport protocol). Explicit access to individual ports is still required to see and initiate Site-To-Site data transfer. /site-to-site

Read (Allow retrieval of data)

View System Diagnostics This policy should be granted in order to retrieve system diagnostic details including JVM memory usage, garbage collection, system load, and disk usage.

/system

Read
Proxy User Requests This policy should be granted to any proxy sitting in front of NiFi or any node in the cluster that will be issuing requests on a user's behalf.

/proxy

Write (granted on Node Users defined in Ranger)
Access Counters This policy should be granted to users to retrieve and reset counters. This policy is separated from each individual component has the counters can also be rolled up according to type.

/counters

Read (Read counter information)

Write (Reset Counters)

Note: Setting authorizations on the /policy resource is not applicable when using Ranger since NiFi’s policy UI is disabled when Ranger Authorization enabled.

Component Policies

Component Level policies can be set in Ranger on individual components on the flow within a process group or on the entire process group (with the root process group being the top level for all flows). Most components types (except for connections) can have a policy applied directly to it. For example the image below demonstrates a policy defined for a specific processor instance (noted by the unique identifier included in the resource url) which grants Read and Modify permissions to the selected user.

Example of Component Level Policy for Kerberos User Principal

If no policy is available on the specific component then it will look to the parent process group for policy information.

Below are the available resources for components where a specific policy can be applied to an instance of that component. Detailed information on component descriptions can be found in NiFi Documentation.

Component Type Resource (Rest API) Description (from NiFi Documentation)
Controller Service /controller-services Extension Point that provides centralized access data/configuration information to other components in a flow
Funnel /funnels Combine data from several connections into one connection
Input Port /input-ports Used to receive data from other data flow components
Label /labels Documentation for flow
Output Port /output-ports Used to send data to other data flow components
Processor /processor NiFi component that pulls data from or publishes to external sources, or route, transforms or extracts information from flow files.
Process Group /process-groups An abstract context used to group multiple components (processors) to create a sub flow. Paired with input and output ports process groups and be used to simplify complex flows and use logical flows
Reporting Task /reporting-tasks Runs in the background and provides reporting data on NiFi instance
Template /templates Represents a predefined dataflow available for reuse within NiFi. Can be imported and exported

The following table describes the types of policies that can be applied to the previously mentioned components. Note: UUID is the unique identifier of an individual component within the flow.

Policy Description REST API
Read or Update Component This policy should be granted to users for retrieving component configuration details and modifying the component. Read/Write on:

/{resource}/{uuid}

e.g.

/processor/{uuid}

View Component Data or Allow Emptying of Queues and Replaying This policy should be granted to users for retrieving or modifying data from a component. Retrieving data will encompass listing of downstream queues and provenance events. Modifying data will encompass emptying of downstream queues and replay of provenance events. Additionally, data specific endpoints will require every link in the request chain is authorized with this policy. Since they will be traversing each link, we need to ensure that each proxy is authorized to have the data. Read/Write on:

/data/{resource}/{uuid}

Write Receive Data, Write Send Data These policies should be granted to other NiFi instances and Site-To-Site clients that will be sending/receiving data from the specified port. Once a client has been added to a port specific Site-To-Site policy, that client will be able to retrieve details about this post and initiate a data transfer. Additionally, data specific endpoints will require every link in the request chain is authorized with this policy. Since the will be traversing each link, we need to ensure that each proxy is authorized to have the data.

Write on:

/data-transfer/input-ports/{uuid}

/data-transfer/output-ports/{uuid}

For more information on Authorization configuration with Ranger and NiFi please review

http://bryanbende.com/development/2016/08/22/apache-nifi-1.0.0-using-the-apache-ranger-authorizer

https://community.hortonworks.com/articles/57980/hdf-20-apache-nifi-integration-with-apache-ambarir....

9,318 Views
Comments
avatar
New Contributor

Resource for processor is not /processor but is instead /processors. Policies for reading and writing processors is /processors/{uuid} and /data/processors/{uuid}