NiFi 2.4.0 on Kubernetes (via Helm) - OIDC User Permissions Issues & Pod Errors After initContainer Setup

[#1]

I'm encountering persistent issues setting up NiFi 2.4.0 on Kubernetes (deployed via a Helm chart, celtic) with OIDC authentication (Keycloak) for an initial admin user. My goal is to have a specific OIDC user (e.g., 'nifi-admin') fully permissioned to manage flows and processors upon deployment.

When I first start up the pod, the users and authorizations file look like the following.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<authorizations>
    <policies>
        <policy identifier="f99bccd1-a30e-3e4a-98a2-dbc708edc67f" resource="/flow" action="R">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
        </policy>
        <policy identifier="b8775bd4-704a-34c6-987b-84f2daf7a515" resource="/restricted-components" action="W">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
        </policy>
        <policy identifier="627410be-1717-35b4-a06f-e9362b89e0b7" resource="/tenants" action="R">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
        </policy>
        <policy identifier="15e4e0bd-cb28-34fd-8587-f8d15162cba5" resource="/tenants" action="W">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
        </policy>
        <policy identifier="ff96062a-fa99-36dc-9942-0f6442ae7212" resource="/policies" action="R">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
        </policy>
        <policy identifier="ad99ea98-3af6-3561-ae27-5bf09e1d969d" resource="/policies" action="W">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
        </policy>
        <policy identifier="2e1015cb-0fed-3005-8e0d-722311f21a03" resource="/controller" action="R">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
            <user identifier="d0e7bf59-f2a1-3b70-a8bb-fb51bcdca184"/>
        </policy>
        <policy identifier="c6322e6c-4cc1-3bcc-91b3-2ed2111674cf" resource="/controller" action="W">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
        </policy>
        <policy identifier="287edf48-da72-359b-8f61-da5d4c45a270" resource="/proxy" action="W">
            <user identifier="d0e7bf59-f2a1-3b70-a8bb-fb51bcdca184"/>
        </policy>
    </policies>
</authorizations>

It appears I can't interact with the flows and process groups with these policies and permissions. Everything is grayed out However, when I delete the users and authorizations file and restart the statefulset, I am now able to interact with the necessary parts. The new authorizations file looks like the following

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<authorizations>
    <policies>
        <policy identifier="f99bccd1-a30e-3e4a-98a2-dbc708edc67f" resource="/flow" action="R">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
        </policy>
        <policy identifier="ad087445-b630-3030-a326-5e39fdf95337" resource="/data/process-groups/efc2e3d9-0196-1000-3993-94c802173dfb" action="R">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
            <user identifier="d0e7bf59-f2a1-3b70-a8bb-fb51bcdca184"/>
        </policy>
        <policy identifier="6d417de5-4472-3162-9d67-7aa6068e8b41" resource="/data/process-groups/efc2e3d9-0196-1000-3993-94c802173dfb" action="W">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
            <user identifier="d0e7bf59-f2a1-3b70-a8bb-fb51bcdca184"/>
        </policy>
        <policy identifier="5218517d-c8f8-39c6-98ff-66acaa37a590" resource="/process-groups/efc2e3d9-0196-1000-3993-94c802173dfb" action="R">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
        </policy>
        <policy identifier="206d7cf3-de24-35f7-ab5b-129a65139f23" resource="/process-groups/efc2e3d9-0196-1000-3993-94c802173dfb" action="W">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
        </policy>
        <policy identifier="b8775bd4-704a-34c6-987b-84f2daf7a515" resource="/restricted-components" action="W">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
        </policy>
        <policy identifier="627410be-1717-35b4-a06f-e9362b89e0b7" resource="/tenants" action="R">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
        </policy>
        <policy identifier="15e4e0bd-cb28-34fd-8587-f8d15162cba5" resource="/tenants" action="W">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
        </policy>
        <policy identifier="ff96062a-fa99-36dc-9942-0f6442ae7212" resource="/policies" action="R">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
        </policy>
        <policy identifier="ad99ea98-3af6-3561-ae27-5bf09e1d969d" resource="/policies" action="W">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
        </policy>
        <policy identifier="2e1015cb-0fed-3005-8e0d-722311f21a03" resource="/controller" action="R">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
            <user identifier="d0e7bf59-f2a1-3b70-a8bb-fb51bcdca184"/>
        </policy>
        <policy identifier="c6322e6c-4cc1-3bcc-91b3-2ed2111674cf" resource="/controller" action="W">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
        </policy>
        <policy identifier="287edf48-da72-359b-8f61-da5d4c45a270" resource="/proxy" action="W">
            <user identifier="d0e7bf59-f2a1-3b70-a8bb-fb51bcdca184"/>
        </policy>
    </policies>
</authorizations>

I tried rectifying this by adding an init container that will essentially populate the files with what they need at startup. The following authorizations file is hardcoded and added to the specific path within the pod.

<authorizations>
    <policies>
        <policy identifier="f99bccd1-a30e-3e4a-98a2-dbc708edc67f" resource="/flow" action="R">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
        </policy>
        <policy identifier="d70f87b2-11d4-30f9-9671-5473a18d857a" resource="/data/process-groups/root" action="R">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
            <user identifier="d0e7bf59-f2a1-3b70-a8bb-fb51bcdca184"/>
        </policy>
        <policy identifier="f5b1762c-6f99-3037-8726-fdbff8366549" resource="/data/process-groups/root" action="W">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
            <user identifier="d0e7bf59-f2a1-3b70-a8bb-fb51bcdca184"/>
        </policy>
        <policy identifier="a2e73784-9fa0-3934-85d1-334aff29bc59" resource="/process-groups/root" action="R">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
        </policy>
        <policy identifier="67277589-3a48-3d30-9996-a4dd2621dbf2" resource="/process-groups/root" action="W">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
        </policy>
        <policy identifier="b8775bd4-704a-34c6-987b-84f2daf7a515" resource="/restricted-components" action="W">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
        </policy>
        <policy identifier="627410be-1717-35b4-a06f-e9362b89e0b7" resource="/tenants" action="R">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
        </policy>
        <policy identifier="15e4e0bd-cb28-34fd-8587-f8d15162cba5" resource="/tenants" action="W">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
        </policy>
        <policy identifier="ff96062a-fa99-36dc-9942-0f6442ae7212" resource="/policies" action="R">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
        </policy>
        <policy identifier="ad99ea98-3af6-3561-ae27-5bf09e1d969d" resource="/policies" action="W">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
        </policy>
        <policy identifier="2e1015cb-0fed-3005-8e0d-722311f21a03" resource="/controller" action="R">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
            <user identifier="d0e7bf59-f2a1-3b70-a8bb-fb51bcdca184"/>
        </policy>
        <policy identifier="c6322e6c-4cc1-3bcc-91b3-2ed2111674cf" resource="/controller" action="W">
            <user identifier="0482ca97-7afc-3d5a-8a3f-21803898bde0"/>
        </policy>
        <policy identifier="287edf48-da72-359b-8f61-da5d4c45a270" resource="/proxy" action="W">
            <user identifier="d0e7bf59-f2a1-3b70-a8bb-fb51bcdca184"/>
        </policy>
    </policies>
</authorizations>

 Unfortunately, this is still not giving me the necessary permissions. Everything is grayed out. Why aren't the proper permissions populated when I start the statefulset? Why does it seem to fix this when I remove the files and restart it? 

And why doesn't my init container approach work? Idk if that is related to the "root" I am specifying after process groups.

Another thing maybe worth mentioning is my authorizers.xml having this access provider

  <accessPolicyProvider>
    <identifier>file-access-policy-provider</identifier>
    <class>org.apache.nifi.authorization.FileAccessPolicyProvider</class>
    <property name="User Group Provider">file-user-group-provider</property>
    <property name="Authorizations File">./auth-conf/authorizations.xml</property>
    <property name="Initial Admin Identity">nifi-admin</property>
    <property name="Node Group"/>
    <property name="Node Identity 0">CN=nifi-0.nifi-headless.nifi.svc.nifi.uds.dev, OU=NIFI</property>
  </accessPolicyProvider>

[#2 · Accepted solution]

@TyTheNiFiGuy 

On first startup of NiFi the flow.json.gz does not yet exist when the authorizers.xml is executed, so the root process group is not yet known to the file-access-policy-provider in order to create the initial policies for that root process group generated ID.  This is why you see all the icons greyed out above the NiFi canvas.

When you tried to construct the authorizations.xml manually you are using that root process group ID manually created in a different startup.  If the flow.json.gz does not exist during that startup, when created it will not have that same ID and thus greyed out icons again above the canvas in the NiFi UI.  So you can only use this constructed authorizations.xml if you are also using the initial generated flow.json.gz along with it that uses that ID for the root Process group.

This is really not a blocker.  Your initial Admin user can access NiFi and does have full  permissions on "/policies" which gives that user ability to set new policies, including for itself. Simply right clicking within the generated canvas (root Process group) and click "manage access policies" will allow hat user to set authorizations (view the component, modify the component, etc) on the root process group needed to make icons all appear.

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

[#3]

Thanks. I don't fully understand the differences between the policies you can modify by right clicking on the canvas and selecting "Manage Access Policies" and the policies you can modify by selecting the dropdown on the top right and selecting "Policies". In either case, the right click and "Manage Access Policies" approach worked.