question Re: How to create process-groups and apply custom policies to the processor via Nifi API in Support Questions

How to create process-groups and apply custom policies to the processor via Nifi API

scoutjohn — Sun, 16 Mar 2025 13:27:14 GMT

Hi All

I have a list of users that i have in an array.

I am trying to create a job in shell script that creates a process-group for each user and apply policy to that particular process group so that only that user and nifi's super user can access or operate in it

here is how the polices config json looks like

'[{"globalAccessPolicies":[{"resource":"flow","action":"read"},{"resource":"provenance","action":"read"}],"componentLevelAccessPolicies":[{"resource":"process-groups","action":"read"},{"resource":"process-groups","action":"write"},{"resource":"operation/process-groups","action":"write"},{"resource":"provenance-data/process-groups","action":"write"},{"resource":"provenance-data/process-groups","action":"read"},{"resource":"data/process-groups","action":"read"},{"resource":"data/process-groups","action":"write"},{"resource":"policies/process-groups","action":"read"},{"resource":"policies/process-groups","action":"write"},{"resource":"data-transfer/input-ports","action":"write"},{"resource":"data-transfer/output-ports","action":"write"}]}]'

For each user in the array, the job iterates through globalAccessPolicies and componentLevelAccessPolicies and assigns permission to the user.

before assigning the componentLevelAccessPolicies the job creates an empty process-group to in the nifi canvas

using the api

nifi_api_request "process-groups/root/process-groups" "POST" "Content-Type: application/json" "{\"revision\":{\"version\":0},\"component\":{\"name\":\"${tenant}\",\"position\":{\"x\":${x},\"y\":${y}},\"comments\":\"Processor group for ${tenant}\"}}"

this returns a json from which the id of the process group is fetched

then the Job uses the fetched ID, uses the componentLevelAccessPolicies array and tries to create a policy configuration using

for policy in $(echo "${componentLevelAccessPolicies}" | jq -c '.[]'); do resource=$(echo "${policy}" | jq -r '.resource') action=$(echo "${policy}" | jq -r '.action') policyConfig=$(echo '{ "revision": { "version": 0 }, "component": { "resource": "'${resource}'", "action": "'${action}'", "configurable": true, "users": [], "userGroups": [] } }' | jq .) policyConfig=$(echo "${policyConfig}" | jq ".component.userGroups += [{\"revision\":{\"version\":0},\"id\":\"${tenant_superadmin_id}\",\"permissions\":{\"canRead\":true,\"canWrite\":true},\"component\":{\"id\":\"${processor_group_id}\",\"identity\":\"${tenant_superadmin}\",\"configurable\":true}}]") policyConfig=$(echo "${policyConfig}" | jq ".component.userGroups += [{\"revision\":{\"version\":0},\"id\":\"${tenant_readonly_id}\",\"permissions\":{\"canRead\":true,\"canWrite\":false},\"component\":{\"id\":\"${processor_group_id}\",\"identity\":\"${tenant_readonly}\",\"configurable\":true}}]") policyConfig=$(echo "${policyConfig}" | jq ".component.userGroups += [{\"revision\":{\"version\":0},\"id\":\"${nifi_superadmin_id}\",\"permissions\":{\"canRead\":true,\"canWrite\":true},\"component\":{\"id\":\"${processor_group_id}\",\"identity\":\"${nifi_superadmin}\",\"configurable\":true}}]") policyConfig=$(echo "${policyConfig}" | jq ".component.userGroups += [{\"revision\":{\"version\":0},\"id\":\"${nifi_readonly_id}\",\"permissions\":{\"canRead\":true,\"canWrite\":false},\"component\":{\"id\":\"${processor_group_id}\",\"identity\":\"${nifi_readonly}\",\"configurable\":true}}]") echo "policy Group Config: ${policyConfig}" nifi_api_request "policies/" "POST" "Content-Type: application/json" "${policyConfig}" echo -e "status ${status}" echo -e "body ${body}" [[ ${status} -eq 201 ]] || exit 1 done

The required Ids are fetched beforehand from the user groups

Since job is trying to create policy to each processor group, i have set the http method as POST

nifi_api_request "policies/" "POST" "Content-Type: application/json" "${policyConfig}"

in the Loop it creates a json paylod like this

{ "revision": { "version": 0 }, "component": { "resource": "data-transfer/output-ports", "action": "write", "configurable": true, "users": [], "userGroups": [ { "revision": { "version": 0 }, "id": "9fd4eabd-5b6f-4a1d-8c5f-ca6049986d96", "permissions": { "canRead": true, "canWrite": true }, "component": { "id": "9ef8a5e3-0195-1000-ffff-ffffb4a7b545", "identity": "TenantID.john.nifi_superadmin", "configurable": true } }, { "revision": { "version": 0 }, "id": "aba0f614-d09a-42ee-9081-3328c86fcd6e", "permissions": { "canRead": true, "canWrite": false }, "component": { "id": "9ef8a5e3-0195-1000-ffff-ffffb4a7b545", "identity": "TenantID.john.nifi_readonly", "configurable": true } }, { "revision": { "version": 0 }, "id": "ebe9c88a-77d7-4070-bead-e24329b2e9c1", "permissions": { "canRead": true, "canWrite": true }, "component": { "id": "9ef8a5e3-0195-1000-ffff-ffffb4a7b545", "identity": "nifi_superadmin", "configurable": true } }, { "revision": { "version": 0 }, "id": "95aedc49-20f6-4dad-95f5-c7311a66c353", "permissions": { "canRead": true, "canWrite": false }, "component": { "id": "9ef8a5e3-0195-1000-ffff-ffffb4a7b545", "identity": "nifi_readonly", "configurable": true } } ] } }

the job did not throw any error for the first user in the loop ( though the process-group did not have the users assigned in the canvas PFA).

but when it run for the second user it returns 400 bad request

status 409 body Found multiple policies for 'process-groups' with 'read'.

Note:

I have tried with configurations by setting the user-group id in the policy config json

policyConfig=$(echo "${policyConfig}" | jq ".component.userGroups += [{\"revision\":{\"version\":0},\"id\":\"${tenant_superadmin_id}\",\"permissions\":{\"canRead\":true,\"canWrite\":true},\"component\":{\"id\":\"${tenant_superadmin_id}\",\"identity\":\"${tenant_superadmin}\",\"configurable\":true}}]")

This did not work, because processor group id is not linked

Can anyone please advise to how to set the policies uniquely for each process group so that only the user and super admins can access it?

From the canvas it is achievable, when i select override as empty, But i need this to be done via API call

Thanks much for your time.

Re: How to create process-groups and apply custom policies to the processor

scoutjohn — Sun, 16 Mar 2025 13:16:35 GMT

Note 2:

I have also tried to fetch the policy id from the processor and appending it a policy configuration

using

nifi_api_request "policies/${action}${resource}" "GET"

example

policies/read/data/process-groups/9486a139-0195-1000-0000-00000ddc9b4f

gives a json

job will fetch the ID of the policy

the call the policy

nifi_api_request "policies/${policyId}" "GET"

use policyConfig to append info where entity_id is the id of the user and entity_name is the name

policyConfig=$(echo ${body} | jq ".component.userGroups[.component.userGroups | length] |= {\"revision\":{\"version\":0},\"id\":\"${entity_id}\",\"permissions\":{\"canRead\":${canRead},\"canWrite\":${canWrite}},\"component\":{\"id\":\"${entity_id}\",\"identity\":\"${entity_name}\",\"configurable\":true}}")

This unfortunately started created large json and ended up assigning all the users access to all the process-groups

Re: How to create process-groups and apply custom policies to the processor via Nifi API

scoutjohn — Mon, 17 Mar 2025 05:51:59 GMT

Update :

i realized that I wasn't adding process-group id to the resources, so i added the process-group to the policy config json

policy Group Config: { "revision": { "version": 0 }, "permissions": { "canRead": true, "canWrite": true }, "component": { "resource": "data-transfer/output-ports/a2a202da-0195-1000-0000-000045d2086d", "action": "write", "configurable": true, "users": [], "userGroups": [ { "revision": { "version": 0 }, "id": "3d872ce1-e08a-4794-85e5-66e5a1f2f4ac", "permissions": { "canRead": true, "canWrite": true }, "component": { "id": "3d872ce1-e08a-4794-85e5-66e5a1f2f4ac", "identity": "TenantID.john.nifi_superadmin", "configurable": true } }, { "revision": { "version": 0 }, "id": "ebe9c88a-77d7-4070-bead-e24329b2e9c1", "permissions": { "canRead": true, "canWrite": true }, "component": { "id": "ebe9c88a-77d7-4070-bead-e24329b2e9c1", "identity": "nifi_superadmin", "configurable": true } } ] } }

"data-transfer/output-ports/a2a202da-0195-1000-0000-000045d2086d",

where a2a202da-0195-1000-0000-000045d2086d is the id of the process group

all the policies were created with 201

when i try to access the policies of the process group , i don't see the respective user-group having privilege to it, it just has super admins in the list

Re: How to create process-groups and apply custom policies to the processor via Nifi API

scoutjohn — Mon, 17 Mar 2025 08:46:13 GMT

Update :

Solved it

i was missing a '/' infront of the resources that i was providing

it should have been

"resource": "/data-transfer/output-ports/a2a202da-0195-1000-0000-000045d2086d",

instead of

"resource": "data-transfer/output-ports/a2a202da-0195-1000-0000-000045d2086d",

comparing the policies created with UI and with API made me realize it