Solved
How to create process-groups and apply custom policies to the processor via Nifi API
Hi All
I have a list of users that i have in an array.
I am trying to create a job in shell script that creates a process-group for each user and apply policy to that particular process group so that only that user and nifi's super user can access or operate in it
here is how the polices config json looks like
'[{"globalAccessPolicies":[{"resource":"flow","action":"read"},{"resource":"provenance","action":"read"}],"componentLevelAccessPolicies":[{"resource":"process-groups","action":"read"},{"resource":"process-groups","action":"write"},{"resource":"operation/process-groups","action":"write"},{"resource":"provenance-data/process-groups","action":"write"},{"resource":"provenance-data/process-groups","action":"read"},{"resource":"data/process-groups","action":"read"},{"resource":"data/process-groups","action":"write"},{"resource":"policies/process-groups","action":"read"},{"resource":"policies/process-groups","action":"write"},{"resource":"data-transfer/input-ports","action":"write"},{"resource":"data-transfer/output-ports","action":"write"}]}]'
For each user in the array, the job iterates through globalAccessPolicies and componentLevelAccessPolicies and assigns permission to the user.
before assigning the componentLevelAccessPolicies the job creates an empty process-group to in the nifi canvas
before assigning the componentLevelAccessPolicies the job creates an empty process-group to in the nifi canvas
using the api
nifi_api_request "process-groups/root/process-groups" "POST" "Content-Type: application/json" "{\"revision\":{\"version\":0},\"component\":{\"name\":\"${tenant}\",\"position\":{\"x\":${x},\"y\":${y}},\"comments\":\"Processor group for ${tenant}\"}}"
this returns a json from which the id of the process group is fetched
then the Job uses the fetched ID, uses the componentLevelAccessPolicies array and tries to create a policy configuration using
for policy in $(echo "${componentLevelAccessPolicies}" | jq -c '.[]'); do
resource=$(echo "${policy}" | jq -r '.resource')
action=$(echo "${policy}" | jq -r '.action')
policyConfig=$(echo '{
"revision": {
"version": 0
},
"component": {
"resource": "'${resource}'",
"action": "'${action}'",
"configurable": true,
"users": [],
"userGroups": []
}
}' | jq .)
policyConfig=$(echo "${policyConfig}" | jq ".component.userGroups += [{\"revision\":{\"version\":0},\"id\":\"${tenant_superadmin_id}\",\"permissions\":{\"canRead\":true,\"canWrite\":true},\"component\":{\"id\":\"${processor_group_id}\",\"identity\":\"${tenant_superadmin}\",\"configurable\":true}}]")
policyConfig=$(echo "${policyConfig}" | jq ".component.userGroups += [{\"revision\":{\"version\":0},\"id\":\"${tenant_readonly_id}\",\"permissions\":{\"canRead\":true,\"canWrite\":false},\"component\":{\"id\":\"${processor_group_id}\",\"identity\":\"${tenant_readonly}\",\"configurable\":true}}]")
policyConfig=$(echo "${policyConfig}" | jq ".component.userGroups += [{\"revision\":{\"version\":0},\"id\":\"${nifi_superadmin_id}\",\"permissions\":{\"canRead\":true,\"canWrite\":true},\"component\":{\"id\":\"${processor_group_id}\",\"identity\":\"${nifi_superadmin}\",\"configurable\":true}}]")
policyConfig=$(echo "${policyConfig}" | jq ".component.userGroups += [{\"revision\":{\"version\":0},\"id\":\"${nifi_readonly_id}\",\"permissions\":{\"canRead\":true,\"canWrite\":false},\"component\":{\"id\":\"${processor_group_id}\",\"identity\":\"${nifi_readonly}\",\"configurable\":true}}]")
echo "policy Group Config: ${policyConfig}"
nifi_api_request "policies/" "POST" "Content-Type: application/json" "${policyConfig}"
echo -e "status ${status}"
echo -e "body ${body}"
[[ ${status} -eq 201 ]] || exit 1
done
The required Ids are fetched beforehand from the user groups
Since job is trying to create policy to each processor group, i have set the http method as POST
nifi_api_request "policies/" "POST" "Content-Type: application/json" "${policyConfig}"
in the Loop it creates a json paylod like this
{
"revision": {
"version": 0
},
"component": {
"resource": "data-transfer/output-ports",
"action": "write",
"configurable": true,
"users": [],
"userGroups": [
{
"revision": {
"version": 0
},
"id": "9fd4eabd-5b6f-4a1d-8c5f-ca6049986d96",
"permissions": {
"canRead": true,
"canWrite": true
},
"component": {
"id": "9ef8a5e3-0195-1000-ffff-ffffb4a7b545",
"identity": "TenantID.john.nifi_superadmin",
"configurable": true
}
},
{
"revision": {
"version": 0
},
"id": "aba0f614-d09a-42ee-9081-3328c86fcd6e",
"permissions": {
"canRead": true,
"canWrite": false
},
"component": {
"id": "9ef8a5e3-0195-1000-ffff-ffffb4a7b545",
"identity": "TenantID.john.nifi_readonly",
"configurable": true
}
},
{
"revision": {
"version": 0
},
"id": "ebe9c88a-77d7-4070-bead-e24329b2e9c1",
"permissions": {
"canRead": true,
"canWrite": true
},
"component": {
"id": "9ef8a5e3-0195-1000-ffff-ffffb4a7b545",
"identity": "nifi_superadmin",
"configurable": true
}
},
{
"revision": {
"version": 0
},
"id": "95aedc49-20f6-4dad-95f5-c7311a66c353",
"permissions": {
"canRead": true,
"canWrite": false
},
"component": {
"id": "9ef8a5e3-0195-1000-ffff-ffffb4a7b545",
"identity": "nifi_readonly",
"configurable": true
}
}
]
}
}
the job did not throw any error for the first user in the loop ( though the process-group did not have the users assigned in the canvas PFA).
but when it run for the second user it returns 400 bad request
status 409
body Found multiple policies for 'process-groups' with 'read'.
Note:
I have tried with configurations by setting the user-group id in the policy config json
policyConfig=$(echo "${policyConfig}" | jq ".component.userGroups += [{\"revision\":{\"version\":0},\"id\":\"${tenant_superadmin_id}\",\"permissions\":{\"canRead\":true,\"canWrite\":true},\"component\":{\"id\":\"${tenant_superadmin_id}\",\"identity\":\"${tenant_superadmin}\",\"configurable\":true}}]")
This did not work, because processor group id is not linked
Can anyone please advise to how to set the policies uniquely for each process group so that only the user and super admins can access it?
From the canvas it is achievable, when i select override as empty, But i need this to be done via API call
Thanks much for your time.
