question Re: No available buckets after secure Nifi connected to secure Nifi registry in Support Questions

No available buckets after secure Nifi connected to secure Nifi registry

Armel316 — Wed, 13 Nov 2024 05:25:14 GMT

Hello,

I have deployed Apache NiFi and NiFi Registry on a Kubernetes cluster, but I am unable to retrieve the buckets in the NiFi interface. However, when I set the buckets to "public", I can see them in the "Network" tab of the browser’s developer tools.

I have been troubleshooting this issue for over a month without success. If anyone has encountered a similar situation or has any suggestions on what might be causing this, I would greatly appreciate any help or advice.

Thank you in advance for your assistance!

@MattWho

Re: No available buckets after secure Nifi connected to secure Nifi registry

VidyaSargur — Wed, 13 Nov 2024 06:25:14 GMT

@Armel316, Welcome to our community! To help you get the best possible answer, I have tagged in our NiFi experts @MattWho @SAMSAL who may be able to assist you further.

Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.

Re: No available buckets after secure Nifi connected to secure Nifi registry

satz — Wed, 13 Nov 2024 12:05:13 GMT

@Armel316 Does the Nifi Registry setup with Authorisation, probably we may need to refer this doc: https://docs.cloudera.com/cfm/2.1.7/cfm-security/topics/cdf-datahub-nifi-security-predefined-nifiregistry-policies.html

Re: No available buckets after secure Nifi connected to secure Nifi registry

Armel316 — Wed, 13 Nov 2024 13:40:14 GMT

@satz I’m not sure I fully understand what you mean regarding the Nifi Registry authorization setup. Could you please clarify and provide more details?

Re: No available buckets after secure Nifi connected to secure Nifi registry

MattWho — Wed, 13 Nov 2024 17:12:10 GMT

@Armel316

Lets discuss first what needs to happen successfully when a secured NiFi is connecting with a secured NiFi-Registry.

When NiFi connects to NiFi-Registry client URL, it does so using the either the keystore and truststore configured in the NiFi-Registry Client's StandardRestrictedSSLContextService setup within NiFi or using the keystore and truststore setup in the nifi.properties when no StandardRestrictedSSLContextService was setup in the NiFi-Registry Client. A mutualTLS handshake will be attempted between NiFi and NiFi-Registry. NiFi-Registry will "WANT" the client (NiFi) to provide clientAuth certificate. If one is not provided, NiFi-Registry will proceed using the anonymous user (Anonymous user only has read on public buckets which align with what you shared from developer tools). So an unsuccessful mutualTLS handshake is most likely your issue currently.

To answer the possible next question.... If It shows "read" on the bucket in developer tools, why does NiFi UI does not show the bucket?

This is because the UI opened was for starting version control on an process group on the NiFi canvas. This UI will only show buckets for which the user identity currently authenticated into NiFi is authorized read and write on.

Next question: My NiFi user is authorized read and write on the bucket in NiFi-Registry, so why is bucket not showing?

NiFi authenticates with NiFi-Registry via a mutualTLS handshake. The client/user identity derived form the clientAuth certificate DN for the NIFi node is used as the identity passed to NiFi-Registry. Assuming the MutualTLS handshake is successful, the node user identity must be authorized "read" on all buckets and "read, write, and delete" on proxy user requests. This allows the node to proxy request on behalf of the user authenticated in NiFi. So only the buckets for which the authenticated user identity in NiFi has been authorized read, write, and delete on within NiFi-Registry will be shown in the list.

Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

Re: No available buckets after secure Nifi connected to secure Nifi registry

Armel316 — Thu, 14 Nov 2024 05:31:32 GMT

Here is one of the configurations of authorizers.xml for the NiFi nodes :

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>

<identifier>file-user-group-provider</identifier>

<class>org.apache.nifi.authorization.FileUserGroupProvider</class>

<property name="Users File">./conf/users.xml</property>

<property name="Initial Admin Identity">initial_admin_identity</property>

</userGroupProvider>

<identifier>ldap-user-group-provider</identifier>

<class>org.apache.nifi.ldap.tenants.LdapUserGroupProvider</class>

<property name="Authentication Strategy">SIMPLE</property>

<property name="Manager DN">ldap_manager_dn</property>

<property name="Manager Password">ldap_manager_password</property>

<property name="TLS - Keystore">/opt/certs/nifi0/keystore.jks</property>

<property name="TLS - Truststore">/opt/certs/nifi0/truststore.jks</property>

<property name="Referral Strategy">FOLLOW</property>

<property name="Group Membership - Enforce Case Sensitivity">false</property>

<property name="User Search Base">ldap_user_search_base</property>

<property name="User Object Class">person</property>

<property name="User Search Scope">SUBTREE</property>

<property name="User Search Filter">ldap_user_search_filter_authorizers</property>

<property name="User Identity Attribute">sAMAccountName</property>

<property name="User Group Name Attribute">memberof</property>

<property name="Identity Strategy">USE_USERNAME</property>

<property name="Group Search Base">ldap_group_search_base</property>

<property name="Group Object Class">group</property>

<property name="Group Search Scope">ONE_LEVEL</property>

<property name="Group Search Filter">(objectClass=group)</property>

<property name="Group Member Attribute">member</property>

<property name="Group Member Attribute - Referenced User Attribute">sAMAccountName</property>

</userGroupProvider>

<identifier>composite-user-group-provider</identifier>

<class>org.apache.nifi.authorization.CompositeUserGroupProvider</class>

<property name="User Group Provider 1">file-user-group-provider</property>

<property name="User Group Provider 2">ldap-user-group-provider</property>

</userGroupProvider>

<identifier>file-access-policy-provider</identifier>

<class>org.apache.nifi.authorization.FileAccessPolicyProvider</class>

<property name="User Group Provider">composite-user-group-provider</property>

<property name="Authorizations File">./conf/authorizations.xml</property>

<property name="Initial Admin Identity">initial_admin_identity</property>

</accessPolicyProvider>

<identifier>managed-authorizer</identifier>

<class>org.apache.nifi.authorization.StandardManagedAuthorizer</class>

<property name="Access Policy Provider">file-access-policy-provider</property>

</authorizer>

<identifier>single-user-authorizer</identifier>

<class>org.apache.nifi.authorization.single.user.SingleUserAuthorizer</class>

</authorizer>

</authorizers>

And here is my authorizers.xml for the NiFi Registry.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>

<identifier>file-user-group-provider</identifier>

<class>org.apache.nifi.registry.security.authorization.file.FileUserGroupProvider</class>

<property name="Users File">./conf/users.xml</property>

<property name="Initial User Identity 1">initial_admin_identity</property>

</userGroupProvider>

<identifier>ldap-user-group-provider</identifier>

<class>org.apache.nifi.registry.security.ldap.tenants.LdapUserGroupProvider</class>

<property name="Authentication Strategy">SIMPLE</property>

<property name="Manager DN">ldap_manager_dn</property>

<property name="Manager Password">ldap_manager_password</property>

<property name="TLS - Keystore">/opt/certs/registry/keystore.jks</property>

<property name="TLS - Truststore">/opt/certs/registry/truststore.jks</property>

<property name="Referral Strategy">FOLLOW</property>

<property name="Group Membership - Enforce Case Sensitivity">false</property>

<property name="User Search Base">ldap_user_search_base</property>

<property name="User Object Class">person</property>

<property name="User Search Scope">SUBTREE</property>

<property name="User Search Filter">ldap_user_search_filter_authorizers</property>

<property name="User Identity Attribute">sAMAccountName</property>

<property name="User Group Name Attribute">memberof</property>

<property name="Identity Strategy">USE_USERNAME</property>

<property name="Group Search Base">ldap_group_search_base</property>

<property name="Group Object Class">group</property>

<property name="Group Search Scope">ONE_LEVEL</property>

<property name="Group Search Filter">(objectClass=group)</property>

<property name="Group Member Attribute">member</property>

<property name="Group Member Attribute - Referenced User Attribute">sAMAccountName</property>

</userGroupProvider>

<identifier>file-access-policy-provider</identifier>

<class>org.apache.nifi.registry.security.authorization.file.FileAccessPolicyProvider</class>

<property name="User Group Provider">ldap-user-group-provider</property>

<property name="Authorizations File">./conf/authorizations.xml</property>

<property name="Initial Admin Identity">initial_admin_identity</property>

</accessPolicyProvider>

<identifier>managed-authorizer</identifier>

<class>org.apache.nifi.registry.security.authorization.StandardManagedAuthorizer</class>

<property name="Access Policy Provider">file-access-policy-provider</property>

</authorizer>

</authorizers>

The certificates for the nodes and NiFi Registry are generated by the NiFi Toolkit using the following command and then mounted on each NiFi and Registry node :

command:

- "bash"

- "-c"

- "/opt/nifi-toolkit/*/bin/tls-toolkit.sh standalone -o /opt/certs -n nifi[0-2] -P xxx -K xxx -S xxx; /opt/nifi-toolkit/*/bin/tls-toolkit.sh standalone -o /opt/certs -n registry -P xxx -K xxx -S xxx;"

Could you let me know if I missed something or if I made any mistakes in my configuration?

Re: No available buckets after secure Nifi connected to secure Nifi registry

MattWho — Thu, 14 Nov 2024 21:35:44 GMT

@Armel316

I see the single-user-authorizer in your NiFi authorizers.xml (hope you are not using this and your nifi.properties is configured to use the managed-authorizer.)

Now of course I can not validate yoru configured providers, but can tell you the structure of your NiFi authorizers.xml is valid.

However, the structure of your NiFi-Registry authorizers.xml is not valid.
Best to read it from bottom up starting with the authorizer which you have as the managed-authorizer. It calls the file-access-policy-provider which in turn references the ldap-user-group-provider. I see that like your NiFi authorizers, your NiFi-Registry authorizers.xml also has the file-user-group-provider configured in it, but the configuration within NiFi-Registry, the managed-authorizer will never use it.

Because the file-user-group-provider is not used, it is not possible to setup the authorization policies I mention in my last response which must be setup for the NiFi node identities.

Also make sure that both your nifi.properties and nifi-registry.properties files have the same configured identity.mapping.* properties. I see you created your node identities using the node full DNs. The identity mapping patterns if setup might be manipulating/trimming those DNs during a mutualTLS handshake causing them to to not match the full DNs.

I suggest you also tail the nifi-registry-app.log while you try to start version control to capture what identities are being used to check authorizations against.

You'll also want to validate the contents of your NiFi and NiFi-Registry truststore.jks to make sure mutual trust can be established between the two services using the clientAuth Private keys found in the keystore.jks files on both services.

Thank you,
Matt

Re: No available buckets after secure Nifi connected to secure Nifi registry

Armel316 — Fri, 15 Nov 2024 12:34:55 GMT

This is the complete configuration for all packages. What should I correct, please?

namespace: "namespace"

nameOverride: ""

fullnameOverride: "dt-service"

serviceAccount:

create: false

ingress:

enabled: true

className: "nginx"

annotations:

nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"

nginx.ingress.kubernetes.io/ssl-passthrough: "true"

hosts:

- host: service_url

paths:

- path: /

pathType: Prefix

backend:

service:

port:

number: 8444

- host: registry_service_url

paths:

- path: /

pathType: Prefix

backend:

service:

port:

number: 18443

tls:

- secretName: app_tls_secret_name

hosts:

- service_url

- registry_service_url

### PROXY ###

proxy:

replicaCount: 1

imagePullSecrets:

- name: secret

restartPolicy: Always

autoscaling:

enabled: false

minReplicas: 1

maxReplicas: 3

targetCPUUtilizationPercentage: 80

targetMemoryUtilizationPercentage: 80

image:

repository: nginx_image

pullPolicy: IfNotPresent

tag: "nginx_version"

service:

type: NodePort

port: 8444

nodePort: 30844

env: []

volumeMounts:

- name: nginx-conf

mountPath: /etc/nginx/nginx.conf

subPath: nginx.conf

- name: ssl-certificates

mountPath: /etc/nginx/ssl

- name: nifi-certs

mountPath: /opt/certs

volumes:

- name: nginx-conf

configMap:

- name: ssl-certificates

secret:

secretName: app_tls_secret_name

- name: nifi-certs

persistentVolumeClaim:

claimName: nifi-certs-pvc

resources:

limits:

cpu: 100m

memory: 128Mi

requests:

cpu: 100m

memory: 128Mi

### NIFI0 ###

nifi0:

replicaCount: 1

imagePullSecrets:

- name: secret

restartPolicy: Always

autoscaling:

enabled: false

minReplicas: 1

maxReplicas: 3

targetCPUUtilizationPercentage: 80

targetMemoryUtilizationPercentage: 80

image:

repository: nifi_image

pullPolicy: IfNotPresent

tag: "nifi_version"

service:

type: ClusterIP

port: 8443

env:

- name: NIFI_WEB_HTTPS_PORT

value: "8443"

- name: NIFI_CLUSTER_IS_NODE

value: "true"

- name: NIFI_ZK_CONNECT_STRING

value: "nifi-zookeeper-service:2181"

- name: NIFI_ELECTION_MAX_WAIT

value: "30 sec"

- name: NIFI_ELECTION_MAX_CANDIDATES

value: "1"

- name: NIFI_SENSITIVE_PROPS_KEY

value: "xxx"

- name: NIFI_CLUSTER_NODE_PROTOCOL_PORT

value: "8082"

- name: NIFI_WEB_PROXY_HOST

value: "nifi_web_proxy_host"

- name: KEYSTORE_TYPE

value: "JKS"

- name: KEYSTORE_PASSWORD

value: "xxx"

- name: TRUSTSTORE_TYPE

value: "JKS"

- name: TRUSTSTORE_PASSWORD

value: "xxx"

- name: NIFI_CLUSTER_ADDRESS

value: "nifi0"

- name: NIFI_WEB_HTTPS_HOST

value: "xxx"

- name: KEYSTORE_PATH

value: "/opt/certs/nifi0/keystore.jks"

- name: TRUSTSTORE_PATH

value: "/opt/certs/nifi0/truststore.jks"

### LDAP ###

- name: AUTH

value: "ldap"

- name: NIFI_SECURITY_USER_LOGIN_IDENTITY_PROVIDER

value: "ldap-provider"

- name: NIFI_SECURITY_USER_AUTHORIZER

value: "managed-authorizer"

- name: INITIAL_ADMIN_IDENTITY

value: "initial_admin_identity"

- name: LDAP_AUTHENTICATION_STRATEGY

value: "SIMPLE"

- name: LDAP_MANAGER_DN

value: "ldap_manager_dn"

- name: LDAP_MANAGER_PASSWORD

value: "ldap_manager_password"

- name: LDAP_USER_SEARCH_BASE

value: "ldap_user_search_base"

- name: LDAP_USER_SEARCH_FILTER

value: "ldap_user_search_filter"

- name: LDAP_GROUP_SEARCH_BASE

value: "ldap_user_search_filter_authorizers"

- name: LDAP_IDENTITY_STRATEGY

value: "USE_USERNAME"

- name: LDAP_URL

value: "ldap_url"

securityContext:

runAsUser: 1000

initContainers:

- name: init-conf

image: nifi_image:nifi_version

command:

[

"sh",

"-c",

"cp -r /opt/nifi/nifi-current/conf/*.conf /mnt/nifi-conf/; cp -r /opt/nifi/nifi-current/conf/*.xml /mnt/nifi-conf/; cp -r /opt/nifi/nifi-current/conf/*.properties /mnt/nifi-conf/",

]

volumeMounts:

- name: nifi0-conf

mountPath: /mnt/nifi-conf

- name: nifi-authorizers

mountPath: /opt/nifi/nifi-current/conf/nifi0-authorizers.xml

subPath: nifi0-authorizers.xml

- name: nifi-bootstrap

mountPath: /opt/nifi/nifi-current/conf/nifi-bootstrap.conf

subPath: nifi-bootstrap.conf

command:

- "/bin/bash"

- "-c"

- |

cp /mnt/jar/*.jar /opt/nifi/nifi-current/lib/ && \

echo -n | openssl s_client -connect registry_service_url:443 |sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > cacert.crt && \

cp /opt/certs/nifi0/*.jks /opt/nifi/nifi-current && \

keytool -delete -alias nifi-registry-cert -keystore /opt/nifi/nifi-current/truststore.jks -storepass xxx || echo "Alias not found, skipping delete" && \

keytool -import -trustcacerts -file ./cacert.crt -keystore /opt/nifi/nifi-current/truststore.jks -storepass xxx -noprompt -alias nifi-registry-cert && \

cp /opt/nifi/nifi-current/conf/nifi-bootstrap.conf /opt/nifi/nifi-current/conf/bootstrap.conf && \

cp /opt/nifi/nifi-current/conf/nifi0-authorizers.xml /opt/nifi/nifi-current/conf/authorizers.xml && \

sed -i 's/nifi.ui.banner.text=.*/nifi.ui.banner.text=nifi (v1.27.0)/' /opt/nifi/nifi-current/conf/nifi.properties && \

../scripts/start.sh

volumeMounts:

- name: nifi-certs

mountPath: /opt/certs

- name: nifi-jar

mountPath: /mnt/jar

- name: timezone

mountPath: /etc/timezone

subPath: timezone

- name: nifi0-conf

mountPath: /opt/nifi/nifi-current/conf

- name: nifi0-extensions

mountPath: /opt/nifi/nifi-current/extensions

- name: nifi0-database-repository

mountPath: /opt/nifi/nifi-current/database_repository

- name: nifi0-flowfile-repository

mountPath: /opt/nifi/nifi-current/flowfile_repository

- name: nifi0-content-repository

mountPath: /opt/nifi/nifi-current/content_repository

- name: nifi0-provenance-repository

mountPath: /opt/nifi/nifi-current/provenance-repository

- name: nifi0-state

mountPath: /opt/nifi/nifi-current/state

- name: nifi0-logs

mountPath: /opt/nifi/nifi-current/logs

- name: nifi-logback

mountPath: /opt/nifi/nifi-current/conf/logback.xml

subPath: logback.xml

volumes:

- name: nifi-certs

persistentVolumeClaim:

claimName: nifi-certs-pvc

- name: nifi-jar

persistentVolumeClaim:

claimName: nifi-jar-pvc

- name: timezone

configMap:

- name: nifi0-conf

persistentVolumeClaim:

claimName: nifi0-conf-pvc

- name: nifi0-extensions

persistentVolumeClaim:

claimName: nifi0-extensions-pvc

- name: nifi0-database-repository

persistentVolumeClaim:

claimName: nifi0-database-repository-pvc

- name: nifi0-flowfile-repository

persistentVolumeClaim:

claimName: nifi0-flowfile-repository-pvc

- name: nifi0-content-repository

persistentVolumeClaim:

claimName: nifi0-content-repository-pvc

- name: nifi0-provenance-repository

persistentVolumeClaim:

claimName: nifi0-provenance-repository-pvc

- name: nifi0-state

persistentVolumeClaim:

claimName: nifi0-state-pvc

- name: nifi0-logs

persistentVolumeClaim:

claimName: nifi0-logs-pvc

- name: nifi-authorizers

configMap:

- name: nifi-bootstrap

configMap:

- name: nifi-logback

configMap:

resources:

requests:

cpu: 1

memory: 1Gi

limits:

cpu: 2

memory: 2Gi

### NIFI1 ###

nifi1:

replicaCount: 1

imagePullSecrets:

- name: secret

restartPolicy: Always

autoscaling:

enabled: false

minReplicas: 1

maxReplicas: 3

targetCPUUtilizationPercentage: 80

targetMemoryUtilizationPercentage: 80

image:

repository: nifi_image

pullPolicy: IfNotPresent

tag: "nifi_version"

service:

type: ClusterIP

port: 8443

env:

- name: NIFI_WEB_HTTPS_PORT

value: "8443"

- name: NIFI_CLUSTER_IS_NODE

value: "true"

- name: NIFI_ZK_CONNECT_STRING

value: "nifi-zookeeper-service:2181"

- name: NIFI_ELECTION_MAX_WAIT

value: "30 sec"

- name: NIFI_ELECTION_MAX_CANDIDATES

value: "1"

- name: NIFI_SENSITIVE_PROPS_KEY

value: "xxx"

- name: NIFI_CLUSTER_NODE_PROTOCOL_PORT

value: "8082"

- name: NIFI_WEB_PROXY_HOST

value: "nifi_web_proxy_host"

- name: KEYSTORE_TYPE

value: "JKS"

- name: KEYSTORE_PASSWORD

value: "xxx"

- name: TRUSTSTORE_TYPE

value: "JKS"

- name: TRUSTSTORE_PASSWORD

value: "xxx"

- name: NIFI_CLUSTER_ADDRESS

value: "nifi1"

- name: NIFI_WEB_HTTPS_HOST

value: "nifi1"

- name: KEYSTORE_PATH

value: "/opt/certs/nifi1/keystore.jks"

- name: TRUSTSTORE_PATH

value: "/opt/certs/nifi1/truststore.jks"

### LDAP ###

- name: AUTH

value: "ldap"

- name: NIFI_SECURITY_USER_LOGIN_IDENTITY_PROVIDER

value: "ldap-provider"

- name: NIFI_SECURITY_USER_AUTHORIZER

value: "managed-authorizer"

- name: INITIAL_ADMIN_IDENTITY

value: "initial_admin_identity"

- name: LDAP_AUTHENTICATION_STRATEGY

value: "SIMPLE"

- name: LDAP_MANAGER_DN

value: "ldap_manager_dn"

- name: LDAP_MANAGER_PASSWORD

value: "ldap_manager_password"

- name: LDAP_USER_SEARCH_BASE

value: "ldap_user_search_base"

- name: LDAP_USER_SEARCH_FILTER

value: "ldap_user_search_filter"

- name: LDAP_GROUP_SEARCH_BASE

value: "ldap_user_search_filter_authorizers"

- name: LDAP_IDENTITY_STRATEGY

value: "USE_USERNAME"

- name: LDAP_URL

value: "ldap_url"

securityContext:

runAsUser: 1000

initContainers:

- name: init-conf

image: nifi_image:nifi_version

command:

[

"sh",

"-c",

"cp -r /opt/nifi/nifi-current/conf/*.conf /mnt/nifi-conf/; cp -r /opt/nifi/nifi-current/conf/*.xml /mnt/nifi-conf/; cp -r /opt/nifi/nifi-current/conf/*.properties /mnt/nifi-conf/",

]

volumeMounts:

- name: nifi1-conf

mountPath: /mnt/nifi-conf

- name: nifi-authorizers

mountPath: /opt/nifi/nifi-current/conf/nifi1-authorizers.xml

subPath: nifi1-authorizers.xml

- name: nifi-bootstrap

mountPath: /opt/nifi/nifi-current/conf/nifi-bootstrap.conf

subPath: nifi-bootstrap.conf

command:

- "/bin/bash"

- "-c"

- |

cp /mnt/jar/*.jar /opt/nifi/nifi-current/lib/ && \

echo -n | openssl s_client -connect registry_service_url:443 |sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > cacert.crt && \

cp /opt/certs/nifi1/*.jks /opt/nifi/nifi-current && \

keytool -delete -alias nifi-registry-cert -keystore /opt/nifi/nifi-current/truststore.jks -storepass xxx || echo "Alias not found, skipping delete" && \

keytool -import -trustcacerts -file ./cacert.crt -keystore /opt/nifi/nifi-current/truststore.jks -storepass xxx -noprompt -alias nifi-registry-cert && \

cp /opt/nifi/nifi-current/conf/nifi-bootstrap.conf /opt/nifi/nifi-current/conf/bootstrap.conf && \

cp /opt/nifi/nifi-current/conf/nifi1-authorizers.xml /opt/nifi/nifi-current/conf/authorizers.xml && \

sed -i 's/nifi.ui.banner.text=.*/nifi.ui.banner.text=nifi (v1.27.0)/' /opt/nifi/nifi-current/conf/nifi.properties && \

../scripts/start.sh

volumeMounts:

- name: nifi-certs

mountPath: /opt/certs

- name: nifi-jar

mountPath: /mnt/jar

- name: timezone

mountPath: /etc/timezone

subPath: timezone

- name: nifi1-conf

mountPath: /opt/nifi/nifi-current/conf

- name: nifi1-extensions

mountPath: /opt/nifi/nifi-current/extensions

- name: nifi1-database-repository

mountPath: /opt/nifi/nifi-current/database_repository

- name: nifi1-flowfile-repository

mountPath: /opt/nifi/nifi-current/flowfile_repository

- name: nifi1-content-repository

mountPath: /opt/nifi/nifi-current/content_repository

- name: nifi1-provenance-repository

mountPath: /opt/nifi/nifi-current/provenance-repository

- name: nifi1-state

mountPath: /opt/nifi/nifi-current/state

- name: nifi1-logs

mountPath: /opt/nifi/nifi-current/logs

- name: nifi-logback

mountPath: /opt/nifi/nifi-current/conf/logback.xml

subPath: logback.xml

volumes:

- name: nifi-certs

persistentVolumeClaim:

claimName: nifi-certs-pvc

- name: nifi-jar

persistentVolumeClaim:

claimName: nifi-jar-pvc

- name: timezone

configMap:

- name: nifi1-conf

persistentVolumeClaim:

claimName: nifi1-conf-pvc

- name: nifi1-extensions

persistentVolumeClaim:

claimName: nifi1-extensions-pvc

- name: nifi1-database-repository

persistentVolumeClaim:

claimName: nifi1-database-repository-pvc

- name: nifi1-flowfile-repository

persistentVolumeClaim:

claimName: nifi1-flowfile-repository-pvc

- name: nifi1-content-repository

persistentVolumeClaim:

claimName: nifi1-content-repository-pvc

- name: nifi1-provenance-repository

persistentVolumeClaim:

claimName: nifi1-provenance-repository-pvc

- name: nifi1-state

persistentVolumeClaim:

claimName: nifi1-state-pvc

- name: nifi1-logs

persistentVolumeClaim:

claimName: nifi1-logs-pvc

- name: nifi-authorizers

configMap:

- name: nifi-bootstrap

configMap:

- name: nifi-logback

configMap:

resources:

requests:

cpu: 1

memory: 1Gi

limits:

cpu: 2

memory: 2Gi

### NIFI2 ###

nifi2:

replicaCount: 1

imagePullSecrets:

- name: secret

restartPolicy: Always

autoscaling:

enabled: false

minReplicas: 1

maxReplicas: 3

targetCPUUtilizationPercentage: 80

targetMemoryUtilizationPercentage: 80

image:

repository: nifi_image

pullPolicy: IfNotPresent

tag: "nifi_version"

service:

type: ClusterIP

port: 8443

env:

- name: NIFI_WEB_HTTPS_PORT

value: "8443"

- name: NIFI_CLUSTER_IS_NODE

value: "true"

- name: NIFI_ZK_CONNECT_STRING

value: "nifi-zookeeper-service:2181"

- name: NIFI_ELECTION_MAX_WAIT

value: "30 sec"

- name: NIFI_ELECTION_MAX_CANDIDATES

value: "1"

- name: NIFI_SENSITIVE_PROPS_KEY

value: "xxx"

- name: NIFI_CLUSTER_NODE_PROTOCOL_PORT

value: "8082"

- name: NIFI_WEB_PROXY_HOST

value: "nifi_web_proxy_host"

- name: KEYSTORE_TYPE

value: "JKS"

- name: KEYSTORE_PASSWORD

value: "xxx"

- name: TRUSTSTORE_TYPE

value: "JKS"

- name: TRUSTSTORE_PASSWORD

value: "xxx"

- name: NIFI_CLUSTER_ADDRESS

value: "nifi2"

- name: NIFI_WEB_HTTPS_HOST

value: "nifi2"

- name: KEYSTORE_PATH

value: "/opt/certs/nifi2/keystore.jks"

- name: TRUSTSTORE_PATH

value: "/opt/certs/nifi2/truststore.jks"

### LDAP ###

- name: AUTH

value: "ldap"

- name: NIFI_SECURITY_USER_LOGIN_IDENTITY_PROVIDER

value: "ldap-provider"

- name: NIFI_SECURITY_USER_AUTHORIZER

value: "managed-authorizer"

- name: INITIAL_ADMIN_IDENTITY

value: "initial_admin_identity"

- name: LDAP_AUTHENTICATION_STRATEGY

value: "SIMPLE"

- name: LDAP_MANAGER_DN

value: "ldap_manager_dn"

- name: LDAP_MANAGER_PASSWORD

value: "ldap_manager_password"

- name: LDAP_USER_SEARCH_BASE

value: "ldap_user_search_base"

- name: LDAP_USER_SEARCH_FILTER

value: "ldap_user_search_filter"

- name: LDAP_GROUP_SEARCH_BASE

value: "ldap_user_search_filter_authorizers"

- name: LDAP_IDENTITY_STRATEGY

value: "USE_USERNAME"

- name: LDAP_URL

value: "ldap_url"

securityContext:

runAsUser: 1000

initContainers:

- name: init-conf

image: nifi_image:nifi_version

command:

[

"sh",

"-c",

"cp -r /opt/nifi/nifi-current/conf/*.conf /mnt/nifi-conf/; cp -r /opt/nifi/nifi-current/conf/*.xml /mnt/nifi-conf/; cp -r /opt/nifi/nifi-current/conf/*.properties /mnt/nifi-conf/",

]

volumeMounts:

- name: nifi2-conf

mountPath: /mnt/nifi-conf

- name: nifi-authorizers

mountPath: /opt/nifi/nifi-current/conf/nifi2-authorizers.xml

subPath: nifi2-authorizers.xml

- name: nifi-bootstrap

mountPath: /opt/nifi/nifi-current/conf/nifi-bootstrap.conf

subPath: nifi-bootstrap.conf

command:

- "/bin/bash"

- "-c"

- |

cp /mnt/jar/*.jar /opt/nifi/nifi-current/lib/ && \

echo -n | openssl s_client -connect registry_service_url:443 |sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > cacert.crt && \

cp /opt/certs/nifi2/*.jks /opt/nifi/nifi-current && \

keytool -delete -alias nifi-registry-cert -keystore /opt/nifi/nifi-current/truststore.jks -storepass xxx || echo "Alias not found, skipping delete" && \

keytool -import -trustcacerts -file ./cacert.crt -keystore /opt/nifi/nifi-current/truststore.jks -storepass xxx -noprompt -alias nifi-registry-cert && \

cp /opt/nifi/nifi-current/conf/nifi-bootstrap.conf /opt/nifi/nifi-current/conf/bootstrap.conf && \

cp /opt/nifi/nifi-current/conf/nifi2-authorizers.xml /opt/nifi/nifi-current/conf/authorizers.xml && \

sed -i 's/nifi.ui.banner.text=.*/nifi.ui.banner.text=nifi (v1.27.0)/' /opt/nifi/nifi-current/conf/nifi.properties && \

../scripts/start.sh

volumeMounts:

- name: nifi-certs

mountPath: /opt/certs

- name: nifi-jar

mountPath: /mnt/jar

- name: timezone

mountPath: /etc/timezone

subPath: timezone

- name: nifi2-conf

mountPath: /opt/nifi/nifi-current/conf

- name: nifi2-extensions

mountPath: /opt/nifi/nifi-current/extensions

- name: nifi2-database-repository

mountPath: /opt/nifi/nifi-current/database_repository

- name: nifi2-flowfile-repository

mountPath: /opt/nifi/nifi-current/flowfile_repository

- name: nifi2-content-repository

mountPath: /opt/nifi/nifi-current/content_repository

- name: nifi2-provenance-repository

mountPath: /opt/nifi/nifi-current/provenance-repository

- name: nifi2-state

mountPath: /opt/nifi/nifi-current/state

- name: nifi2-logs

mountPath: /opt/nifi/nifi-current/logs

- name: nifi-logback

mountPath: /opt/nifi/nifi-current/conf/logback.xml

subPath: logback.xml

volumes:

- name: nifi-certs

persistentVolumeClaim:

claimName: nifi-certs-pvc

- name: nifi-jar

persistentVolumeClaim:

claimName: nifi-jar-pvc

- name: timezone

configMap:

- name: nifi2-conf

persistentVolumeClaim:

claimName: nifi2-conf-pvc

- name: nifi2-extensions

persistentVolumeClaim:

claimName: nifi2-extensions-pvc

- name: nifi2-database-repository

persistentVolumeClaim:

claimName: nifi2-database-repository-pvc

- name: nifi2-flowfile-repository

persistentVolumeClaim:

claimName: nifi2-flowfile-repository-pvc

- name: nifi2-content-repository

persistentVolumeClaim:

claimName: nifi2-content-repository-pvc

- name: nifi2-provenance-repository

persistentVolumeClaim:

claimName: nifi2-provenance-repository-pvc

- name: nifi2-state

persistentVolumeClaim:

claimName: nifi2-state-pvc

- name: nifi2-logs

persistentVolumeClaim:

claimName: nifi2-logs-pvc

- name: nifi-authorizers

configMap:

- name: nifi-bootstrap

configMap:

- name: nifi-logback

configMap:

resources:

requests:

cpu: 1

memory: 1Gi

limits:

cpu: 2

memory: 2Gi

### ZOOKEEPER ###

zookeeper:

replicaCount: 1

imagePullSecrets:

- name: secret

restartPolicy: Always

image:

repository: zookeeper_image

pullPolicy: IfNotPresent

tag: "zookeeper_version"

service:

type: ClusterIP

port: 2181

env:

- name: ALLOW_ANONYMOUS_LOGIN

value: "yes"

volumeMounts: []

volumes: []

resources:

requests:

cpu: 500m

memory: 1Gi

limits:

cpu: 1

memory: 2Gi

### TOOLKIT ###

toolkit:

replicaCount: 1

imagePullSecrets:

- name: secret

restartPolicy: Never

image:

repository: nifi_toolkit_image

pullPolicy: IfNotPresent

tag: "nifi_version"

service:

type: ClusterIP

port: 80

env: []

securityContext:

runAsUser: 1000

command:

- "bash"

- "-c"

- "/opt/nifi-toolkit/*/bin/tls-toolkit.sh standalone -o /opt/certs -n nifi[0-2] -C 'CN=initial_admin_identity' -P xxx -K xxx -S xxx; /opt/nifi-toolkit/*/bin/tls-toolkit.sh standalone -o /opt/certs -n registry -P xxx -K xxx -S xxx; chown -R nifi:nifi /opt/certs"

volumeMounts:

- name: nifi-certs

mountPath: /opt/certs

volumes:

- name: nifi-certs

persistentVolumeClaim:

claimName: nifi-certs-pvc

resources:

limits:

cpu: 100m

memory: 128Mi

requests:

cpu: 100m

memory: 128Mi

### NIFI REGISTRY ###

nifiregistry:

replicaCount: 1

imagePullSecrets:

- name: secret

restartPolicy: Always

autoscaling:

enabled: false

minReplicas: 1

maxReplicas: 3

targetCPUUtilizationPercentage: 80

targetMemoryUtilizationPercentage: 80

image:

repository: nifi_registry_image }}

pullPolicy: IfNotPresent

tag: "nifi_version"

service:

type: NodePort

port: 18443

nodePort: 30843

env:

- name: NIFI_REGISTRY_WEB_HTTPS_PORT

value: "18443"

- name: NIFI_REGISTRY_SECURITY_NEED_CLIENT_AUTH

value: "true"

### CERTIFICATE ###

- name: KEYSTORE_TYPE

value: "JKS"

- name: KEYSTORE_PASSWORD

value: "xxx"

- name: TRUSTSTORE_TYPE

value: "JKS"

- name: TRUSTSTORE_PASSWORD

value: "xxx"

- name: KEYSTORE_PATH

value: "/opt/certs/registry/keystore.jks"

- name: TRUSTSTORE_PATH

value: "/opt/certs/registry/truststore.jks"

### LDAP ###

- name: AUTH

value: "ldap"

- name: NIFI_SECURITY_USER_LOGIN_IDENTITY_PROVIDER

value: "ldap-provider"

- name: NIFI_SECURITY_USER_AUTHORIZER

value: "managed-authorizer"

- name: INITIAL_ADMIN_IDENTITY

value: "initial_admin_identity"

- name: LDAP_AUTHENTICATION_STRATEGY

value: "SIMPLE"

- name: LDAP_MANAGER_DN

value: "ldap_manager_dn"

- name: LDAP_MANAGER_PASSWORD

value: "ldap_manager_password"

- name: LDAP_USER_SEARCH_BASE

value: "ldap_user_search_base"

- name: LDAP_USER_SEARCH_FILTER

value: "ldap_user_search_filter"

- name: LDAP_GROUP_SEARCH_BASE

value: "ldap_user_search_filter_authorizers"

- name: LDAP_IDENTITY_STRATEGY

value: "USE_USERNAME"

- name: LDAP_URL

value: "ldap_url"

securityContext:

runAsUser: 1000

initContainers:

- name: init-conf

image: nifi_registry_image }}:nifi_version

command:

- "sh"

- "-c"

- |

cp -r /opt/nifi-registry/nifi-registry-current/conf/*.conf /mnt/nifi-conf/ && \

cp -r /opt/nifi-registry/nifi-registry-current/conf/*.xml /mnt/nifi-conf/ && \

cp -r /opt/nifi-registry/nifi-registry-current/conf/*.properties /mnt/nifi-conf/

volumeMounts:

- name: nifiregistry-conf

mountPath: /mnt/nifi-conf

- name: registry-authorizers

mountPath: /opt/nifi-registry/nifi-registry-current/conf/registry-authorizers.xml

subPath: registry-authorizers.xml

command:

- "/bin/bash"

- "-c"

- |

echo -n | openssl s_client -connect service_url:443 |sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > cacert.crt && \

keytool -delete -alias nifi-cert -keystore /opt/certs/registry/truststore.jks -storepass xxx || echo "Alias not found, skipping delete" && \

keytool -import -trustcacerts -file ./cacert.crt -keystore /opt/certs/registry/truststore.jks -storepass xxx -noprompt -alias nifi-cert && \

cp /opt/nifi-registry/nifi-registry-current/conf/registry-authorizers.xml /opt/nifi-registry/nifi-registry-current/conf/authorizers.xml && \

sed -i 's/^nifi.registry.security.needClientAuth=false/nifi.registry.security.needClientAuth=true/' /opt/nifi-registry/nifi-registry-current/conf/nifi-registry.properties && \

../scripts/start.sh

volumeMounts:

- name: nifi-certs

mountPath: /opt/certs

- name: timezone

mountPath: /etc/timezone

subPath: timezone

- name: nifiregistry-conf

mountPath: /opt/nifi-registry/nifi-registry-current/conf

- name: nifiregistry-database

mountPath: /opt/nifi-registry/nifi-registry-current/database

- name: nifiregistry-ext

mountPath: /opt/nifi-registry/nifi-registry-current/ext

- name: nifiregistry-ext-bundles

mountPath: /opt/nifi-registry/nifi-registry-current/extension_bundles

- name: nifiregistry-flow-storage

mountPath: /opt/nifi-registry/nifi-registry-current/flow_storage

- name: nifiregistry-flow-storage-versioned

mountPath: /opt/nifi-registry/nifi-registry-current/versioned_flows

- name: nifi-providers

mountPath: /opt/nifi-registry/nifi-registry-current/conf/providers.xml

subPath: providers.xml

- name: nifiregistry-logs

mountPath: /opt/nifi-registry/nifi-registry-current/logs

volumes:

- name: nifi-certs

persistentVolumeClaim:

claimName: nifi-certs-pvc

- name: timezone

configMap:

- name: nifiregistry-conf

persistentVolumeClaim:

claimName: nifiregistry-conf-pvc

- name: nifiregistry-database

persistentVolumeClaim:

claimName: nifiregistry-database-pvc

- name: nifiregistry-ext

persistentVolumeClaim:

claimName: nifiregistry-ext-pvc

- name: nifiregistry-ext-bundles

persistentVolumeClaim:

claimName: nifiregistry-ext-bundles-pvc

- name: nifiregistry-flow-storage

persistentVolumeClaim:

claimName: nifiregistry-flow-storage-pvc

- name: nifiregistry-flow-storage-versioned

persistentVolumeClaim:

claimName: nifiregistry-flow-storage-versioned-pvc

- name: nifi-providers

configMap:

- name: nifiregistry-logs

persistentVolumeClaim:

claimName: nifiregistry-logs-pvc

- name: registry-authorizers

configMap:

resources:

requests:

cpu: 500m

memory: 1Gi

limits:

cpu: 1

memory: 2Gi

### STORAGE ###

storage:

accessModes:

- ReadWriteOnce

server: nfs_host

basePath: host_path

services:

nifi-certs:

path: certs

size: 100Mi

nifi-jar:

path: jar

size: 100Mi

## NIFI 0 ##

nifi0-conf:

path: nifi0/conf

size: 1Gi

nifi0-extensions:

path: nifi0/extensions

size: 1Gi

nifi0-database-repository:

path: nifi0/database_repository

size: 1Gi

nifi0-flowfile-repository:

path: nifi0/flowfile_repository

size: 1Gi

nifi0-content-repository:

path: nifi0/content_repository

size: 1Gi

nifi0-provenance-repository:

path: nifi0/provenance_repository

size: 1Gi

nifi0-state:

path: nifi0/state

size: 1Gi

nifi0-logs:

path: nifi0/logs

size: 1Gi

## NIFI 1 ##

nifi1-conf:

path: nifi1/conf

size: 1Gi

nifi1-extensions:

path: nifi1/extensions

size: 1Gi

nifi1-database-repository:

path: nifi1/database_repository

size: 1Gi

nifi1-flowfile-repository:

path: nifi1/flowfile_repository

size: 1Gi

nifi1-content-repository:

path: nifi1/content_repository

size: 1Gi

nifi1-provenance-repository:

path: nifi1/provenance_repository

size: 1Gi

nifi1-state:

path: nifi1/state

size: 1Gi

nifi1-logs:

path: nifi1/logs

size: 1Gi

## NIFI 2 ##

nifi2-conf:

path: nifi2/conf

size: 1Gi

nifi2-extensions:

path: nifi2/extensions

size: 1Gi

nifi2-database-repository:

path: nifi2/database_repository

size: 1Gi

nifi2-flowfile-repository:

path: nifi2/flowfile_repository

size: 1Gi

nifi2-content-repository:

path: nifi2/content_repository

size: 1Gi

nifi2-provenance-repository:

path: nifi2/provenance_repository

size: 1Gi

nifi2-state:

path: nifi2/state

size: 1Gi

nifi2-logs:

path: nifi2/logs

size: 1Gi

## NIFI REGISTRY ##

nifiregistry-conf:

path: nifiregistry/conf

size: 1Gi

nifiregistry-database:

path: nifiregistry/database

size: 1Gi

nifiregistry-ext:

path: nifiregistry/ext

size: 1Gi

nifiregistry-ext-bundles:

path: nifiregistry/extension_bundles

size: 1Gi

nifiregistry-flow-storage:

path: nifiregistry/flow_storage

size: 1Gi

nifiregistry-flow-storage-versioned:

path: nifiregistry/versioned_flows

size: 1Gi

nifiregistry-logs:

path: nifiregistry/logs

size: 1Gi

And This concerns my Nginx configuration:

events { worker_connections 1024; }

stream {

upstream nifi_nodes {

hash $remote_addr consistent;

server nifi0:8443;

server nifi1:8443;

server nifi2:8443;

}

server {

listen [::]:8444;

listen 8444;

proxy_ssl_certificate /opt/certs/nifi-cert.pem;

proxy_ssl_certificate_key /opt/certs/nifi-key.key;

proxy_ssl_trusted_certificate /opt/certs/nifi-cert.pem;

ssl_protocols TLSv1.2 TLSv1.3;

ssl_ciphers 'TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:ECDHE-RSA-AES128-GCM-SHA256';

ssl_certificate /etc/nginx/ssl/tls.crt;

ssl_certificate_key /etc/nginx/ssl/tls.key;

proxy_pass nifi_nodes;

}

Re: No available buckets after secure Nifi connected to secure Nifi registry

MattWho — Fri, 15 Nov 2024 14:12:36 GMT

@Armel316

I am no nginx expert, but what you need to do is setup your NiFi-Registry authorizers.xml similar to how you already setup your NiFi-authorizers.xml. You need to add the missing composite-configurable-user-group-provider (setup with file-user-group-provider and ldap-user-group-provider) and modify your file-access-policy-provider to point at the composite-configurable-user-group-provider instead of pointing directly at the ldap-user-group-provider. The only difference between NiFi and NiFi-Registry authorizers.xml are the class names.

Once the file-user-group-provider is actually being used by your NiFi-Registry, you'll need to access NiFi-Registry UI and login as you admin user so you can setup the required policies for your NiFi nodes: "Can proxy user requests" (R,W,D), and "Can manage buckets" (R).

Thank you,
Matt

Re: No available buckets after secure Nifi connected to secure Nifi registry

Armel316 — Mon, 18 Nov 2024 08:26:22 GMT

I have already adjusted the NiFi Registry authorizers.xml configuration as follows: :

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>

<identifier>file-user-group-provider</identifier>

<class>org.apache.nifi.registry.security.authorization.file.FileUserGroupProvider</class>

<property name="Users File">./conf/users.xml</property>

<property name="Initial User Identity 4">CN=localhost, OU=NIFI</property>

</userGroupProvider>

<identifier>ldap-user-group-provider</identifier>

<class>org.apache.nifi.registry.security.ldap.tenants.LdapUserGroupProvider</class>

<property name="Authentication Strategy">SIMPLE</property>

<property name="Manager DN">ldap_manager_dn</property>

<property name="Manager Password">ldap_manager_password</property>

<property name="TLS - Keystore">/opt/certs/registry/keystore.jks</property>

<property name="TLS - Keystore Password">supersecretkeystore</property>

<property name="TLS - Truststore">/opt/certs/registry/truststore.jks</property>

<property name="TLS - Truststore Password">supersecrettruststore</property>

<property name="Referral Strategy">FOLLOW</property>

<property name="Group Membership - Enforce Case Sensitivity">false</property>

<property name="User Search Base">ldap_user_search_base</property>

<property name="User Object Class">person</property>

<property name="User Search Scope">SUBTREE</property>

<property name="User Search Filter">ldap_user_search_filter_authorizers</property>

<property name="User Identity Attribute">sAMAccountName</property>

<property name="User Group Name Attribute">memberof</property>

<property name="Identity Strategy">USE_USERNAME</property>

<property name="Group Search Base">ldap_group_search_base</property>

<property name="Group Object Class">group</property>

<property name="Group Search Scope">ONE_LEVEL</property>

<property name="Group Search Filter">(objectClass=group)</property>

<property name="Group Member Attribute">member</property>

<property name="Group Member Attribute - Referenced User Attribute">sAMAccountName</property>

</userGroupProvider>

<identifier>composite-user-group-provider</identifier>

<class>org.apache.nifi.registry.security.authorization.CompositeUserGroupProvider</class>

<property name="User Group Provider 1">file-user-group-provider</property>

<property name="User Group Provider 2">ldap-user-group-provider</property>

</userGroupProvider>

<identifier>file-access-policy-provider</identifier>

<class>org.apache.nifi.registry.security.authorization.file.FileAccessPolicyProvider</class>

<property name="User Group Provider">composite-user-group-provider</property>

<property name="Authorizations File">./conf/authorizations.xml</property>

<property name="Initial Admin Identity">initial_admin_identity</property>

</accessPolicyProvider>

<identifier>managed-authorizer</identifier>

<class>org.apache.nifi.registry.security.authorization.StandardManagedAuthorizer</class>

<property name="Access Policy Provider">file-access-policy-provider</property>

</authorizer>

</authorizers>

And i've setup the required policies for NiFi nodes

but still no available bucket in nifi

Re: No available buckets after secure Nifi connected to secure Nifi registry

Armel316 — Tue, 19 Nov 2024 08:21:56 GMT

is there anyone who can help me please ?

Re: No available buckets after secure Nifi connected to secure Nifi registry

MattWho — Tue, 19 Nov 2024 18:56:24 GMT

@Armel316

You used the wrong composite provider "composite-user-group-provider".
Since one of your providers is the "file-user-group-provider" (a configurable provider- meaning one you can add users and groups to via the UI), you need to use the "composite-configurable-user-group-provider".

I see in the Apache NiFi-Registry documentation that the example is wrong.
https://nifi.apache.org/docs/nifi-registry-docs/html/administration-guide.html#composite-implementations

<userGroupProvider> <identifier>composite-configurable-user-group-provider</identifier> <class>org.apache.nifi.registry.security.authorization.CompositeConfigurableUserGroupProvider</class> <property name="Configurable User Group Provider">file-user-group-provider</property> <property name="User Group Provider 1">ldap-user-group-provider</property> </userGroupProvider>

What do you see in the nifi-registry-app.log when you try to start version control?

What is the output from:
openssl s_client -connect <nifi-registry hostname>:<port> -v
openssl s_client -connect <nifi node hostname>:<port> -v

Above can be checked to verify proper trust exists between NiFi and NiFi-Registry.

What is the exact case sensitive user identity displayed in the upper right corner of your NiFi UI for the user authenticated to NiFi that is attempting to start version control?

Does the same user Identity (case sensitive) exist in your NiFi-Registry and been assigned read, write, and delete on the bucket?

Thank you,
Matt

Re: No available buckets after secure Nifi connected to secure Nifi registry

Armel316 — Fri, 22 Nov 2024 08:26:16 GMT

Hello,

I've use composite configurable and i don't remember what change i've done in the nifi registry but now i'm getting this error : "2024-11-22 10:59:12,704 ERROR [NiFi Registry Web Server-20] o.s.b.w.servlet.support.ErrorPageFilter Forwarding to error page from request [/access] due to exception [Multiple UserGroupProviders claim to provide user xxx]
java.lang.IllegalStateException: Multiple UserGroupProviders claim to provide user xxx"

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>

<identifier>file-user-group-provider</identifier>

<class>org.apache.nifi.registry.security.authorization.file.FileUserGroupProvider</class>

<property name="Users File">./conf/users.xml</property>

</userGroupProvider>

<identifier>ldap-user-group-provider</identifier>

<class>org.apache.nifi.registry.security.ldap.tenants.LdapUserGroupProvider</class>

<property name="Authentication Strategy">SIMPLE</property>

<property name="Manager DN">ldap_manager_dn</property>

<property name="Manager Password">ldap_manager_password</property>

<property name="TLS - Keystore">/opt/certs/registry/keystore.jks</property>

<property name="TLS - Keystore Password">supersecretkeystore</property>

<property name="TLS - Truststore">/opt/certs/registry/truststore.jks</property>

<property name="TLS - Truststore Password">supersecrettruststore</property>

<property name="Referral Strategy">FOLLOW</property>

<property name="Group Membership - Enforce Case Sensitivity">false</property>

<property name="User Search Base">ldap_user_search_base</property>

<property name="User Object Class">person</property>

<property name="User Search Scope">SUBTREE</property>

<property name="User Search Filter">ldap_user_search_filter_authorizers</property>

<property name="User Identity Attribute">sAMAccountName</property>

<property name="User Group Name Attribute">memberof</property>

<property name="Identity Strategy">USE_USERNAME</property>

<property name="Group Search Base">ldap_group_search_base</property>

<property name="Group Object Class">group</property>

<property name="Group Search Scope">ONE_LEVEL</property>

<property name="Group Search Filter">(objectClass=group)</property>

<property name="Group Member Attribute">member</property>

<property name="Group Member Attribute - Referenced User Attribute">sAMAccountName</property>

</userGroupProvider>

<identifier>composite-configurable-user-group-provider</identifier>

<class>org.apache.nifi.registry.security.authorization.CompositeConfigurableUserGroupProvider</class>

<property name="Configurable User Group Provider">file-user-group-provider</property>

<property name="User Group Provider 1">ldap-user-group-provider</property>

</userGroupProvider>

<identifier>file-access-policy-provider</identifier>

<class>org.apache.nifi.registry.security.authorization.file.FileAccessPolicyProvider</class>

<property name="User Group Provider">composite-configurable-user-group-provider</property>

<property name="Authorizations File">./conf/authorizations.xml</property>

<property name="Initial Admin Identity">initial_admin_identity</property>

</accessPolicyProvider>

<identifier>managed-authorizer</identifier>

<class>org.apache.nifi.registry.security.authorization.StandardManagedAuthorizer</class>

<property name="Access Policy Provider">file-access-policy-provider</property>

</authorizer>

</authorizers>

Re: No available buckets after secure Nifi connected to secure Nifi registry

MattWho — Fri, 22 Nov 2024 13:25:33 GMT

@Armel316

Since you only have two user group providers (ldap and file), that means that both are returning user "xxx". If the ldap-user-group-provider is returning user "xxx" you don't want to define that same user through the file-user-group-provider. What this means is that the users.xml file that the file-user-group-provider is loading users from on startup contains user "xxx".

The file-user-group-provider will ONLY generate a users.xml file if one does not already exist. If one already exist the file-user-group-provider will NOT make any modifications to an existing users.xml if you modify the provider configuration. Once a users.xml file exists, the expectation is that all future user/group modification happen via the UI.

NOTE: The users.xml does not contain any users or group being loaded by other providers in to NiFi memory.

So you have two options here:

Rename the current users.xml file so a new one is created on startup with only the 3 defined node-identities. (this is preferred method)
Manually modify the users.xml to remove all users that are being synced by the ldap-user-group-provider.

Thank you,
Matt