Good Morning -- any takers on helping answer this question? I would be super appreciative. ... View more

I'm hoping to add some additional clarity to this question in the form of a partial log extract of relevant information, which you will see below, followed by additional context that I will provide: Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.apache.nifi.registry.web.security.authentication.oidc.OidcService]: Constructor threw exception at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:223) at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:111) at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:315) ... 60 common frames omitted Caused by: java.lang.RuntimeException: OpenId Connect support cannot be enabled if the Login Identity Provider or Apache Knox SSO is configured. at org.apache.nifi.registry.web.security.authentication.oidc.StandardOidcIdentityProvider.validateOIDCConfiguration(StandardOidcIdentityProvider.java:200) at org.apache.nifi.registry.web.security.authentication.oidc.StandardOidcIdentityProvider.initializeProvider(StandardOidcIdentityProvider.java:120) at org.apache.nifi.registry.web.security.authentication.oidc.OidcService.<init>(OidcService.java:71) at org.apache.nifi.registry.web.security.authentication.oidc.OidcService.<init>(OidcService.java:54) at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62) at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486) at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:210) ... 62 common frames omitted 2024-03-20 12:46:17,451 INFO [main] o.e.jetty.server.handler.ContextHandler Started oeje10w.WebAppContext@9742012{nifi-registry-docs,/nifi-registry-docs,b=file:///opt/nifi-registry/nifi-registry-current/work/jetty/nifi-registry-web-docs-2.0.0-M2.war/webapp/,a=AVAILABLE,h=oeje10s.SessionHandler@60773809{STARTED}}{./lib/nifi-registry-web-docs-2.0.0-M2.war} Here is the additional context: 1. Upon initial loading of the Docker Stack, I had the Environment Variable relating to the login identity set as follows:  CN=admin 2. I spun up the docker stack. 3. I had two new files which I had set up as I indicated in the original post, above. 4. While the stack was up, I copied in the two new files: `authorizers.xml` and `nifi-registry.properties`.  5. importantly, I had the relevant volume created in a semi-persistent way.  At the bottom of the docker file, I had this: volumes: nifi_registry_conf: I also had the correlative setting up higher in the yaml, like this: volumes: - nifi_registry_conf:/opt/nifi-registry/nifi-registry-current/conf 6. I took down the docker stack. 7. I spun up the docker stack, again, confirming that the contents of the two files in question were indeed present as anticipated. 8. The above logs occurred, which may have something to do with NiFi Registry failing to redirect a request to Keycloak. IMPORTANTLY:  I have tried this various ways -- all of which produced similar messages in the logs: 1. I have spun up the docker stack without an INITIAL_ADMIN_IDENTITY at all, and repeated the aforementioned processes, with the same basic result. 2. I have spun up the docker stack with an INITIAL_ADMIN_IDENTITY, and this time, I copied out four files: `nifi-registry.properties`, `authorizers.xml`, `authorizations.xml`, and `users.xml`.  Upon doing this, I changed the values in the `users.xml` from `CN=admin` to `obfuscatedemail@random.com`. Then I brought the docker stack down and removed the semi-persistent volumes.  I then commented out the INITIAL_ADMIN_IDENTITY field, and spun up the docker stack again.  Now I copied back in these four files, and brought the docker stack down again.  Then I spun it back up again (so that it would start with the new values).  No matter -- the same basic result with the same basic logs. 3. I did a similar thing to item 2, above, but only with the `nifi-registry.properties` file and the `authorizers.xml` file.  Then I exec'd into the docker container and removed the `users.xml` and `authorizations.xml` files completely.  Then I took down the stack and spun it back up (which forces it to recreate new `users.xml` and `authorizations.xml` files. No matter -- the same basic result with the same basic logs. OBSERVATIONS:  The error message `Caused by: java.lang.RuntimeException: OpenId Connect support cannot be enabled if the Login Identity Provider or Apache Knox SSO is configured.` seems to imply or reference a setting in `nifi.properties` that does NOT exist in `nifi-registry.properties` -- the `nifi.security.user.login.identity.provider` field.  A correlative field in `nifi-registry.properties` would be, for example, `nifi.registry.security.user.login.identity.provider`.  However, as I said, this field doesn't exist.  The error message is generic in the sense that it seems to indicate potential frustration with EITHER Apache Knox SSO, or Login Identity Provider.  Since none of my setup involved Apache Knox, the assumption is that the program is upset about something pertaining to the Login Identity Provider.  However, I have no idea what it might be upset about or what it is referring to. It is of very little help that while the `Nifi System Administrator's Guide` has an entire section on OpenID Connect: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#openid_connect , the `Nifi Registry System Administrator's Guide` has no correlative section or correlative information. ... View more

You are very correct, that it works. And your efforts to simplify the process are much appreciated. It is perhaps more difficult when launching it from a Docker Compose file. I wound up making the following modifications: volumes: - /opt/nifi-custom-setup/version_2-0-0/add_nars:/opt/nifi/nifi-current/add_nars # Add additional nars - /mnt/scripts:/opt/nifi/nifi-current/custom_scripts # Add Bind mount for `custom_entrypoint.sh` Then I wrote a custom script to handle things: #!/bin/bash # Copy the custom NARs to the NiFi lib directory cp /opt/nifi/nifi-current/add_nars/*.nar /opt/nifi/nifi-current/lib/ # Call the default NiFi startup script exec /opt/nifi/nifi-current/bin/nifi.sh run In the end, I very much wish the documentation would provide some alerts to the end users on how to proceed. Again, I'm very appreciative of the assist. ... View more

Thank you for the advisement. I understand what you are saying. ... View more

I'm very appreciative of the quick reply. I guess what I don't have clarity on is how this affects the ConvertRecord processor. Are you advising that by installing these nars, that will actually cause a change to the available record readers that are present in the ConvertRecord processor? It's unclear that this is the case by following the link you referenced: https://central.sonatype.com/artifact/org.apache.nifi/nifi-parquet-nar/overview I guess I don't see anything in the documentation that references this information, and a review of the hyperlink doesn't actually provide any information about what is made available by installing the nars. I went to the trouble of downloading the nar from this link and expanding it https://repo1.maven.org/maven2/org/apache/nifi/nifi-parquet-nar/2.0.0-M1/nifi-parquet-nar-2.0.0-M1.nar I see the following: f nifi-parquet-processors-2.0.0-M1.jar created: META-INF/ inflated: META-INF/MANIFEST.MF created: META-INF/services/ created: org/ created: org/apache/ created: org/apache/nifi/ created: org/apache/nifi/parquet/ created: org/apache/nifi/parquet/hadoop/ created: org/apache/nifi/parquet/record/ created: org/apache/nifi/parquet/stream/ created: org/apache/nifi/parquet/utils/ created: org/apache/nifi/processors/ created: org/apache/nifi/processors/parquet/ created: META-INF/maven/ created: META-INF/maven/org.apache.nifi/ created: META-INF/maven/org.apache.nifi/nifi-parquet-processors/ inflated: META-INF/DEPENDENCIES inflated: META-INF/LICENSE inflated: META-INF/NOTICE inflated: META-INF/services/org.apache.nifi.controller.ControllerService inflated: META-INF/services/org.apache.nifi.processor.Processor inflated: org/apache/nifi/parquet/ParquetReader.class inflated: org/apache/nifi/parquet/ParquetRecordSetWriter.class inflated: org/apache/nifi/parquet/hadoop/AvroParquetHDFSRecordReader.class inflated: org/apache/nifi/parquet/hadoop/AvroParquetHDFSRecordWriter.class inflated: org/apache/nifi/parquet/record/ParquetRecordReader.class inflated: org/apache/nifi/parquet/record/WriteParquetResult.class inflated: org/apache/nifi/parquet/stream/NifiOutputStream.class inflated: org/apache/nifi/parquet/stream/NifiParquetInputFile.class inflated: org/apache/nifi/parquet/stream/NifiParquetOutputFile.class inflated: org/apache/nifi/parquet/stream/NifiSeekableInputStream.class inflated: org/apache/nifi/parquet/utils/ParquetConfig.class inflated: org/apache/nifi/parquet/utils/ParquetUtils.class inflated: org/apache/nifi/processors/parquet/ConvertAvroToParquet.class inflated: org/apache/nifi/processors/parquet/FetchParquet.class inflated: org/apache/nifi/processors/parquet/PutParquet.class inflated: META-INF/maven/org.apache.nifi/nifi-parquet-processors/pom.xml inflated: META-INF/maven/org.apache.nifi/nifi-parquet-processors/pom.properties It does seem like it is possible that going through all of this would allow for an additional record reader to be made available in the ConvertRecord processor, but to be honest with you, since there is no documentation on this, it still remains unclear to me that creating a custom distribution will solve the issue. ... View more

No, I never received a reply. I was able solve the problem on my own, eventually. ... View more