Hello,
I am trying to upgrade nifi from 1.8 to 1.19.1 as per the documentation, but I get the below error, and I am unable to start the nifi service. please help.
Error:
at org.apache.nifi.NiFi.main(NiFi.java:331)
Caused by: org.apache.nifi.properties.SensitivePropertyProtectionException: Sensitive Properties Key [nifi.sensitive.props.key] not found: See Admin Guide section [Updating the Sensitive Properties Key]
at org.apache.nifi.properties.NiFiPropertiesLoader.getDefaultProperties(NiFiPropertiesLoader.java:233)
at org.apache.nifi.properties.NiFiPropertiesLoader.get(NiFiPropertiesLoader.java:218)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.nifi.NiFi.initializeProperties(NiFi.java:370)
... 3 common frames omitted
Created 03-08-2023 01:15 PM
Welcome to the community @anoop89 while you wait for an expert to chime in, I thought I could share some resources in case they help.
Per documentation: Starting with version 1.14.0, NiFi requires a value for 'nifi.sensitive.props.key' in nifi.properties.
Additionally, I found a community article from @MattWho that addresses if you forgot or lost the sensitive props key.
Oh no, I forgot or lost my NiFi sensitive.props.key, what can I do?
I hope this helps get you closer to a solution.
Created 03-08-2023 08:38 PM
@cjervis Many thanks for your prompt response and help for my query. I am a novice when it comes to nifi, so please pardon my silly question.
Upon updating nifi.sensitive.props.key, I am seeing the below message in logs and nifi refuses to start.
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [java.lang.Object]: Factory method 'loginIdentityProvider' threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'loginIdentityProviderFactoryBean': FactoryBean threw exception on object creation; nested exception is java.lang.Exception: The specified login identity provider class 'com.batchiq.nifi.authentication.file.FileIdentityProvider' is not known to this nifi.
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185)
at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:653)
... 53 common frames omitted
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'loginIdentityProviderFactoryBean': FactoryBean threw exception on object creation; nested exception is java.lang.Exception: The specified login identity provider class 'com.batchiq.nifi.authentication.file.FileIdentityProvider' is not known to this nifi.
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:176)
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:101)
at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1898)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.getObjectForBeanInstance(AbstractAutowireCapableBeanFactory.java:1284)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:267)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.lambda$createCglibProxyForFactoryBean$1(ConfigurationClassEnhancer.java:540)
at org.apache.nifi.web.security.spring.LoginIdentityProviderFactoryBean$$EnhancerBySpringCGLIB$$3e2a6f6e.getObject(<generated>)
at org.apache.nifi.web.security.configuration.AuthenticationSecurityConfiguration.loginIdentityProvider(AuthenticationSecurityConfiguration.java:80)
at org.apache.nifi.web.security.configuration.AuthenticationSecurityConfiguration$$EnhancerBySpringCGLIB$$218ddc49.CGLIB$loginIdentityProvider$0(<generated>)
at org.apache.nifi.web.security.configuration.AuthenticationSecurityConfiguration$$EnhancerBySpringCGLIB$$218ddc49$$FastClassBySpringCGLIB$$b60cd7bc.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244)
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331)
at org.apache.nifi.web.security.configuration.AuthenticationSecurityConfiguration$$EnhancerBySpringCGLIB$$218ddc49.loginIdentityProvider(<generated>)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154)
... 54 common frames omitted
Caused by: java.lang.Exception: The specified login identity provider class 'com.batchiq.nifi.authentication.file.FileIdentityProvider' is not known to this nifi.
at org.apache.nifi.web.security.spring.LoginIdentityProviderFactoryBean.createLoginIdentityProvider(LoginIdentityProviderFactoryBean.java:169)
at org.apache.nifi.web.security.spring.LoginIdentityProviderFactoryBean.getObject(LoginIdentityProviderFactoryBean.java:121)
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:169)
... 72 common frames omitted
Created 03-09-2023 12:53 AM
Is NiFi is secure?
The latest error to do with how you want to setup Authentication in NiFi and I guess the current setting is not known to nifi , Please check what is set for property value in nifi properties file
nifi.security.user.login.identity.provider
It should be , ldap-provider or kerberos-provider
Pleaser refer https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#user_authentication
Thank you
Created on 03-09-2023 09:30 PM - edited 03-09-2023 09:30 PM
@ckumar Many thanks for your response.
No, the NiFi is not secure. I am trying this out in our dev environment before upgrading our Prod.
So in the new version 1.19.1 only ldap or kerberos can be used for authentication and not the file identity provider unlike the older versions?
Created on 03-09-2023 11:15 PM - edited 03-09-2023 11:24 PM
hi @anoop89 I can confirm you that version 1.19.1, 1.20.1 work very well without ldap or kerberos. I have installed two clusters, one in which there is no security active (cluster not secure) and one in which I have only activated the login with a single user and password.
But here I think it mostly depends on the version (the open-source, the Cloudera version, etc) you are ussing.
What I can tell from your logs is that you might have defined a false class for your login identify provider.
By default, when I have unzipped the NiFi ZIP File, the nifi.properties file contained the following lines:
nifi.login.identity.provider.configuration.file=./conf/login-identity-providers.xml
nifi.security.user.login.identity.provider=single-user-provider
The login-identify-providers.xml is defined as seen below, but you have two other options which are commented: LDAP(<identifier>ldap-provider</identifier>) and KERBEROS (<identifier>kerberos-provider</identifier>)
<provider>
<identifier>single-user-provider</identifier>
<class>org.apache.nifi.authentication.single.user.SingleUserLoginIdentityProvider</class>
<property name="Username"/>
<property name="Password"/>
</provider>
Maybe you are trying to use the option file-provider from within the authorizers.xml file, which comes by default as commented and it is not recognized when starting NiFi?
I think that your best solution here would be to compare the configuration files from your Dev Environment with the configuration files from your PROD Environment. By doing that you will identify where you defined the wrong property and you can correct it straight away.
Created 03-09-2023 11:42 PM
Many Thanks @cotopaul for the detailed response. I am using the same file-provider setup in both dev and Prod setups, but its still throwing error when trying to startup NiFi.
Here is the snippet from login-identity-providers.xml file:
<provider>
<identifier>file-identity-provider</identifier>
<class>com.batchiq.nifi.authentication.file.FileIdentityProvider</class>
<property name="Credentials File">conf/login-credentials.xml</property>
<property name="Authentication Expiration">12 hours</property>
</provider>
Please let me know how do sort this out to get the new NiFi version started.
Best regards
Created 03-09-2023 11:48 PM
@anoop89I never used file-identity-provider so I am not really experienced with that 😞 would it be possible to provide a short snippet from conf/login-credentials/xml? You can remove all the PII data and replace them with something dummy, but I would really like to see how the file is structured and try to reproduce the behavior on my local device. Was this file generated automatically or have you created manually and kept on using on your prod/dev instances?
PS: are you using the NiFi Cloudera version?
Created 03-10-2023 02:08 AM
Here's the snippet of the login-ceredentials.xml which is present. I have unzipped the file and manually entered the values looking at the existing nifi version:
<!--
This file contains users and their hashed passwords. Please see the
com.batchiq.nifi.authentication.file.CredentialsStore for details.
User Format:
name - must match the "identity" in authorized-users.xml
passwordHash - hashed passwords in Bcrypt 2a format / 10 rounds, looks
like "$2a$10$24wB0UAUsRbOXz4KRZ5KlenzcEddnhIyXMyPkpTnS/29Tt12jfJJW"
-->
<credentials>
<user name="aaaa" passwordHash="$$$$$"/>
<user name="aaaa" passwordHash="$$$$$"/>
<user name="aaaa" passwordHash="$$$$$"/>
<user name="aaaa" passwordHash="$$$$$"/>
<user name="aaaa" passwordHash="$$$$$"/>
</credentials>
Also, I am using Apache NiFi for the dev environment.
Created 03-10-2023 02:50 AM
I see that you are using something which is not default nor belonging to NiFi. I would suggest you have a look in your JAR Files from PROD and see if you can find something which might point to something like batchiq. Most likely the JAR file is missing from your dev environment.