Nifi upgrade error: [nifi.sensitive.props.key] not found

Welcome to the community @anoop89 while you wait for an expert to chime in, I thought I could share some resources in case they help. 

Per documentation: Starting with version 1.14.0, NiFi requires a value for 'nifi.sensitive.props.key' in nifi.properties.

Additionally, I found a community article from @MattWho that addresses if you forgot or lost the sensitive props key.  
Oh no, I forgot or lost my NiFi sensitive.props.key, what can I do? 

I hope this helps get you closer to a solution.

@cjervis Many thanks for your prompt response and help for my query. I am a novice when it comes to nifi, so please pardon my silly question. 

Upon updating nifi.sensitive.props.key, I am seeing the below message in logs and nifi refuses to start.

Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [java.lang.Object]: Factory method 'loginIdentityProvider' threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'loginIdentityProviderFactoryBean': FactoryBean threw exception on object creation; nested exception is java.lang.Exception: The specified login identity provider class 'com.batchiq.nifi.authentication.file.FileIdentityProvider' is not known to this nifi.
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185)
at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:653)
... 53 common frames omitted
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'loginIdentityProviderFactoryBean': FactoryBean threw exception on object creation; nested exception is java.lang.Exception: The specified login identity provider class 'com.batchiq.nifi.authentication.file.FileIdentityProvider' is not known to this nifi.
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:176)
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:101)
at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1898)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.getObjectForBeanInstance(AbstractAutowireCapableBeanFactory.java:1284)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:267)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.lambda$createCglibProxyForFactoryBean$1(ConfigurationClassEnhancer.java:540)
at org.apache.nifi.web.security.spring.LoginIdentityProviderFactoryBean$$EnhancerBySpringCGLIB$$3e2a6f6e.getObject(<generated>)
at org.apache.nifi.web.security.configuration.AuthenticationSecurityConfiguration.loginIdentityProvider(AuthenticationSecurityConfiguration.java:80)
at org.apache.nifi.web.security.configuration.AuthenticationSecurityConfiguration$$EnhancerBySpringCGLIB$$218ddc49.CGLIB$loginIdentityProvider$0(<generated>)
at org.apache.nifi.web.security.configuration.AuthenticationSecurityConfiguration$$EnhancerBySpringCGLIB$$218ddc49$$FastClassBySpringCGLIB$$b60cd7bc.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244)
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331)
at org.apache.nifi.web.security.configuration.AuthenticationSecurityConfiguration$$EnhancerBySpringCGLIB$$218ddc49.loginIdentityProvider(<generated>)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154)
... 54 common frames omitted
Caused by: java.lang.Exception: The specified login identity provider class 'com.batchiq.nifi.authentication.file.FileIdentityProvider' is not known to this nifi.
at org.apache.nifi.web.security.spring.LoginIdentityProviderFactoryBean.createLoginIdentityProvider(LoginIdentityProviderFactoryBean.java:169)
at org.apache.nifi.web.security.spring.LoginIdentityProviderFactoryBean.getObject(LoginIdentityProviderFactoryBean.java:121)
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:169)
... 72 common frames omitted

Is NiFi is secure?

The latest error to do with how you want to setup Authentication in NiFi and I guess the current setting is not known to nifi , Please check what is set for property value in nifi properties file 

nifi.security.user.login.identity.provider

It should be , ldap-provider or kerberos-provider 

Pleaser refer https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#user_authentication

Thank you 

@ckumar Many thanks for your response.

No, the NiFi is not secure. I am trying this out in our dev environment before upgrading our Prod.

So in the new version 1.19.1 only ldap or kerberos can be used for authentication and not the file identity provider unlike the older versions?

hi @anoop89 I can confirm you that version 1.19.1, 1.20.1 work very well without ldap or kerberos. I have installed two clusters, one in which there is no security active (cluster not secure) and one in which I have only activated the login with a single user and password.
But here I think it mostly depends on the version (the open-source, the Cloudera version, etc) you are ussing.
What I can tell from your logs is that you might have defined a false class for your login identify provider.

By default, when I have unzipped the NiFi ZIP File, the nifi.properties file contained the following lines:

nifi.login.identity.provider.configuration.file=./conf/login-identity-providers.xml
nifi.security.user.login.identity.provider=single-user-provider

The login-identify-providers.xml is defined as seen below, but you have two other options which are commented: LDAP(<identifier>ldap-provider</identifier>) and KERBEROS (<identifier>kerberos-provider</identifier>)

<provider>
<identifier>single-user-provider</identifier>
<class>org.apache.nifi.authentication.single.user.SingleUserLoginIdentityProvider</class>
<property name="Username"/>
<property name="Password"/>
</provider>

Maybe you are trying to use the option file-provider from within the authorizers.xml file, which comes by default as commented and it is not recognized when starting NiFi?

I think that your best solution here would be to compare the configuration files from your Dev Environment with the configuration files from your PROD Environment. By doing that you will identify where you defined the wrong property and you can correct it straight away.

Many Thanks @cotopaul for the detailed response. I am using the same file-provider setup in both dev and Prod setups, but its still throwing error when trying to startup NiFi.

Here is the snippet from login-identity-providers.xml file:

<provider>
<identifier>file-identity-provider</identifier>
<class>com.batchiq.nifi.authentication.file.FileIdentityProvider</class>
<property name="Credentials File">conf/login-credentials.xml</property>
<property name="Authentication Expiration">12 hours</property>
</provider>

Please let me know how do sort this out to get the new NiFi version started.

Best regards

@anoop89I never used file-identity-provider so I am not really experienced with that 😞 would it be possible to provide a short snippet from conf/login-credentials/xml? You can remove all the PII data and replace them with something dummy, but I would really like to see how the file is structured and try to reproduce the behavior on my local device. Was this file generated automatically or have you created manually and kept on using on your prod/dev instances?
PS: are you using the NiFi Cloudera version?

By default, there is no such provider name "file-identity-provider" with calsscom.batchiq.nifi.authentication.file.FileIdentityProvider. 

nifi.security.user.login.identity.provider was always ldap or kerberos but from Apache nifi 1.14 , SingleUserLoginIdentityProvider is added as by default login.identity.provider. Running NiFi with standard HTTP is basically an anonymous remote code execution platform. Single User mode is added f puts a lock on the door, which offers basic protection. 

This is applicable to both Apache NiFI and Cloudera CFM NiFi Versions.

Found it , So you are using this  https://github.com/frank-dkvan/nifi-file-identity-provider-bundle which is a custom identity provider, and that is the reason it is not working if you upgrade the cluster.

I would advise rework on you requirement related to how you would like to get Authenticated and Authorized in NiFi and I would suggest using the default ones which come from NiFi. 

Thank you 

Thank you @ckumar @MattWho for your valuable inputs. through various methods, we were able to get it working. But facing a different problem now.

Do you have any idea on what could be wrong this time?