Support Questions

Find answers, ask questions, and share your expertise
Announcements
Welcome to the upgraded Community! Read this blog to see What’s New!

Nifi upgrade error: [nifi.sensitive.props.key] not found

avatar
Explorer

Hello,

I am trying to upgrade nifi from 1.8 to 1.19.1 as per the documentation, but I get the below error, and I am unable to start the nifi service. please help.

 

Error:

at org.apache.nifi.NiFi.main(NiFi.java:331)
Caused by: org.apache.nifi.properties.SensitivePropertyProtectionException: Sensitive Properties Key [nifi.sensitive.props.key] not found: See Admin Guide section [Updating the Sensitive Properties Key]
at org.apache.nifi.properties.NiFiPropertiesLoader.getDefaultProperties(NiFiPropertiesLoader.java:233)
at org.apache.nifi.properties.NiFiPropertiesLoader.get(NiFiPropertiesLoader.java:218)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.nifi.NiFi.initializeProperties(NiFi.java:370)
... 3 common frames omitted

13 REPLIES 13

avatar
Community Manager

Welcome to the community @anoop89 while you wait for an expert to chime in, I thought I could share some resources in case they help. 

 

Per documentation: Starting with version 1.14.0, NiFi requires a value for 'nifi.sensitive.props.key' in nifi.properties.

 

Additionally, I found a community article from @MattWho that addresses if you forgot or lost the sensitive props key.  
Oh no, I forgot or lost my NiFi sensitive.props.key, what can I do? 

 

I hope this helps get you closer to a solution.


Cy Jervis, Manager, Community Program
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

avatar
Explorer

@cjervis Many thanks for your prompt response and help for my query. I am a novice when it comes to nifi, so please pardon my silly question. 

 

Upon updating nifi.sensitive.props.key, I am seeing the below message in logs and nifi refuses to start.

 

Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [java.lang.Object]: Factory method 'loginIdentityProvider' threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'loginIdentityProviderFactoryBean': FactoryBean threw exception on object creation; nested exception is java.lang.Exception: The specified login identity provider class 'com.batchiq.nifi.authentication.file.FileIdentityProvider' is not known to this nifi.
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185)
at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:653)
... 53 common frames omitted
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'loginIdentityProviderFactoryBean': FactoryBean threw exception on object creation; nested exception is java.lang.Exception: The specified login identity provider class 'com.batchiq.nifi.authentication.file.FileIdentityProvider' is not known to this nifi.
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:176)
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:101)
at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1898)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.getObjectForBeanInstance(AbstractAutowireCapableBeanFactory.java:1284)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:267)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.lambda$createCglibProxyForFactoryBean$1(ConfigurationClassEnhancer.java:540)
at org.apache.nifi.web.security.spring.LoginIdentityProviderFactoryBean$$EnhancerBySpringCGLIB$$3e2a6f6e.getObject(<generated>)
at org.apache.nifi.web.security.configuration.AuthenticationSecurityConfiguration.loginIdentityProvider(AuthenticationSecurityConfiguration.java:80)
at org.apache.nifi.web.security.configuration.AuthenticationSecurityConfiguration$$EnhancerBySpringCGLIB$$218ddc49.CGLIB$loginIdentityProvider$0(<generated>)
at org.apache.nifi.web.security.configuration.AuthenticationSecurityConfiguration$$EnhancerBySpringCGLIB$$218ddc49$$FastClassBySpringCGLIB$$b60cd7bc.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244)
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331)
at org.apache.nifi.web.security.configuration.AuthenticationSecurityConfiguration$$EnhancerBySpringCGLIB$$218ddc49.loginIdentityProvider(<generated>)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154)
... 54 common frames omitted
Caused by: java.lang.Exception: The specified login identity provider class 'com.batchiq.nifi.authentication.file.FileIdentityProvider' is not known to this nifi.
at org.apache.nifi.web.security.spring.LoginIdentityProviderFactoryBean.createLoginIdentityProvider(LoginIdentityProviderFactoryBean.java:169)
at org.apache.nifi.web.security.spring.LoginIdentityProviderFactoryBean.getObject(LoginIdentityProviderFactoryBean.java:121)
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:169)
... 72 common frames omitted

avatar
Expert Contributor

Is NiFi is secure?

The latest error to do with how you want to setup Authentication in NiFi and I guess the current setting is not known to nifi , Please check what is set for property value in nifi properties file 

nifi.security.user.login.identity.provider 

It should be , ldap-provider or kerberos-provider 

 

Pleaser refer https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#user_authentication

 

Thank you 

avatar
Explorer

@ckumar Many thanks for your response.

No, the NiFi is not secure. I am trying this out in our dev environment before upgrading our Prod.

 

So in the new version 1.19.1 only ldap or kerberos can be used for authentication and not the file identity provider unlike the older versions?

avatar
Super Collaborator

hi @anoop89 I can confirm you that version 1.19.1, 1.20.1 work very well without ldap or kerberos. I have installed two clusters, one in which there is no security active (cluster not secure) and one in which I have only activated the login with a single user and password.
But here I think it mostly depends on the version (the open-source, the Cloudera version, etc) you are ussing.
What I can tell from your logs is that you might have defined a false class for your login identify provider.

By default, when I have unzipped the NiFi ZIP File, the nifi.properties file contained the following lines:

 

 

nifi.login.identity.provider.configuration.file=./conf/login-identity-providers.xml
nifi.security.user.login.identity.provider=single-user-provider

 

 



The login-identify-providers.xml is defined as seen below, but you have two other options which are commented: LDAP(<identifier>ldap-provider</identifier>) and KERBEROS (<identifier>kerberos-provider</identifier>)

 

 

<provider>
<identifier>single-user-provider</identifier>
<class>org.apache.nifi.authentication.single.user.SingleUserLoginIdentityProvider</class>
<property name="Username"/>
<property name="Password"/>
</provider>

 

 



Maybe you are trying to use the option file-provider from within the authorizers.xml file, which comes by default as commented and it is not recognized when starting NiFi?

I think that your best solution here would be to compare the configuration files from your Dev Environment with the configuration files from your PROD Environment. By doing that you will identify where you defined the wrong property and you can correct it straight away.

avatar
Explorer

Many Thanks @cotopaul for the detailed response. I am using the same file-provider setup in both dev and Prod setups, but its still throwing error when trying to startup NiFi.

 

Here is the snippet from login-identity-providers.xml file:

 

<provider>
<identifier>file-identity-provider</identifier>
<class>com.batchiq.nifi.authentication.file.FileIdentityProvider</class>
<property name="Credentials File">conf/login-credentials.xml</property>
<property name="Authentication Expiration">12 hours</property>
</provider>

 

Please let me know how do sort this out to get the new NiFi version started.

 

 

Best regards

avatar
Super Collaborator

@anoop89I never used file-identity-provider so I am not really experienced with that 😞 would it be possible to provide a short snippet from conf/login-credentials/xml? You can remove all the PII data and replace them with something dummy, but I would really like to see how the file is structured and try to reproduce the behavior on my local device. Was this file generated automatically or have you created manually and kept on using on your prod/dev instances?
PS: are you using the NiFi Cloudera version?

avatar
Explorer

Here's the snippet of the login-ceredentials.xml which is present. I have unzipped the file and manually entered the values looking at the existing nifi version:

 

<!--
This file contains users and their hashed passwords. Please see the
com.batchiq.nifi.authentication.file.CredentialsStore for details.

User Format:
name - must match the "identity" in authorized-users.xml
passwordHash - hashed passwords in Bcrypt 2a format / 10 rounds, looks
like "$2a$10$24wB0UAUsRbOXz4KRZ5KlenzcEddnhIyXMyPkpTnS/29Tt12jfJJW"
-->
<credentials>

<user name="aaaa" passwordHash="$$$$$"/>
<user name="aaaa" passwordHash="$$$$$"/>
<user name="aaaa" passwordHash="$$$$$"/>
<user name="aaaa" passwordHash="$$$$$"/>
<user name="aaaa" passwordHash="$$$$$"/>
</credentials>

 

Also, I am using Apache NiFi for the dev environment.

avatar
Super Collaborator

I see that you are using something which is not default nor belonging to NiFi. I would suggest you have a look in your JAR Files from PROD and see if you can find something which might point to something like batchiq. Most likely the JAR file is missing from your dev environment.

avatar
Expert Contributor

By default, there is no such provider name "file-identity-provider" with calsscom.batchiq.nifi.authentication.file.FileIdentityProvider. 

nifi.security.user.login.identity.provider was always ldap or kerberos but from Apache nifi 1.14 , SingleUserLoginIdentityProvider is added as by default login.identity.provider. Running NiFi with standard HTTP is basically an anonymous remote code execution platform. Single User mode is added f puts a lock on the door, which offers basic protection. 

This is applicable to both Apache NiFI and Cloudera CFM NiFi Versions.

 

 

 

 

avatar
Expert Contributor

Found it , So you are using this  https://github.com/frank-dkvan/nifi-file-identity-provider-bundle which is a custom identity provider, and that is the reason it is not working if you upgrade the cluster.

I would advise rework on you requirement related to how you would like to get Authenticated and Authorized in NiFi and I would suggest using the default ones which come from NiFi. 

Thank you 

 

 

avatar
Explorer

Thank you @ckumar @MattWho for your valuable inputs. through various methods, we were able to get it working. But facing a different problem now.

Do you have any idea on what could be wrong this time?

 

anoop89_0-1679033070372.png

 

avatar
Mentor

@anoop89 
This is an unrelated issue to this original thread. Please start a new question.  Fell free to @ckumar and @MattWho  in your question so we get notified.  This issue is related to authorization of your user.

Thanks,

Matt

Labels