question Re: Nifi upgrade error: [nifi.sensitive.props.key] not found in Support Questions

Nifi upgrade error: [nifi.sensitive.props.key] not found

anoop89 — Wed, 08 Mar 2023 21:04:22 GMT

Hello,

I am trying to upgrade nifi from 1.8 to 1.19.1 as per the documentation, but I get the below error, and I am unable to start the nifi service. please help.

Error:

at org.apache.nifi.NiFi.main(NiFi.java:331)
Caused by: org.apache.nifi.properties.SensitivePropertyProtectionException: Sensitive Properties Key [nifi.sensitive.props.key] not found: See Admin Guide section [Updating the Sensitive Properties Key]
at org.apache.nifi.properties.NiFiPropertiesLoader.getDefaultProperties(NiFiPropertiesLoader.java:233)
at org.apache.nifi.properties.NiFiPropertiesLoader.get(NiFiPropertiesLoader.java:218)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.nifi.NiFi.initializeProperties(NiFi.java:370)
... 3 common frames omitted

Re: Nifi upgrade error: [nifi.sensitive.props.key] not found

cjervis — Wed, 08 Mar 2023 21:15:03 GMT

Welcome to the community @anoop89 while you wait for an expert to chime in, I thought I could share some resources in case they help.

Per documentation: Starting with version 1.14.0, NiFi requires a value for 'nifi.sensitive.props.key' in nifi.properties.

Additionally, I found a community article from @MattWho that addresses if you forgot or lost the sensitive props key.

I hope this helps get you closer to a solution.

Re: Nifi upgrade error: [nifi.sensitive.props.key] not found

anoop89 — Thu, 09 Mar 2023 04:38:57 GMT

@cjervis Many thanks for your prompt response and help for my query. I am a novice when it comes to nifi, so please pardon my silly question.

Upon updating nifi.sensitive.props.key, I am seeing the below message in logs and nifi refuses to start.

Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [java.lang.Object]: Factory method 'loginIdentityProvider' threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'loginIdentityProviderFactoryBean': FactoryBean threw exception on object creation; nested exception is java.lang.Exception: The specified login identity provider class 'com.batchiq.nifi.authentication.file.FileIdentityProvider' is not known to this nifi.
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185)
at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:653)
... 53 common frames omitted
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'loginIdentityProviderFactoryBean': FactoryBean threw exception on object creation; nested exception is java.lang.Exception: The specified login identity provider class 'com.batchiq.nifi.authentication.file.FileIdentityProvider' is not known to this nifi.
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:176)
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:101)
at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1898)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.getObjectForBeanInstance(AbstractAutowireCapableBeanFactory.java:1284)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:267)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.lambda$createCglibProxyForFactoryBean$1(ConfigurationClassEnhancer.java:540)
at org.apache.nifi.web.security.spring.LoginIdentityProviderFactoryBean$$EnhancerBySpringCGLIB$$3e2a6f6e.getObject(<generated>)
at org.apache.nifi.web.security.configuration.AuthenticationSecurityConfiguration.loginIdentityProvider(AuthenticationSecurityConfiguration.java:80)
at org.apache.nifi.web.security.configuration.AuthenticationSecurityConfiguration$$EnhancerBySpringCGLIB$$218ddc49.CGLIB$loginIdentityProvider$0(<generated>)
at org.apache.nifi.web.security.configuration.AuthenticationSecurityConfiguration$$EnhancerBySpringCGLIB$$218ddc49$$FastClassBySpringCGLIB$$b60cd7bc.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244)
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331)
at org.apache.nifi.web.security.configuration.AuthenticationSecurityConfiguration$$EnhancerBySpringCGLIB$$218ddc49.loginIdentityProvider(<generated>)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154)
... 54 common frames omitted
Caused by: java.lang.Exception: The specified login identity provider class 'com.batchiq.nifi.authentication.file.FileIdentityProvider' is not known to this nifi.
at org.apache.nifi.web.security.spring.LoginIdentityProviderFactoryBean.createLoginIdentityProvider(LoginIdentityProviderFactoryBean.java:169)
at org.apache.nifi.web.security.spring.LoginIdentityProviderFactoryBean.getObject(LoginIdentityProviderFactoryBean.java:121)
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:169)
... 72 common frames omitted

Re: Nifi upgrade error: [nifi.sensitive.props.key] not found

ckumar — Thu, 09 Mar 2023 08:53:22 GMT

Is NiFi is secure?

The latest error to do with how you want to setup Authentication in NiFi and I guess the current setting is not known to nifi , Please check what is set for property value in nifi properties file

nifi.security.user.login.identity.provider

It should be , ldap-provider or kerberos-provider

Pleaser refer https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#user_authentication

Thank you

Re: Nifi upgrade error: [nifi.sensitive.props.key] not found

anoop89 — Fri, 10 Mar 2023 05:30:59 GMT

@ckumar Many thanks for your response.

No, the NiFi is not secure. I am trying this out in our dev environment before upgrading our Prod.

So in the new version 1.19.1 only ldap or kerberos can be used for authentication and not the file identity provider unlike the older versions?

Re: Nifi upgrade error: [nifi.sensitive.props.key] not found

cotopaul — Fri, 10 Mar 2023 07:24:49 GMT

hi @anoop89 I can confirm you that version 1.19.1, 1.20.1 work very well without ldap or kerberos. I have installed two clusters, one in which there is no security active (cluster not secure) and one in which I have only activated the login with a single user and password.
But here I think it mostly depends on the version (the open-source, the Cloudera version, etc) you are ussing.
What I can tell from your logs is that you might have defined a false class for your login identify provider.

By default, when I have unzipped the NiFi ZIP File, the nifi.properties file contained the following lines:

nifi.login.identity.provider.configuration.file=./conf/login-identity-providers.xml nifi.security.user.login.identity.provider=single-user-provider

The login-identify-providers.xml is defined as seen below, but you have two other options which are commented: LDAP(<identifier>ldap-provider</identifier>) and KERBEROS (<identifier>kerberos-provider</identifier>)

<provider> <identifier>single-user-provider</identifier> <class>org.apache.nifi.authentication.single.user.SingleUserLoginIdentityProvider</class> <property name="Username"/> <property name="Password"/> </provider>

Maybe you are trying to use the option file-provider from within the authorizers.xml file, which comes by default as commented and it is not recognized when starting NiFi?

I think that your best solution here would be to compare the configuration files from your Dev Environment with the configuration files from your PROD Environment. By doing that you will identify where you defined the wrong property and you can correct it straight away.

Re: Nifi upgrade error: [nifi.sensitive.props.key] not found

anoop89 — Fri, 10 Mar 2023 07:42:26 GMT

Many Thanks @cotopaul for the detailed response. I am using the same file-provider setup in both dev and Prod setups, but its still throwing error when trying to startup NiFi.

Here is the snippet from login-identity-providers.xml file:

<provider>
<identifier>file-identity-provider</identifier>
<class>com.batchiq.nifi.authentication.file.FileIdentityProvider</class>
<property name="Credentials File">conf/login-credentials.xml</property>
<property name="Authentication Expiration">12 hours</property>
</provider>

Please let me know how do sort this out to get the new NiFi version started.

Best regards

Re: Nifi upgrade error: [nifi.sensitive.props.key] not found

cotopaul — Fri, 10 Mar 2023 07:48:58 GMT

@anoop89I never used file-identity-provider so I am not really experienced with that 😞 would it be possible to provide a short snippet from conf/login-credentials/xml? You can remove all the PII data and replace them with something dummy, but I would really like to see how the file is structured and try to reproduce the behavior on my local device. Was this file generated automatically or have you created manually and kept on using on your prod/dev instances?
PS: are you using the NiFi Cloudera version?

Re: Nifi upgrade error: [nifi.sensitive.props.key] not found

ckumar — Fri, 10 Mar 2023 08:38:48 GMT

By default, there is no such provider name "file-identity-provider" with calsscom.batchiq.nifi.authentication.file.FileIdentityProvider.

nifi.security.user.login.identity.provider was always ldap or kerberos but from Apache nifi 1.14 , SingleUserLoginIdentityProvider is added as by default login.identity.provider. Running NiFi with standard HTTP is basically an anonymous remote code execution platform. Single User mode is added f puts a lock on the door, which offers basic protection.

This is applicable to both Apache NiFI and Cloudera CFM NiFi Versions.

Re: Nifi upgrade error: [nifi.sensitive.props.key] not found

anoop89 — Fri, 10 Mar 2023 10:08:14 GMT

Here's the snippet of the login-ceredentials.xml which is present. I have unzipped the file and manually entered the values looking at the existing nifi version:

<!--
This file contains users and their hashed passwords. Please see the
com.batchiq.nifi.authentication.file.CredentialsStore for details.

User Format:
name - must match the "identity" in authorized-users.xml
passwordHash - hashed passwords in Bcrypt 2a format / 10 rounds, looks
like "$2a$10$24wB0UAUsRbOXz4KRZ5KlenzcEddnhIyXMyPkpTnS/29Tt12jfJJW"
-->
<credentials>

<user name="aaaa" passwordHash="$$$$$"/>
<user name="aaaa" passwordHash="$$$$$"/>
<user name="aaaa" passwordHash="$$$$$"/>
<user name="aaaa" passwordHash="$$$$$"/>
<user name="aaaa" passwordHash="$$$$$"/>
</credentials>

Also, I am using Apache NiFi for the dev environment.

Re: Nifi upgrade error: [nifi.sensitive.props.key] not found

cotopaul — Fri, 10 Mar 2023 10:50:22 GMT

I see that you are using something which is not default nor belonging to NiFi. I would suggest you have a look in your JAR Files from PROD and see if you can find something which might point to something like batchiq. Most likely the JAR file is missing from your dev environment.

Re: Nifi upgrade error: [nifi.sensitive.props.key] not found

ckumar — Fri, 10 Mar 2023 14:00:34 GMT

Found it , So you are using this https://github.com/frank-dkvan/nifi-file-identity-provider-bundle which is a custom identity provider, and that is the reason it is not working if you upgrade the cluster.

I would advise rework on you requirement related to how you would like to get Authenticated and Authorized in NiFi and I would suggest using the default ones which come from NiFi.

Thank you

Re: Nifi upgrade error: [nifi.sensitive.props.key] not found

anoop89 — Fri, 17 Mar 2023 06:05:21 GMT

Thank you @ckumar @MattWho for your valuable inputs. through various methods, we were able to get it working. But facing a different problem now.

Do you have any idea on what could be wrong this time?

Re: Nifi upgrade error: [nifi.sensitive.props.key] not found

MattWho — Fri, 17 Mar 2023 14:30:57 GMT

@anoop89
This is an unrelated issue to this original thread. Please start a new question. Fell free to @ckumar and @MattWho in your question so we get notified. This issue is related to authorization of your user.

Thanks,

Matt