Upgrading from 1.13 to 1.19 and fallback

[#1]

Hi there, any support would be much appreciated. First things first, here is our setup. We are running nifi on aws ec2 in a docker container. It's a two node cluster setup. We are planning to do an in place upgrade by upgrading the docker container and changing the nifi.properties parameters needed to get 1.19 running.

• changing the port number to 8443
• using https instead of http
• setting the sensitive properties, was empty in 1.13
• we are using the web.proxy.host with 3 entries, the hostname of the 2 nodes and the clustername
• we are running anonymously, without ldap. We'll set this up later.

We made a backup of the flow.xml.gz file and deleted the original. We are planning to use the nifi-registry to import our flow on the clean canvas.

So this works great! We have succesfully upgraded to 1.19. Now what we're trying to figure out is our fallback plan, just in case upgrading doesn't work in the production environment.

When reverting all the changes above and deleting the flow.xml.gz, we restart nifi. The 1.13 image is loaded, but keep on getting this error in our logs, which results in not getting nifi started.

{"@timestamp":"2023-04-03T09:46:29.153Z","@version":"1","message":"Failed to start web server: Error creating bean with name 'org.springframework.security.config.annotation.web.configuration.WebSecurityConfiguration': Unsatisfied dependency expressed through method 'setFilterChainProxySecurityConfigurer' parameter 1; nested exception is org.springframework.beans.factory.BeanExpressionException: Expression parsing failed; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'org.apache.nifi.web.NiFiWebApiSecurityConfiguration': Unsatisfied dependency expressed through method 'setJwtAuthenticationProvider' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jwtAuthenticationProvider' defined in class path resource [nifi-web-security-context.xml]: Cannot resolve reference to bean 'idpUserGroupService' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'idpUserGroupService' defined in class path resource [nifi-administration-context.xml]: Cannot resolve reference to bean 'idpTransactionBuilder' while setting bean property 'transactionBuilder'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'idpTransactionBuilder' defined in class path resource [nifi-administration-context.xml]: Cannot resolve reference to bean 'idpDataSource' while setting bean property 'dataSource'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'idpDataSource': FactoryBean threw exception on object creation; nested exception is org.h2.jdbc.JdbcSQLNonTransientConnectionException: File corrupted while reading record: null. Possible solution: use the recovery tool [90030-199]","logger_name":"org.apache.nifi.StdErr","thread_name":"NiFi logging handler","level":"ERROR","level_value":40000,"role":"nifi","hostname":"i-0406687dc7f729e5a.xxx.xxx.prod.xxx.xxx"}

Looks like the messages has something to do with authentication, which we weren't using in 1.13.

Any Idea's? 

[#2 · Accepted solution]

@Dave0x1 
Apache NiFi takes carful consideration when upgrading/migrating from an older version to a newer version with in reason.  Like upgrading from 1.13 to 1.19 would be expected to pose no real issues; however, jumping from 1.10 directly to 1.19 may pose some challenges.

That being said, rolling back is not something that can be guaranteed to go smoothly as some versions can result in changes to database structure/content, modification/change to NiFi components (processor, controller services, reporting tasks, etc).  In a best case scenario, a rollback my result in a working NiFi, with unexpected unexpected invalid components on the NiFi canvas.  This happens when new properties are introduced in  components being used, Those component changes get written to the flow.xml.gz and new flow.josn.gz files.  So on rollback those new properties turn in to dynamic properties which may result in an invalid component until they are removed manually.  In other scenarios, there may have been upgrades/changes to H2 DB that won't rollback smoothly.    So perhaps taking a backup of the flow.xml.gz. flow.json.gz, and database_repository/ directory contents would be useful in the vent of rollback. 

Good news is that there is nothing in the H2 DBs located in the database_repository you need to retain.
The nifi-flow-audit DB tracks history of changes you made on the canvas.  The other user and identity dbs can just be removed and allowed to be recreated on next startup.

So in your rollback test, I would rename the existing DB files and restart allowing new to be created.

If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.

Thank you,

Matt

[#3]

@Dave0x1 have you been able to resolved your issue. If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.