Error importing NiFi workflow template from version 1.x to 2.x

[#1]

Error importing NiFi workflow template from version 1.x to 2.x

I'm having trouble migrating my NiFi workflow from version 1.9.2 to version 2.0 using the export template and import template functions. Any assistance would be greatly appreciated.

[#2 · Accepted solution]

@Bern 

NiFi templates were deprecated in Apache NiFi 1.x and removed completely in Apache NiFi 2.x major release.

Around Apache NiFi 1.12 the xml format was deprecated, but remained as a part or NiFi 1.x major release versions.  This impacted for both templates and the flow.xml.gz file that held everything you see on the canvas.  NiFi introduced Flow Definitions (template replacement which is in json format) and the flow.json.gz  to replace the flow.xml.gz for persisting the entire NiFi canvas's dataflows.  When you upgrade to a version of NiFi greater then 1.12, NiFi will still load from a flow.xml.gz file and will then convert and write back out the newer flow.json.gz format.

Flow Definitions can be created by right clicking in a process group on the NiFi canvas and selecting option to download a flow definition for that specific process group.

The screenshot you shared show you attempting to import a flow definition in Apache NiFi 2 but providing a NiFi template instead of a Flow Definition.  That is why you encountered the exception about it not being a valid json.

Unfortunately, Apache NiFi does not have utility for converting NiFi templates into Flow Definitions.  So you will need to have a NiFi 1.13+ version of Apache NiFi where you can still load your templates and then download them as flow definitions )some templates will require manual modification before they can be downloaded as a flow definition.  There are numerous breaking changes in Apache NiFi 2.x major release so I suggest reading through the Apache NiFi migration guide to understand what those changes are any the manual actions you may need to make to your NiFi 1.x release flows before they can be used in NiFi 2.x.
https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance

NOTE:  Cloudera Flow Management (CFM) licensed users have access to Cloudera specific automation tools that can auto transform templates into valid flow definitions and automated migration of CFM 2.1.7 SP2 (Apache NiFi 1.x based) flow.json.gz files into CFM 4.1x (Apache NiFi 2.x based) compatible version. This automation handles deprecated components, converting NiFi variables (deprecated) into NiFi parameters (replacement), etc.
https://docs.cloudera.com/cfm/4.11.0/cfm-migration-tool/topics/cfm-mt-overview.html#concept_wlv_sl3_...

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

[#3 · Accepted solution]

@Bern 

NiFi templates were deprecated in Apache NiFi 1.x and removed completely in Apache NiFi 2.x major release.

Around Apache NiFi 1.12 the xml format was deprecated, but remained as a part or NiFi 1.x major release versions.  This impacted for both templates and the flow.xml.gz file that held everything you see on the canvas.  NiFi introduced Flow Definitions (template replacement which is in json format) and the flow.json.gz  to replace the flow.xml.gz for persisting the entire NiFi canvas's dataflows.  When you upgrade to a version of NiFi greater then 1.12, NiFi will still load from a flow.xml.gz file and will then convert and write back out the newer flow.json.gz format.

Flow Definitions can be created by right clicking in a process group on the NiFi canvas and selecting option to download a flow definition for that specific process group.

The screenshot you shared show you attempting to import a flow definition in Apache NiFi 2 but providing a NiFi template instead of a Flow Definition.  That is why you encountered the exception about it not being a valid json.

Unfortunately, Apache NiFi does not have utility for converting NiFi templates into Flow Definitions.  So you will need to have a NiFi 1.13+ version of Apache NiFi where you can still load your templates and then download them as flow definitions )some templates will require manual modification before they can be downloaded as a flow definition.  There are numerous breaking changes in Apache NiFi 2.x major release so I suggest reading through the Apache NiFi migration guide to understand what those changes are any the manual actions you may need to make to your NiFi 1.x release flows before they can be used in NiFi 2.x.
https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance

NOTE:  Cloudera Flow Management (CFM) licensed users have access to Cloudera specific automation tools that can auto transform templates into valid flow definitions and automated migration of CFM 2.1.7 SP2 (Apache NiFi 1.x based) flow.json.gz files into CFM 4.1x (Apache NiFi 2.x based) compatible version. This automation handles deprecated components, converting NiFi variables (deprecated) into NiFi parameters (replacement), etc.
https://docs.cloudera.com/cfm/4.11.0/cfm-migration-tool/topics/cfm-mt-overview.html#concept_wlv_sl3_...

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

[#4]

thanks very much, I've already done the first part: I imported my flow templates to version 1.13 and exported the JSON, but I'm not clear on how to load it into version 2.0.

[#5 · Accepted solution]

@Bern 
You just need to follow same steps you used in your original post question.  

You drag the add process group icon to the canvas.:

In the pop-up window select the "browse" icon to far right instead of entering a "Name":

Navigate to and select your downloaded flow definition you created in Apache NiFi 1.13 (I would strongly encourage you to be using Apache NiFi 1.28). There have been many changes a fixes and you'll want to make sure your Apache NiFi 1.x dataflows are valid on the latest 1.28 before attempting to move them to Apache NiFi 2.x).  Apache NiFi 1.28 will also have deprecation logging that will help make user aware if they are using components that no longer exist in Apache NiFi 2.x.  You'll need to make modification to your flows so you are no longer using those components before moving your flow definitions over to NiFi 2.x.   Also be aware that NiFi 2.x no longer support NiFi Variables. These were deprecated and removed.  The replacement is NiFi Parameters.  So if you are using Variables in your NiFi 1 dataflows, you' need to modify your dataflows to use paramters before moving your flow definitons over to NiFi 2.

After you have selected your flow definition json file, you have option to change name that is displays from json and click "Add":

Remember that Apache NiFi 2.x is a major release change and the expectation is that you are on the latest NiFi 1.28 release before attempting to move to NiFi 2. 

NOTE:  Cloudera Flow Management (CFM) licensed users have access to Cloudera specific automation tools that can auto transform templates into valid flow definitions and automated migration of CFM 2.1.7 SP2 (Apache NiFi 1.x based) flow.json.gz files into CFM 4.1x (Apache NiFi 2.x based) compatible version. This automation handles deprecated components, converting NiFi variables (deprecated) into NiFi parameters (replacement), etc.
https://docs.cloudera.com/cfm/4.11.0/cfm-migration-tool/topics/cfm-mt-overview.html#concept_wlv_sl3_...

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt