Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to save the data flow mapping in Apache NiFI

Highlighted

How to save the data flow mapping in Apache NiFI

Explorer

Hi All,

I have two questions related to the data flow creation in Apache Nifi. Please find the queries below:

1. Once we create the data flow using processors how to save the flow and how to view the same flow once again.

2. How to deploy the code in different environment. For example : Once we are done with the development we want to move the code to production. So wanted to know how to move the code from dev to prod.

3 REPLIES 3
Highlighted

Re: How to save the data flow mapping in Apache NiFI

Hi @Ayaskant Das,

You are looking after templates:

https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#templates

Be aware that some component properties are sensitive, like passwords, and thus are not included in the templates. So you'll have to reenter them when you apply the template in the new environment. Second, there are at times properties that you'd want to have different values for in different environments. In that case, you may want to look at the variable registry to externalize such properties and take advantage of the expression language:

https://community.hortonworks.com/articles/57304/supporting-custom-properties-for-expression-langua....

Hope this helps.

Highlighted

Re: How to save the data flow mapping in Apache NiFI

Master Guru

@Ayaskant Das

I agree 100% with everything Pierre stated.

I just want to add that every change you make on the NiFi canvas is immediately persisted in the flow.xml.gz file.

If the goal is to move the entire flow and not just a portion of it, you can simply copy the flow.xml.gz from one NiFi to another. There is a caveats to that:

The flow.xml.gz file will contain the entire flow including the encrypted sensitive property values. When moving the flow.xml.gz to another NiFi, that target NiFi must also be using the same configured sensitive props key as source NiFi in the nifi.properties file or it will not be able to decrypt the sensitive properties from the flow.xml.gz you provided.

Thanks,

Matt

Re: How to save the data flow mapping in Apache NiFI

Explorer

Just to add some color: Matt and Pierre are 100% accurate regarding the current capability around flow deployment mechanism in NIFI. Moving forward, version control plays an important role in the entire flow deployment/migration scenario, especially in an enterprise environment. It is going to be a focal point for the community in the near future. For more details, you may want to read through this feature proposal: https://cwiki.apache.org/confluence/display/NIFI/Configuration+Management+of+Flows

Don't have an account?
Coming from Hortonworks? Activate your account here