Created on 08-09-2018 04:10 PM - edited 08-17-2019 06:44 AM
With the release of NiFi Registry 0.2.0, flow contents can now be stored under a Git directory using the new
This tutorial walks you through how to configure this provider in NiFi Registry so that versioned flows in NiFi are automatically saved to a Git repository.
A video version of this tutorial can be seen here: https://youtu.be/kK7eVppg9Aw
This tutorial was tested using the following environment and components:
First, create a new GitHub repo:
then clone it locally using the
git clone command (e.g.
git clone https://github.com/andrewmlim/versioned_flows.git):
Next, go to GitHub’s “Developer settings” and create a new “Personal access token”:
In the ./conf/providers.xml file, configure the following properties:
org.apache.nifi.registry.provider.flow.git.GitFlowPersistenceProvideras the qualified class name
Here is an example of these changes in providers.xml:
<flowPersistenceProvider> <class>org.apache.nifi.registry.provider.flow.git.GitFlowPersistenceProvider</class> <property name="Flow Storage Directory">./versioned_flows</property> <property name="Remote To Push">origin</property> <property name="Remote Access User">andrewmlim</property> <property name="Remote Access Password">f1295e16f933d4468d948ea276372da8b0585bda</property> </flowPersistenceProvider>
Note: The “Remote To Push” property specifies the name of the remote to automatically push to. This property is optional and if not specified, commits will remain in the local repository unless a push is performed manually.
Start up NiFi Registry and create a bucket:
Start up a NiFi instance and connect to the Registry:
Create a process group. Start version control:
Save the flow:
In GitHub, you will see that the Bucket and Flow have been saved in your repo:
As shown, Buckets are represented as directories and Flow contents are stored as files in the Bucket directory they belong to. Flow snapshot histories are managed as Git commits, meaning only the latest version of Buckets and Flows exist in the Git directory.
Note: The commit message states "By NiFi Registry user: anonymous" since the environment was unsecured and there was no user logged into NiFi. The commit message would have the user's identity if secure.
Here are some helpful links that were used as references for this article:
Created on 04-30-2021 07:17 AM - edited 04-30-2021 08:30 AM
Cannot do similar with docker images apache/nifi-registry:latest, because
Do you have any suggestions?
Created on 06-08-2021 02:40 PM - edited 06-08-2021 02:46 PM
Created on 06-08-2021 02:43 PM
@thuylevn I have customized the image from base image by adding the updated providers.xml file to image.
this is my Dockerfile content,
Created on 06-14-2021 08:25 AM - edited 06-14-2021 09:20 AM
@alim Thanks for the details for git integration, its working fine. However its always pushing the changes to default branch (master) configured on git. Is there a way to specify the branch to clone and push always irrespective of main branch. We are running registry as a k8s container so it gets restarted some times and its pulling the default branch always. I have tried below property to set branch name "origin/develop" it complains and says only "origin" found in remote Please share your thoughts on this appreciate anyones help on this.
<property name="Flow Storage Directory">./versioned_flows</property>
<property name="Remote To Push">origin/develop</property>
<property name="Remote Access User">my-gituserid</property>
<property name="Remote Access Password">my-token</property>
<property name="Remote Clone Repository">https://github.com/myrepo-name.git</property>
Created on 07-08-2021 10:51 PM
- Login to git in browser and create branch called 'develop'.
- Login to server(node) where registry is running, got to '/versioned_flows' directory
- Clone git repo with newly created specific branch
>git clone -b develop https://github.com/myrepo-name.git
Thats it. you should see your templates committing to 'develop' branch instead of default master branch.
Created on 07-28-2021 02:17 AM
Hello @alim ,
Thanks for the detailed explanation.
Do you know any solution for this problem -
Created on 12-17-2021 06:18 AM
Can you please indicate where I can find the ./conf/providers.xml file ?
Also regarding this "Flow snapshot histories are managed as Git commits, meaning only the latest version of Buckets and Flows exist in the Git directory.", is there any way to save all versions of a specific flow in GIT?