Support Questions
Find answers, ask questions, and share your expertise

NiFi-Registry - GitHub connection - Not possible to take over already existing buckets/flows ?

NiFi-Registry - GitHub connection - Not possible to take over already existing buckets/flows ?

Expert Contributor

Hi all,

I tried to connect NiFi-Registry 0.3.0 with github following this description https://community.hortonworks.com/articles/210286/storing-apache-nifi-versioned-flows-in-a-git-repos...

It works as far so I can create a new bucket and start a new version control and all this appears in github-repository.

BUT isn't it possible to get already existing buckets and versioned flows into the github-repository?

Nothing of this was pushed to github so I turned the providers.xml back to before

Do I get this right that the whole version control has to be set up new when connecting to github-repository?

Read here: https://nifi.apache.org/docs/nifi-registry-docs/html/administration-guide.html#switching-from-other-...

Thanks for any clarification!

3 REPLIES 3

Re: NiFi-Registry - GitHub connection - Not possible to take over already existing buckets/flows ?

Rising Star

Hi @justenji , have you got any solution for this?

I am also looking to migrate the existing bucket/flows of file system provider to GIT.

Re: NiFi-Registry - GitHub connection - Not possible to take over already existing buckets/flows ?

Community Manager

Hi @hegdemahendra, as this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post.


Regards,

Vidya Sargur,
Community Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:

Re: NiFi-Registry - GitHub connection - Not possible to take over already existing buckets/flows ?

Master Guru

@justenji and @hegdemahendra 

Answering this old question:


Tracking version controlled flows in NiFi-Registry involves two parts:

1. Metadata Database [1] - The metadata database maintains the knowledge of which buckets exist, which versioned items belong to which buckets, as well as the version history for each item.

2. Persistence Provider [2] - NiFi Registry uses a pluggable persistence provider to store the content of each versioned item. Each type of versioned item, such as a versioned flow or extension bundle, has its own persistence provider.

The GitFlowPersistenceProvider only hold the flow templates.  In NiFi-Registry versions 0.3.0 and older, there is not ability to construct a new Metadata Database from the contents stored in the GitFlowPersistenceProvider.  However, NIFIREG-209 introduced this ability in NiFi-Registry 0.4.0.  So of course not all the metadata from previous DB is going to exist in newly created DB because that data is not part of what is pushed to the persistence provider.  But it will create new metadata needed to support the versioned flows found in the GitFlowPersistenceProvider.  This only works if there is no exiting DB already.


Note: If starting a brand new NiFi Registry instance, the metadata database can be automatically rebuilt from the information in the GitFlowPersistenceProvider. This is a one-time operation during the first start of the application, and is not meant to keep the DB in sync with external changes made in Git. This feature only applies to flows and would not be able to restore information about extension bundles.


[1] https://nifi.apache.org/docs/nifi-registry-docs/html/administration-guide.html#metadata-database

[2] https://nifi.apache.org/docs/nifi-registry-docs/html/administration-guide.html#persistence-providers


If you found this addressed your query, please take a moment to login and click "Accept" on this solution.
Thank you,

Matt