Problem integrating Nifi Registry and github

@Angelvillar 

I think you have numerous unrelated questions here.

The "GitFlowPersistenceProvider" allows you to configure a git repo in which your version controlled process groups can be pushed for persistent storage outside of NiFi Registry server's file system. What is most important here is that NiFi only reads from the git repo on service startup.  While running everything is local to the NiFi-Registry server.  So if some changes are manually made on the got repo, the NiFi -Registry will not see them.  Additionally, the metadata about those stored versions flows is stored in the NiFi-Registry metadata-database and not in the got repo.  Also keep in mind that if you created flows originally using the local file based flow provider and then switched to git repo provider, those flows will not get moved to git.  Only new flows get created in git and old flows no longer are reachable.

1. Which Flow Persistence provider is configured for use in the NiFi-Registry has nothing to do with NiFi being able to connect and import flows.  NiFi connects to the  NiFi-Registry client URL configured in NiFi and gets a list of bucket flows to which the NiFi user has authorized access.  That flow information comes from the NiFi-Registry metadata DB.  So when you mad a change to the git repo, that would have had no affect until a NiFi-Registry restart.  What is in the new repo also would have had no affect on what is in the NiFi-Registry Metadata DB.  My guess here is that NiFi was give a list of version controlled flows known to the NiFi-Registry via the metadata DB and then when you tried to import one of them, NiFi-Registry could not find the actual flow locally.  Review the "Switching from other Flow Persistence Provider" section under the metadata-database section in the NiFi-Registry docs.  What changes did you make in configs when you cloned the git repo to tell NiFi to start using the new cloned repo over the original repo?

If you configured git repository has existing flows committed to it, if you have nothing in the metadata-database, NiFi-registry will generate metadata for the flows imported from the flow persistence provider on NiFi-Registry startup.

NiFi or NiFi-Registry being secured has nothing to do with the error you described.  If NiFi was able to display a list of flows for selection to import, then connectivity to Registry seems fine.  However, keep in mind that if you secure Registry, you must secure NiFi in order to write to any buckets.  A secured NiFi can access a non secured NiFi-Registry and a non secured NiFi can access a non secured NiFi-Registry.  It is also possible for a non secured NiFi to import flows from "public" buckets in a secured NiFi-registry.

2. It does not matter whether you run your NiFi-Registry on a VM or on Docker as long as the configured ports are reachable by your NiFi. This is all a matter of your personal preference.

3. Any Version controlled Process Group in NiFi will have a NiFi background thread that checks with NiFi-Registry to see if newer version of the PG are available.  If NiFi is unable to access the NiFi-registry buckets or the persisted Flows no longer exist in NiFi-registry, you can expect to see exceptions about not being able synchronize PG with NiFi-Registry.  Same would happen if you deleted the configured Registry client in the NiFi configuration and created a new Registry client pointing to same NiFi-Registry.  When a NiFi-Registry client is configured that client is assigned a UUID.  When a process group is version controlled, what is written to the local flow.xml.gz or flow.json.gz file is that UUID along with version controlled flow ID and version.  If you delete and re-create the NiFi-Registry client it will create a new unique UUID.  Your flows will not update to that new UUID, so those version controlled PGs will not be able to synchronize anymore as well.


Sounds like you have been making a lot of changes and it is not clear what state everything was in before you started making changes. I'd suggest starting fresh by stopping version control on all your current PGs that have been version controlled, getting your flow persistence provider working, version control your first PG, and restart both NiFi and NiFI-registry to make sure everything is still functioning as expected.  Then proceed to make one change at a time you want to try and repeat the restart to see what if anything breaks.  

If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post.

Thank you,

Matt